"Unison: a new distributed programming language" by Paul Chiusano

▶ this video This video is a little old but I remember finding it really interesting and I think dives into it a bit.

Looking at the comments it seems that this talk might be a better introduction than just skimming at the landing page. ▶ this video I watched that in 2020 or so and got amazed at how much a seemingly simple idea could work it's way into the core of a language. What I recall is this, The main new idea is turning codebases into a content-addressed store, this needs some new tooling, but makes a couple of interesting things possible, * Having different versions of the "same" function. They are actually different functions anyway, so you can do things like partial library upgrades when there's breaking changes. Large migrations like python 2 to 3 wouldn't take O(10yr). * Formatting is not a thing anymore. This goes further than just having a formatter as there's really no format nor names you can't personalize locally without affecting anybody. * Similarly, renaming is free, so you can come up with better names in the future and chanting it just needs a tweak to the canonical name for the function. * Functions are serialiazable, which makes it possible to send them to remote nodes for execution without any concerns about their deus, since a function is now more of a Merkle tree, so if there's any difference the client would know and be able to ask for missing dependencies * The tool to manage your function repo might seem like a burden to some, but once you realize that having this as a first class tool that understands the language you can propose safe mass refactoring of code you do

I first encountered Algebraic Effects in Unison where they're called "abilities" [0] via the strangeloop talk from 2 years ago [1]. Just from the little I've seen of it I feel like AE is a fundamental abstraction tool that's been missing in programming language design. "Fundamental" as in the same level as function arguments . So many problems that were solved with myriad complex programming language constructs are just absorbed as a trivial user-implementation with an effect system: exceptions, sync vs async, dependency injection, cancellation tokens, dynamic contexts... all of these problems where the essential complexity is a need to have an effect that cuts through the function call stack. I'm not saying that all our problems are solved and the programming world will now be rainbows and butterflies, I'm just saying that this feature is the correct framing and abstraction for issues we've run into many times in the past, and it has the potential to greatly simplify and unify the hacky, bespoke, situational solutions we've found. [0]: link [1]: ▶ this video

This seems like exactly the sort of issue the Unison language is trying to tackle. They want straightforward execution of distributed computations using the same language, with no friction across machine boundaries, no sacrificing composibility. The article mentions: > remote procedure calls with higher-order functional arguments would require serializing those functions, which is not easy to do safely or efficiently. This is exactly what Unison is designed to do. Code is not stored as text, but is stored in a serialized AST form and identified by hash. Therefore, transmission between nodes is trivial. The language is still a work in progress, but it's making rapid strides. Here's an explanation: ▶ this video The part relevant to distributed computation is at 30:08, but the earlier parts go through the basics of how code is stored, and why this design choice was made, which might help with understanding what's going on.

I had a really hard time wrapping my mind around this just reading the website alone. If you are in the same boat, watch the first 10 minutes of this video at 1.5x speed: ▶ this video and it will make so, so much more sense. ...and if you are like me you'll probably need to read this twitter thread to get the answer to your #1 question: link Basically the core idea (or one of the core ideas) is instead of a function (like fib(n) which returns nth Fibonacci number) being identified by its name (fib) as is the case with most traditional languages, it's instead identified by a hash of its implementation.

Not necessarily the same thing, but have you looked into the Unison project at all? Unison is an open source functional programming language based on a simple idea with big implications: code is content-addressed and immutable. Homepage: link Strange Loop talk: ▶ this video

I was just racking my memory and searching through my library of interesting links to find exactly this! Paul Chiusano gave a nice introductory talk at strangeloop last year: ▶ this video

> Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file? This is something unison is playing with with their codebase manager. ▶ this video

I recently came across Unison through YouTube recommending me a series of videos from the "Strange Loop" channel [1]. The fundamental idea of uniquely addressing functions based on a hash of their AST is mind-blowing to me. Immediately my mind started to consider many of the possible paths such an idea could lead down, many of which are clearly tickling the minds of many of the commenters in this thread. My first thought was the same insight from von Neumann architectures: code is data. So I thought of package repositories with URLs corresponding to hashes of function definitions. link could represent a specific function. A process/server could spin up completely dumb other than bootstrapped with the language and an ability to download necessary functions. You could seed it with a hash url pointer to the root algorithm to employ and a hash url to the data to run against and it could download all of it's dependencies then run. I imagine once such a repo was up and running you could do something like a co-routine like call and that entire process just happens in the background either on a separate thread or even scheduled to an entirely different machine in a cluster. You could then memoize that process such that the result of running repo-hash-url against data-hash-url is cached. e.g. I have link run against link and store the result at link 1. ▶ this video

There's an example in this talk: ▶ this video Basically you can quote code (like in Lisp) and pass it to an executor (thread, remote server, ...). The main difference is that thanks to the immutability, you can also have a protocol which shares efficiently all the dependencies required to execute the code (like a git pull). It also uses an effect system, so the implementation does not need to decide which executor to use, it only uses an interface called an "ability", and the caller can choose the implementation.