Spec-ulation Keynote - Rich Hickey

Reminds me of the Rich Hickey talk Speculation[1]. There is a special place in hell reserved for programmers that break back compat (for non-security impacting reasons) with widely used libraries, including google's guava developers. Linus Torvalds seems to be the only engineer with his head on straight on this topic and he has to constantly dive in and berate people that are trying to violate it in his project. [1] ▶ this video

This is all excellent, and I'll add one more sub point to "Avoid becoming a bottleneck at all costs": - Be backwards compatible. If you need to break backwards compatibility, advertise it widely and put your people towards the migration. Otherwise, only expand the abilities of components, never remove them and only require less, never require more. If you want to remove from what you give or ask for more than you currently require then make a new version of the component (e. g. `components/Button/v2`). Plan for this (e. g. don't ship `components/Button`, ship `components/Button/v1`). Rich Hickey's Spec-ulation talk goes into the principles in some more detail: ▶ this video The reason for this is because your goal is to empower other teams to move faster. The minute that "keeping up to date with the UI library" requires any significant amount of attention you are draining any velocity you added. People will stop (either by not updating or by moving off the library).

How? SemVer. Semantic Versioning is permission to break things. As Rich Hickey would say, just use a new name [1]. [1] Rich Hickey - Spec-ulation: ▶ this video

Rich Hickey also came to mind, but a different talk[0]. There he talks in part about software stability and how spec (the primary way to encode data and function contracts in Clojure), is about communicating what is required and provided by a function or library (among other things). Rust's decision to make signatures mostly explicit (sans lifetime parameters here and there), is self-documenting and useful, but most importantly it creates an explicit contract that people can rely on. Rust code adheres to semantic versioning so at least it will also adhere to not breaking callers without bumping the major version. Which the further discussion in the talk do not agree with. But the fundamental decision to make signatures explicit at least enables people to write stable software. [0] ▶ this video

Seems like this person may have been inspired by Rich Hickey's talk ▶ this video Regarding "nothing stopping us from making this versioning system completely automated" it seems like that depends on whether your language's type system supports that, and whether programmers follow the rules. For example, if you're relying on varargs/kwargs too much, it's going to be difficult to tell before runtime whether you've broken something.

The two are not mutually exclusive. Clojure has namespaced keywords and specs[0] to cover that. (There is also the third-party malli, which takes a slightly different appproach.) The advantage is that maps are extensible. So, you can have middleware that e.g. checks authentication and authorization, adds keys to the map, that later code can check it directly. Namespacing guarantees nobody stomps on anyone else's feet. Spec/malli and friends tell you what to expect at those keys. You can sort of do the same thing in some other programming languages, but generally you're missing one of 1) typechecking 2) namespacing 3) convenience. [0]: spec-ulation keynote from a few years ago does a good job explaining the tradeoffs; ▶ this video

To anyone who's interested in this topic, I strongly recommend this talk from Rich Hickey about "giving something to somebody that they can use and making a commitment ": Spec-ulation ( ▶ this video )

Rich Hickey explains a similar thinking in this talk ▶ this video but it’s more directed at code libraries. The same arguments apply. I 100% think this is the way to go for internal APIs. For public APIs I may change my opinion here for more sociological/political reasons. The rational/technical reasons all still remain.

For an interesting take on versioning, see the Spec-ulation keynote by Rich Hickey [1]. It specifically touches upon SemVer being itself versioned. [1]: ▶ this video

I don't find that too surprising, Rich tends to let things bake internally (in his head and at Cognitect) for a while before releasing anything publicly for consumption and feedback. Most of Clojure's current development is around tooling [1] and some changes to move clojure.spec out of alpha. I believe they want to accomplish this for 1.10, but I'm not sure where I remember hearing that. Alex Miller has mentioned that Cognitect built and continues to work on a tool that tracks all of your functions with spec definitions and generatively tests them only when the function or one of its dependencies changes. This would be extremely helpful since generative testing can be expensive to run, especially if you want to generate huge amounts of test cases. It'd be nice to say "test all of my codebase's functions 10 million times" and then have that information stored locally with the code. In Rich's "Effective Programs" talk [2] he outlines the "10x problems" that Clojure was targeted at solving. In it there are two 10x problems that weren't solved by Clojure: resource utilization and libraries. There's not much Clojure can do to solve resource utilization as it's dependent on the host (JVM), but the library problem (especially Clojure libraries with spec definitions) is solvable (though there seems to be some disagreement on whether only allowing growth in libraries in a practical). Pure speculation: All of their tooling projects (spec, codec, tool.deps, and the above mentioned gene

A very interesting talk by Rich Hikey (creator of Clojure) on the topic: ▶ this video To summarize, any change that offers more (as in, more fields in the output) and/or requires less (more of the original parameters made optional) is non-breaking. Edit: Of course, this is all under the assumption that the semantics of methods should never change. Instead of changing semantics, he proposes to just create a new method with a new name.

But it just delays the problem and introduces new ones... Patch changes, don't care, minor changes, don't care, major changes, you're screwed [maybe]. ▶ this video ( link )

I wasn't aware of greenkeeper, thanks. Can you give some examples of "Semver-based package managers"? I'm curious what you think of the two posts I was basing mine on: "Spec-ulation" by Rich Hickey: ▶ this video "Volatile Software" by Steve Losh: link I think there may partly be a "universes colliding" effect here, and partly just the future being non-uniformly distributed.

> I have a hard time being too impressed because more modern languages are doing this out of the gates The problem with Java is that it's an already established and very popular platform and it's been so for the past decade at least. When starting from scratch, it's easy to just throw it all away and start fresh, it's easy to fix mistakes that were made in the past. The irony though is that we still have a hard time learning from history. Just look at Go. Sometimes this industry feels like Groundhog Day, the movie. Also, no platform or language that I know of has gotten " modules " right, with Java being one of the platforms that has gotten closest to fixing the problem actually (by means of OSGi). To see why modules are still a big problem and why we won't have a fix for the foreseeable future, I invite you to watch this keynote by Rich Hickey: ▶ this video

Whenever an idea about API versioning springs into mind it is always worthwhile to watch Rich Hickey's Spec-ulation talk, ▶ this video for an alternative view of the world.

See this talk [0] about this addition to Clojure called core.spec [1]. I read the paper first, and it isn't until the end of the presentation that I understood how it was related to the feature at all. Basically, core.spec is a kind of formal verification tool designed to deal with the growth of software. It is not a type system, though it resembles one in some ways. The objective is to support comparisons between the signatures of a function at two different points and say, are these compatible? Is this a breaking change? You have to design for growth: make no assumptions about unspecified keys and always assume that later versions may return more than they used to. And so on. If you're a fan of semver, be warned. [0] ▶ this video link [1] link

> Ultimately, the talk tears into version numbers in general, but not semantic versioning He does talk explicitly about the trouble of making major changes in SemVer[0]. The gist of his argument was that minor changes in semver are relatively useless while major changes have a high probability of breaking your software. Major changes in semver are backwards incompatible and update the program's API in place. This leads to dowstream breakage and fear around doing upgrades. > If I need to upgrade my application due to a security vuln, SemVer lets me know if I can just "drop-in" the upgrade, or if I need to work more. I think the point he was trying to make was that upstream developers could change internals of a library but keep the API consistent so that downstream devs would never have to worry about scary updates. As you said with SemVer, if the security upgrade is a significant change, then you can expect breakage in the library. What he was advocating was patching issues like security vulns under the hood while keeping everything backwards compatible. Major upgrades could even add new namespaces, functions, and arguments but there's no real point to deleting or mutating old code, that just creates breakage. He wants software libraries to be immutable to take care of dependency issues and versioning to better reflect the changes made in the code. [0] ▶ this video ▶ this video ▶ this video

This seems inspired and perhaps even duplicated from what Rich Hickey recently said about Semantic Versioning (found here: ▶ this video ) I really enjoy how the package-manager for the Elm programming language automates semantic versioning by diffing your apis based on the type system. I think a bit part of the real criticism of SEMVER stems from this inability to either automate it or use it correctly. Hickey's discussion of the subject obviously has a lot more great points to it, and even when I don't agree, I can't help to enjoy listening :)

This talk [1] by Rich Hickey seems more pertinent than ever. It's a contrarian stance on semver (which, almost as much as vc itself, is lauded as "the Right way" among developers). He definitely challenged my beliefs, it's likely to be worth your time too. ▶ this video

I'm curious to hear how folks how adopt GraphQL handle server side schema changes over time. I'm all for the idea of not breaking clients almost ever [1] but sometimes you need to coordinate between client and server logic; how do you do that if the dependency of any client can literally be your entire db schema? Perhaps there is a subset of the server side schema that is designed to be GraphQL-able and is kept very stable over time? [1] ▶ this video