"Transducers" by Rich Hickey

Transducers are IMHO one of the most under-appreciated features of Clojure. If you want a really good intro, watch Rich Hickey's talk about transducers: ▶ this video (Rich is an excellent speaker and most of his talks are really worth watching, even if you don't use Clojure). The thing about transducers is that once you start using them, the abstractions click in your mind and you start using them more and more in your code. Many things our software does can be looked at as pipelines, and transducers let you write composable functions and put them together into pipelines. As you see more pipelines in your code, you will write those composable functions and reuse them in various places. Experienced Clojure programmers will rarely use loops, for me transducers are the #1 choice whenever I have to process a collection of any kind. Their efficiency (no intermediate collections!) is a nice bonus — you get both high-level composable code and efficiency, which is incredible. One thing I dislike about transducers is when I need a stateful one: these are hard to understand and very difficult to write. I rarely do this, except when I really want to stick to the transducer abstraction and I have a couple of hours to spend on a puzzle. But you rarely need to write your own stateful transducers, usually the built-in ones are enough (things like take/drop).

Quick nit/question. For the fold method, I don't think I've seen it called sentinel value. Usually it is seed or initial? I think in Scheme it is common to call it knil, mirroring how lists use nil as the "sentinel" value which marks the end of a proper list. I opted to name it sentinel in that article (and in the docs) for two reasons: 1. Sentinel values are a common topic in many languages link 2. Transducers necessarily abstract across a lot more than loops / lists. Lisps do a lot of really cool (and optimized) stuff with lists alone and Scheme is no different in this regard. However, because of how Scheme primarily exports list operations in (scheme base) is really easy to run into a situation where lists are used in an ad-hoc way where another data structure is more appropriate. This includes vectors, sets, mappings, hashmaps, etc. Transducers-the-library is meant to be general across operations that work on all of these types, so I chose language that intentionally departs from thinking in a list-focused way. Now, my main question. Transducer? I'm curious on the etymology of that word. By itself, I don't think I could ever guess what it was referencing. :( This is from Rich Hickey's presentation: ▶ this video It's not a reducer, because they serve as higher-order functions that operate on reducers. Instead, the values they accept are transient through the function(s), so they transduce. You should watch the video, I think Rich explains the origins of his language very well

Rich gave a talk on this, and there is an associated HN thread for the talk, but some of you may be interested in the original transducers presentation. You can view it here: ▶ this video

Transducers. See Richard Hickey's talk about transducers here: ▶ this video You can implement transducers in JavaScript using generators. It was somewhat mind bending but fun.

This reminds me of a talk by Rich Hickey [0], where he introduces the Transducer pattern, which is actually an abstraction for map/reduce/fold/filter etc. (But I'm not trying to invalidate your claim that patterns exist in FP in general, only that specific case. Afaik, the Transducer abstraction isn't even widely-known nor used.) [0]: ▶ this video

Here's another video by Rick Hickey about transducers: ▶ this video - it was referenced in this great post detailing the transducer notion with Clojure, Scala, and Haskell: link

This is interesting, but it's still missing some very important features of transducers. As you alluded, stateful operations [1] won't work at all in your Haskell model. Also, you've missed early termination [2], which Rich's talk [3] covered well. [1] link [2] link [3] ▶ this video

That and they're collection agnostic. I highly recommend watching Rich Hickey's talk for more background: ▶ this video

Rich brought this up in his Strange Loop talk [1]. All the attempts he had seen either disallowed perfectly valid uses or allowed invalid ones. I believe to really guarantee early termination and a few other tricky invariants of reducers you would need a linear type system. [2] [1] ▶ this video [2] link