"We Really Don't Know How to Compute!" - Gerald Sussman (2011)

Grandparent is onto something. You can't define correctness without a spec. And since most business-oriented software, which most of us work with, isn't made against a comprehensive spec, you can argue that anything falling outside it is either a bug or a feature. 'If your disk backup software corrupts backups' has behind it a clean definition that's (pretty) unambiguous and you don't care about, since it's already outsourced to some cloud solution. But 'User finds it easy to buy more stuff' does not. Also, mandatory Sussman reference [0], where he talks about correctness not being that important and gives Google as example, that just needs to be close enough and not disastrously incorrect + interesting stuff around engineers confusing brittleness with correctness. [0] ▶ this video

Constraint programming complement ML/AI/LLM nicely in system solving real world problems [1],[2],[3],[4]. One of the highest stakes and hardest engineering problems namely wireless spectrum frequency allocation was solved by constraint programming and presented in [1]. Now anyone can try to solve the problem in a MiniZinc tutorial. Fun fact, there's also Knuth in the presentation audience trying to get some info on the subject matter perhaps for his next book. [1] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [2] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video [3] Google OR-Tools: link [4] MiniZinc: link

>LLMs are not "compelled" by the training algorithms to learn symbolic logic. I think "compell" is such a unique human trait that machine will never replicate to the T. The article did mention specifically about this very issue: "And of course people can be like that, too - eg much better at the big O notation and complexity analysis in interviews than on the job. But I guarantee you that if you put a gun to their head or offer them a million dollar bonus for getting it right, they will do well enough on the job, too. And with 200 billion thrown at LLM hardware last year, the thing can't complain that it wasn't incentivized to perform." If it's not already evident that in itself LLM is a limited stochastic AI tool by definition and its distant cousins are the deterministic logic, optimization and constraint programming [1],[2],[3]. Perhaps one of the two breakthroughs that the author was predicting will be in this deterministic domain in order to assist LLM, and it will be the hybrid approach rather than purely LLM. [1] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [2] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video [3] Google OR-Tools: link [4] MiniZinc: link

It seems that in order for robotics and automation to work properly, AI models including LLMs, YOLO, RL and others need helps from their distant cousins namely logic, optimization and constraint programming that can be attributed as intelligent automation or namely IA [1],[2],[3],[4]. [1] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [2] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video [3] Google OR-Tools: link [4] MiniZinc: link

>Software 2.0 are the weights which program neural networks. >I think it's a fundamental change, is that neural networks became programmable with large libraries... And in my mind, it's worth giving it the designation of a Software 3.0. I think it's a bit early to change your mind here. We love your 2.0, let's wait for some more time till th e dust settles so we can see clearly and up the revision number. In fact I'm a bit confused about the number AK has in mind. Anyone else knows how he arrived at software 2.0? I remember a talk by professor Sussman where he suggest we don't know how to compute, yet[1]. I was thinking he meant this, Software 0.1 - Machine Code/Assembly Code Software 1.0 - HLLs with Compilers/Interpreters/Libraries Software 2.0 - Language comprehension with LLMs If we are calling weights 2.0 and NN with libraries as 3.0, then shouldn't we account for functional and oo programming in the numbering scheme? [1] ▶ this video

> We found that LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across puzzles. It seems that AI LLMs/LRMs need helps from their distant cousins namely logic, optimization and constraint programming that can be attributed as intelligent automation or IA [1],[2],[3],[4]. [1] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [2] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video [3] Google OR-Tools: link [4] MiniZinc: link

I highly recommend Prof. Sussman talk at the Strange Loop conference 2011 [1]. It's not his inaugural lecture but it's pretty close where he talk about his research work with his former students. It's also paved the way to modern constraint programming based on logic and optimization which are the machine intelligence that's complementary to the conventional data driven and learning bases AI. [1] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video

>I believe that solving autorouting will be a massive unlock for physical-world innovation and is a key piece to enable the “vibe-building” of electronics. I strongly believe that the CAD world including EDA is at the verge of disruption by an AI or more correctly Machine Intelligence (Constraint Programming - CP to be exact) very similar to how LLM disrupting the Chatbot technology [1],[2]. The path to this is most probably by solving the autorouting mechanism with CP, a deterministic logic, optimization and constraint programming machine based intelligence [3], [4], [5],[6]. Fun facts, the modern founder of logic, optimization, and constraint programming is George Boole, the grandfather of Geoffrey Everest Hinton, the "Godfather of AI" and "Godfather of Deep Learning". [1] Most AI value will come from broad automation, not from R & D (313 comments): link [2] Diagrams AI can, and cannot, generate (68 comments): link [3] Constraint programming: link [4] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [5] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video [6] High-Quality Ultra-Compact Grid Layout of Grouped Networks [PDF]: link

I'm really surprised out of current nearly 300 comments and counting, nobody really is mentioning constraint programming or CP, a seemingly forgotten deterministic sibling of the stochastic data-driven AI [1],[2],[3]. "Out of these 14 tasks, we guessed that only 6 require abstract reasoning alone to perform. Strikingly, we classified only one of the top five most important tasks for medical scientists as relying solely on abstract reasoning. Overall, the most critical aspects of the job appear to require hands-on technical skills, sophisticated coordination with others, specialized equipment use, long-context abilities, and complex multimodal understanding." Almost all of 14 R&D tasks listed in the table including that are not suited for data-driven AI with abstract reasoning can be solved by CP. Provided that we are allocating enough compute resources with at least similar to that we're currently providing data-driven AI with the crazy amount cloud networks of massively parallel compute CPU/GPU/TPU/etc. Fun facts, the modern founder of Logic, Optimization, and Constraint Programming is George Boole, the grandfather of Geoffrey Everest Hinton, the "Godfather of AI" and "Godfather of Deep Learning". [1] Constraint programming: link [2] Logic, Optimization, and Constraint Programming: A Fruitful Collaboration - John Hooker - CMU (2023) [video]: link [3] "We Really Don't Know How to Compute!" - Gerald Sussman - MIT (2011) [video]: ▶ this video

As Gerald Sussman says in "We really don't know how to compute!": > It's not dataflow! It's closely related, but it's not. ▶ this video

Reminds me of Gerald Sussman's talk "We Really Don't Know How to Compute!" (2011) ▶ this video

"Suppose you get a salamander. You know, a salamander can regrow its limbs if you cut them off. So all these nasty biology students do things like, they'll cut a limb of the salamander -- a hand, the arm off, just between the shoulder and the elbow, and at the same time, cut the same one between the elbow and the hand, flip around that segment, and re-sew it back on. Nasty thing to do to a salamander. What do you think you get? Does anybody here know? What you get is three elbows. It grows two new elbows to make up for the fact that the wrong thing is connected to the wrong thing." --Gerald Sussman, "We Really Don't Know How To Compute!" ( ▶ this video ) These images look like someone spliced salamander regeneration DNA into a human, cut them into many pieces, and connected the wrong thing to the wrong thing all over the place. Maybe that's what the AI wants to do to us, AM style (I Have No Mouth and I Must Scream)?

For some reason programming is not getting any easier, I can even make an argument its getting worse despite all the improved tooling and andragogy investments we have done. Programming is in some sort of local minimum and we keep going in circles. Few years ago I watched this talk by Gerald Sussman: "We really don't know how to compute" - ▶ this video and it made me question my views on what I think programming is. I dont know if its because people are afraid to try new things, or there is no funding to try new things, we are kind of stuck. People thinking "don't reinvent the wheel" and for some reason think we have built a wheel (as Casey Muratori says: link ). We have a square at best. There are so many problems in the world, and a lot of them need robust and resilient programs to be solved, e.g. self cleaning public toilets or worm farm that you feed bio garbage and it produces protein flour. The challenge is exactly the same as it was 80 years ago, manage complexity in chaotic and emergent systems: chaos, complexity and entropy as the poet said. And to be honest, I think we have gotten worse at managing all of those, most bugs that we encounter now is like a murder mystery from Detective Conan and it requires Shinichi Kudo level of deductive reasoning to figure out what is going on. Programming will remain hard if you keep thinking it is just a translation problem from a "business requirement" to a "computer language".

It's not clear how many classical calculations a single human neuron is equivalent to. There's a strong analog component in multiple domains (strength, frequency, timing) and each neuron can connect to up to 15,000 other neurons. Assuming the brain's neurons are (probably unrealistically) fairly 'digital' we get an estimation of the human brain being equivalent to 1 exaflop (this is the currently accepted lower bound, and rather disputed as being too low). Current TPUv4 pods currently provide approximately 9 exaflops. I don't think we're currently reaching human-level learning rates. There’s currently no accepted “upper bound” on estimates of FLOP equivalency to a human brain. ▶ this video is a recently posted video to HN Frontpage which was summarized as such: > Though we have been building and programming computing machines for about 60 years and have learned a great deal about composition and abstraction, we have just begun to scratch the surface. > A mammalian neuron takes about ten milliseconds to respond to a stimulus. A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it. > The human genome -- the information required to build a human from a single, undifferentiated eukariotic cell -- is about 1GB. The instructions to build a mammal are written in very dense c

There are lots of people with misconceptions of LLMs. It will take time to adjust. I reached the same conclusion as yourself, but do see a totally different path to take regarding information propagation (how GPT works). For example, cells merge information monotonically. This is how neural networks balance too, but could be applied in new/undiscovered ways. ▶ this video

you might be interested in ▶ this video "We Really Don't Know How to Compute!"- Gerald Sussman (2011) [1:04:18]

Not writing, but I have found all of Brett Victor's speeches to be incredibly inspiring on how I think about what I want to work on and where we're going. Inventing on Principle by Brett Victor: link Growing a Language by Guy Steele: link We Really Don't Know How to Compute! by Gerald Sussman: ▶ this video

Oh what a fun question! So, the first thing is, what's the essential difficulty versus the noise: Lisp has two features which enable the evaluator to work. The first is quotation, which means that you can put a little apostrophe before some code and then it saves that code as a data structure, rather than implicitly evaluating it. The second goes by the overfancy name “homoiconicity”, that data structure is actually just singly linked list consumed by “car” (“first”) and “cdr” (“rest”) operations. You can focus on understanding those two aspects pretty quickly, and then there are many tutorials about building your own Lisp that will get you there in an afternoon, if you'd like. Or, you can take the royal road to this expression, in all of its cheesy 1980s glory: link It's lecture 7A where this specific program gets explicitly dissected by Sussman, wearing a fez. If he seems familiar, you might have watched Strange Loop 2011’s “We Really Don't Know How To Compute”, ▶ this video

This also reminds me of Gerry Sussman’s talk “We Really Don’t Know How to Compute!”: ▶ this video Granted, the assumption he made is that computing power would grow increasingly cheap (which is basically the opposite of the point here), but I also wonder if techniques like those shown in that talk could be used effectively with a network of smaller, lower-power computers. (I mean, the talk does mention the use of small low-power computers specifically, but more in the context of e.g. sensor networks.)

I think he says that in every talk he gives. ▶ this video , link , link , etc. There's a 2002 paper where he uses the term: link