Future Computers Will Be Radically Different (Analog Computing)

This paradigm for computing was already covered three years ago by Veratasium in ▶ this video . Maybe not the specific photonic system that they are describing. Which I'm sure has some significant improvements over what existed then. But the idea of using analog approximations of existing neural net AI models, to allow us to run AI models far more cheaply, with far less energy. Whether or not this system is the one that wins out, I'm very sure that AI run on an analog system will have a very important role to play in the future. It will allow technologies like guiding autonomous robots with AI models running on hardware inside of the robot.

I think they're saying analog computers could be the future of computing. Veritasium explains it really well in general here (and demos the device) ▶ this video

Veritasium has a good video about analog computers. It covers how they work, what they are used for, pros and cons compared to digital, what they look like today, a bit of the history and their possible future. ▶ this video

A couple years ago Veritasium did a video on analog computers, which included a segment on Mythic AI that uses NAND flash cells kind of "undervolted" as an analog computer to run neural networks. ▶ this video link

The Veritasium Youtube channel did a video about this about a year ago: ▶ this video They visit Texas company Mythic AI to discuss how they use flash memory for machine learning. There's a California company named Syntiant doing something similar.

analog chips like what MythicAI is developing seem like the next obvious leap forward for deploying inferences broadly. ASIC/FPGA wouldn't be much different than a GPU. ASIC seems like a brittle solution ▶ this video

Actually, I have a Pixel 7 with the Tensor G2. I can't find the tera operations per second on it, but Google has said that its [1] 60% more powerful than their Tensor Coral (4 TOPS at 2W of power). That puts the G2 at about 6.4 TOPS. The Mythic analog chip [2] is pumping 25 TOPS at 3W. For comparison, modern (digital) GPUs are doing 25-100 TOPS at 50W-100W...while costing >$1000. So, its a big jump. [1] link [2] ▶ this video

link is an analog chip for doing neural nets. ▶ this video (this is a time stamp'ed link for a video mentioned elsewhere in the comments).

That's really neat. Veritasium did a couple of videos on analog computing a year ago. I'm glad there are people lowering the barrier to entry. link ▶ this video

The company (Mythic AI) in the article was featured on a Veritasium video[1] about analog computing. The interesting part for analog is obviously going to be wear and repeatability, because the digital layer auto-corrects the errors until it becomes damaged and unusable clearly, while analog will slide off somewhat gracefully but you need a way to detect it happening multiple layers later. AI is somewhat tolerant of errors since, the cumulative error across a lot of biases can approach zero if the errors are random. [1] - ▶ this video

I found this Veritasium video interesting, about a company that is paring down models so they fit on analog chips, so the chips can be used in self-contained devices. It also appears that quantum computers may be even better for matrix math that transformers rely on. If all these hardware advances pan out... ▶ this video link

Fire control computers, like on a navy ship, were faster than digital computers of the day. YouTube has a number of videos on them. An opamp performs multiplication faster than a digital computer (speed of light vs a few cycles). It's not super useful on its own, but it does fit the criteria. In Veritasium's video 2/2 on analog computers [0] they show some startup products near the end. [0] ▶ this video

Considering evolution optimized for biological fitness and not intelligence, it would be incredibly surprising if it wasn't possible to do better, especially using a vastly different architecture. You could also build an AGI to run on a digital CPU but interact with a simulation that used analog coprocessors (which is a thing these days ▶ this video ). You can also include quantum coprocessors (which already exist in their infancy) for various things (probably only useful for quantum simulations at this point though). Also considering recent ML work, an AGI is more likely going to run on something similar to a GPU than a CPU.

It's a big area of research in AI. Veritasium has two great videos on it: ▶ this video link I found a hackernews discussion for the second one: link

Great Veritasium video on YouTube about analog neuromorphic architectures and their promise for lower-power operations: ▶ this video

Warning: Bit of a 'Hitch hikers guide to the galaxy"/conway game of life yak shave for a reply. Nice start to providing a simple way to visually interact/experiment/manipulate different multiple Neural network concepts by loading/transforming any traditional code (working or not) & reshuffling/tesselating the structual representation of the code to generate a turing tape path via color execution paths. Switching to pi calculus could provide the theory to support coroutine/coprossing/ threading" multiple interacting tape paths/color combinations/path diffs/scaled recusion to perform assembly language instructions. Raster graphics without the yak ( link ) Feed back between tape & "traditional code" as neural network layer(s)! Flexiable code (via tessilations) without changing the actual imported source code! So, complete svg (tape interactions) web page as code source on how to interact with web page, ( link ) what's displayed/played, documentation & usage manual included within code structure! Unicode permits implied piping rich kern font without all the tech fluff. Unicode fails because allows infinate multics and is there for k&r incomplete. But, this approach is way more dimensional/colorful than unicode/tech stuff. ?? can one's code unintentially provide a wordle problem & solution in one package ?? ( link ) ?? turing complete fourier transforms link toolkit concepts: Neural networks / analog stuff : ▶ this video perceptrons: ▶ this video spreadsheets (: link (turing sprea

link This is a rather intuitive mathematical proof that neural nets can emulate all functions in many dimensions. Knowing integrals (Calc 102) would help. With this information it’s either that self-driving is eventually possible or human brains are not mathematically representable (i.e. something akin to a soul). Meanwhile, the data collection/storage is getting better (all Teslas are continually testing FSD and reporting failure cases, synthetic generation of “good and unique” perfectly labeled data[0] is only a year or two “old”). Sensors/cameras are improving (and have a lot more room to improve). The compute hardware is getting better and cheaper (analog computers[1], TSMC 2nm and beyond, Apple/Google/Teslas neural cores, etc). Additionally, given Waymo is already in production without humans in PHX it’s arguable that we already have “autonomous vehicles”. We now just need them to improve slightly and for Waymo or similar to test in the top 25 cities in the US. Which is mostly a matter of capital and time not technology limits. Maybe weather will be an issue for a few more years/decades - but even if Waymo only operated in Summer, I’d be happy to reduce my Lyft costs - and with enough data and maybe better sensors the neural nets will be able to handle weather too. Especially if the car just goes slow. [0] link [1] ▶ this video [2] link

basicly, we can do things like matrix operations and differntial equations using specialized hardware that performs computation over analog signals rather than buidlign discrete systems. the end result is that we dont' need anywhere near the level of compute to do the job as its done using an analog of the physical process. here's two youtube videos that explain is much better than I could. link ▶ this video suffice to say, your average house fly exhibits behavior that absolutely dominates our best drones and with a fraction of the circuitry.

Agreed. I've never really gotten into neural networks, most of the times I start but get distracted or bored/lost and suddenly enough a Vertiasium video pops up and about everything just slots into place, stuff you've glossed over or heard before just make sense. ▶ this video . Start at minute 4:00.

▶ this video Start here to hear the deplorable and utterly uninspiring uses the guy comes up with for analogue chips, then rewind to the start. The tech is great, but the (presumably most compelling) use cases this guy can come up with are automated surveillance and efficiencies in junk food production.