Sparks of AGI: early experiments with GPT-4

Smart people have been falling into this trap as long as LLMs have hit production. Supposedly early internal versions of GPT-4 had "sparks of AGI" but the public version was "dumbed down for safety" ▶ this video

It goes back to Sparks of AGI [0] unless I am mistaken. Can recommend the talk, one that has stayed in the back of my mind since I first saw it two years ago. Personally, still have major reservations about throwing claims of intelligence or understanding around, but I do agree that SVG code generation can be a very effective source to get a quick and easy to present understanding of a models ability to output code with a rather open ended prompt that needs a high degree of coherence and were a lot of layers depend/build on each other. Helps that these are eye catching (literally as the output is visual) and easy to grasp. Same reason a lot of hype is created around the web desktops. [0] ▶ this video

Disagree. We know it _can_ learn out of distribution capabilities based on similarities to other distributions. Like the TikZ Unicorn[1] (which was not in training data anywhere) or my code (which has variable names and methods/ideas probably not seen 1:1 in training). IMO this out of distribution learning is all we need to scale to AGI. Sure there are still issues, it doesn't always know which distribution to pick from. Neither do we, hence car crashes. [1]: link or on YT ▶ this video

It's in this presentation ▶ this video The most significant part I took away is that when safety "alignment" was done the ability plummeted. So that really makes me wonder how much better these models would be if they weren't lobotomized to prevent them from saying bad words.

You might think that, but there is a great deal of compression and it seems that compression is understanding, giving rise to emergent properties ▶ this video

> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] ▶ this video

And there's an army of people that dismiss concerns, discussion, and evidence presented on this topic regardless of what is provided. On subjective topics people can always dispute examples. Some people abuse the courtesy of those that provide such examples, you've further proven that here. Regardless of if we should benchmark imagery with something that was claimed to be multimodal, can you genuinely not see the difference here? link Maybe your internal prompt is primed to disagree regardless of what is presented? Edit: ▶ this video even mentions what you claim not to see. Safety degrades the model.

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: ▶ this video [1]: link

You should take a look at this video. He is a researcher at Microsoft and had accès to private version of ChatGPT. He literally claims that ChatGPT 4 is not as good as before. His talk actually demonstrates the different evolutions. ▶ this video

▶ this video about 5 minutes in. > People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. [...] We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't think about it as pattern matching and just trying to predict the next word, yes it was trained to predict the next word but what emerged out of this is a lot more than just a statistical pattern matching object. We need to think about it as learning algorithms. [..] it's something very different from what we are used to. - Sebastien Bubeck, Sr. Principal Research Manager in the Machine Learning Foundations group at Microsoft Research

Here's the full talk[0] from a Microsoft lead researcher that worked with the early / uncensored version of GPT-4. Simplified, tuning it for censorship heavily limits the dimensionality the model can move in to find an answer which means worse results in general for some reason. [0]: ▶ this video

Highly highly highly recommend listen to this lecture by Sebastien Bubeck (the dude who wrote the paper), it's extremely interesting: ▶ this video

You might find this video[1] on why "GPT is smarter than you think" interesting. They're older, but the "Sparks of AGI" paper[2] and video[3] are also well worth your time. [1] - link [2] - link [3] - ▶ this video

The relevant comment from the second video is here: ▶ this video He says that GPT-4 started out being much better than GPT-3.5 at this task (draw a unicorn in TikZ), as the model trained longer it got to the point of drawing a great unicorn, but as it was tuned for "safety" it regressed to the point of being no better than GPT-3 was. Quite sad, in a way. It's not totally clear why there should be a tradeoff between "safety" and drawing skill / intelligence in this way. But it's interesting to speculate by analogy to humans.

Here are two. First one is Nathan Labenz who ended up Red Teaming for OpenAI during the six months after the base model was fully trained but before it was released (especially relevant parts are after 45 minutes): link Other one is Sebastien Bubeck who put Sydney into Bing before she was tamed down some notches; he explains the lobotomization process in a particularly poignant way, where they had the GPT keep trying to draw a unicorn even as it was progressively deadened: ▶ this video

I got the 1T GPT-4 number from here - this is the video that goes with the Microsoft "Sparks of AGI" paper, by a Microsoft researcher that had early access to GPT-4 as part of their relationship with OpenAI. ▶ this video

It is not hypocritical: whatever "Open"AI does to make GPT-4 "safer", makes it dumber. Specific example is drawing a unicorn ▶ this video

I used to say this as well, I’ve written blog posts and made videos saying exactly this. But - I would strongly recommend trying GPT4. There’s a very good video that is worth watching as well that may not change your mind, but will certainly make you think: ▶ this video The thing to remember is that these models are unbelievably huge and deep. We don’t know what is really happening in the layers or what it has really learned. Thinking that it’s a simple language model that is just predicting the next most likely word is unwise.

I just watched a video where the guy touches on GPT-4 limitations and one of those is simple math. He asks it some order of operations question and it outputs the correct answer but only if it does it step by step. It then apologizes and says its original incorrect answer was “a typo.” Video: ▶ this video

If you (or anyone else is) interested in the topic, I'd highly recommend giving some of these a look: Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” arXiv, March 27, 2023. link . or watching Sebastien Bubeck's recent talk he gave describing what GPT-4 can do that previous LLMs couldn't: ▶ this video Geoffrey Hinton recently gave a very interesting interview and he specifically wanted to address the "auto-complete" topic: link Here's another way that Ilya Sutskever recently described it (comparing GPT 4 to 3): link I'd also recommend this recent Sam Bowman article that does a goood job reviewing some of the surprising recent developments/properties of the current crop of LLMs that's pretty fascinating: Bowman, Samuel R. “Eight Things to Know about Large Language Models.” arXiv, April 2, 2023. link .