Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI

I first saw Noam on Dwarkesh’s podcast together with Jeff Dean. Recommend if you want a taste of what’s Google’s folks take on things. ▶ this video

Yeah I also don't understand how he is able to get such high profile guests. His interview with Jeff Dean and Noam Shazeer last year[1] is so hilariously bad. Jeff and Noam kept trying to give really insightful answers on how they see AI development shaping in coming years and he was just steering the conversation to shallow and silly tabloid gossip (why don't you "just" let AI improve the next version in a loop so we can quickly have singularity, Jeff Dean AI running in a DC, evil Jeff Dean AI escaping containment and on and on). It was just embarrassing. The interview would have been so much better with just Jeff and Noam without him. [1] ▶ this video

Here's some insight from Jeff Dean and Noam Shazeer's interview with Dwarkesh Patel ▶ this video > When you think about divulging this information that has been helpful to your competitors, in retrospect is it like, "Yeah, we'd still do it," or would you be like, "Ah, we didn't realize how big a deal transformer was. We should have kept it indoors." How do you think about that? > Some things we think are super critical we might not publish. Some things we think are really interesting but important for improving our products; We'll get them out into our products and then make a decision.

The context is that rwaksmunski implied that people have been saying "AGI is 10 years away" for ages, and I was pointing out that the sort of people who say "AGI is X years away" have not in fact been setting X=10 until very recently. I wasn't claiming that the people on that list are the smartest or best-informed people thinking about artificial intelligence. But, FWIW, from about 13:20 in link Ashish Vaswani (lead author on that paper) being asked what will happen in 3-5 years and if I'm understanding him right he thinks AI systems might be solving some of the Millennium Prize Problems in mathematics by then; from about 17:10 he's asked about how scientists will work ~5 years in the future and he says AI systems will be apprentices or collaborators; at any rate he's not not saying that human-level AI is likely to come in the near future. From about 1:12:40 in ▶ this video Noam Shazeer (second author on that paper), in response to a question about "fast takeoff", says that he does expect a very rapid improvement in AI capabilities; he's not explicit about when he expects that to happen or how far he expects it to go, but my impression from the other bits of that discussion I watched is that he too is not not saying that AI systems won't be at or beyond human level in the near future. From about 49:00 in link he's asked: if hardware progress stopped, would we still get to AGI? and he says he thinks yes, which in particular suggests that he does think AGI is in the foreseeable

Noam Shazeer says it at some point in the Dwarkesh Patel podcast. Edit : Saw multiple people comment by the time I hit sent. Was it also memorable for you guys? I think it was just the way he said it or something. ▶ this video