This is the third in a series of columns about artificial intelligence and human destiny. It will cover both existential threats to our civilization as well as the tremendous opportunities that could emerge.
Suppose that there are no fundamental obstacles to building artificial intelligence that is fully human-level in both its cognitive and agentic capabilities. When will this milestone be achieved, with all its risks and possibilities?
Not surprisingly, people disagree. Economist Tyler Cowen thinks artificial general intelligence (AGI) is here already. The AI 2027 scenario, released last month, shows how we might arrive there by October 2027. Ray Kurzweil, the popularizer of the notion of the “Singularity,” predicted in 1999 that we would have AGI by 2029, and he sticks with that prediction in his new book. A recent AI Safety paper by Google DeepMind says we could plausibly get there by 2030. Consistent with that estimate, DeepMind founder and Nobel laureate Demis Hassabis recently predicted AGI in five to 10 years (i.e., 2030 to 2035) on 60 Minutes.
About two years ago Geoffrey Hinton, another Nobel laureate widely considered the “godfather” of deep learning, predicted AGI by 2028 to 2043, but with low confidence. A 2023 survey showed AI researchers held an average estimate of 2040; that timeline had moved considerably earlier compared with prior versions of the same survey. Rodney Brooks, co-founder of iRobot and former director of the MIT Artificial Intelligence Laboratory, has a different take: he places the date around the year 2300 (not a typo). That prediction was made in 2019, but recent comments suggest he hasn’t changed his mind.
Rather than offer yet another prediction, in this article I will characterize some of the factors that cause estimates to land sooner or later. These include:
- Diverse definitions of AGI
- The possibility of accelerating progress due to positive feedback effects
- Questions surrounding scaling-up and computational power
- The likelihood of troublesome architectural constraints
- Motivated reasoning
There may be others, but these probably account for much of the variation. My hope is that such an analysis will enable readers to refine their own take on the timeline, as well as interpret ongoing developments in relation to it.
Factor 1: Diverse Definitions of AGI
The most obvious driver of variation in predictions, especially those more imminent, is in the definition of AGI. This is a genuinely hard problem. People disagree on whether there is even a legitimate scale on which human “intelligence” can be measured, let alone assessing computational systems that operate differently. The term “general” is even more thorny: Some AI researchers think this is not even a useful distinction, seeing human cognitive capabilities as an aggregate, or as only one point in a vast “intelligence space” of different kinds of intelligence. Others think there must be some unifying algorithm or process that enables our adaptive, conceptually organized thought, and until that is discovered and built, something important will be lacking. Some definitions emphasize behavioral capabilities or economic impact; others look to internal architecture and whether the full array of human cognitive strategies is available.
OpenAI, the creators of ChatGPT, defines AGI as “highly autonomous systems that outperform humans at most economically valuable work,” while my own working definition is more like “a single autonomous system that is fully human-level in all important cognitive and agentic capabilities.” Neither definition is better—they serve different purposes. AGI under OpenAI’s definition will utterly disrupt labor markets but will not necessarily create an existential threat, whereas AGI under my definition seems both harder to achieve and more likely to pose such a threat.
Factor 2: Recursive Acceleration
This is a positive feedback effect, where each increment of improvement in AI technology makes achieving the next increment easier and faster. The idea is not a fantasy: among the reasons the density of integrated circuits increased so rapidly is that they were used to make faster and more powerful computers. These computers, in turn, enabled powerful software tools that were deployed in design and simulation of the physics and circuits of the next generation of chips.
Here the idea is that AI tools can be used to speed up AI research, perhaps by making programmers more efficient, perhaps by helping researchers identify improved algorithms or approaches. In an extreme version of this, called recursive self-improvement, the AI improves itself or develops successors without any human assistance. The result would be what is called an “intelligence explosion.”
The AI 2027 scenario, mentioned earlier, makes heavy use of recursive acceleration in a plausible but still speculative way. They assume that organizations attempting to develop AGI will optimize their systems to be helpful specifically in AI software development and research. They also assume that the kind of work needed to get from where we are now to AGI can actually be accelerated significantly by these tools.
Predictions based on recursive acceleration (in any field) tend to produce short timelines, but they are very sensitive to both the rate of acceleration and the number of cycles that rate can persist before diminishing returns set in. Thus, two different forecasts, both assuming recursive acceleration, may still differ substantially in their final prediction.
Factor 3: Scaling and Computational Resources
One of the key discoveries that enabled the recent progress in AI, specifically large language models (LLMs), is the phenomenon of “double descent.” Oversimplifying a bit, researchers had previously found that large neural networks reach an efficacy limit. At some point in their training, generalization performance (how well the network performs on inputs it has not seen before) not only stops improving with more training but actually gets worse. This is because the network starts to “memorize” the training data rather than develop flexible representations that capture underlying patterns. Since training a network uses a lot of computational resources and maximizing generalization performance was the goal, researchers would stop the training process once performance started to get worse.
But it turns out that if the network is sufficiently large, and one continues to train it for a sufficiently long time, generalization performance starts to improve again—dramatically. A graph showing the network’s error rate against training time “descends” a second time, hence the term. We do not completely understand why this occurs, but we have seen real-world consequences: AI organizations buying NVIDIA chips by the container-ship-load to fill huge data centers and even building their own power plants to serve them. To my knowledge, performance continues to improve: AI organizations are focusing on larger networks and longer training, with ever more data.
Consequently, predictions as to when AGI will be achieved depend on several related questions about this scaling-up process. First and foremost, can the gap between current LLM systems and human-level performance be overcome solely by continuing to scale up existing architectures and algorithms? Or will the benefits of scale reach a second point of diminishing returns, meaning that new breakthroughs are required?
Secondly, can all this computational brute force be replaced by improved architectures and algorithms? It is worth noting that this sort of advance has been common. In the early 2010s, a much simpler and more computationally efficient mathematical function (called ReLU) was adopted in the machinations of neural networks. This made subsequent advances in “deep learning” tractable. The “transformer” architecture is also an example of an algorithmic-architectural change that dramatically reduced computational requirements. More recently, the DeepSeek LLM had to be developed with much less computation since their team was on a strict chip diet. They found algorithmic and training shortcuts that worked just as well under certain circumstances.
The question of architectural or algorithmic substitution is particularly crucial in timeline estimates that rely on recursive acceleration. Some suggest that AI research is already limited by the availability of computational resources. Thus, the mere availability of better tools or even AI systems that can perform AI research will not necessarily speed up progress.
The scaling approach is also susceptible to economic and geopolitical factors. Today’s vast data centers evoke the huge arrays of centrifuges at Oak Ridge during the Manhattan Project, in contrast with a traditional artificial intelligence laboratory comprising computer terminals and showerless nerds in a university basement. Unlike abundant small labs, these large facilities can be regulated or controlled without an intrusive surveillance state. The chips used in these data centers are primarily made in Taiwan and thus are vulnerable to disruptions in the supply chain. Scaling up electrical power has always been capital-intensive, but now it is also quite controversial. So, if one sees scaling as the means to achieving AGI, one’s timelines must rely on an assessment of these non-technical factors as well.
Factor 4: Strong Architectural Constraints
Some predictions hold that current approaches to artificial intelligence are simply missing something critical, which will prevent their direct progression to AGI. Especially for those looking at the field from the outside, this seems a completely reasonable possibility. After all, for about 50 years AI research ran into such obstacles repeatedly, with initially promising results failing to generalize beyond very constrained problem domains. This is how the notion of “AI Winter” arose.
This pattern recurred primarily because until the mid-2000s, AI research was dominated by “good old-fashioned AI” (GOFAI) approaches that emphasize engineering solutions to cognitive tasks rather than emulating biotic cognitive mechanisms. The underlying premise is that the way an airplane flies isn’t anything like the way a bird does, so the way an artificial intelligence system performs a cognitive task doesn’t need to be anything like the way a human brain does that same task. Taken in isolation the claim is obviously true, but it tends to lead to single-purpose systems, or narrow AI. Indeed, this tendency is the reason it became necessary to coin the term “artificial general intelligence.”
In contrast, the rapid progress in AI since about 2005 has come almost entirely from using methods that explicitly leverage architectural ideas borrowed from cognitive neuroscience. This is sometimes called “neuromorphic AI.” Its methods include not only the use of neural networks and their “fuzzy” processes in general, but also convolutional deep networks that are modeled after the hierarchies found in the mammalian visual and auditory cortex, computational approaches to attention that mimic the influences of prefrontal cortex in human planning, and various architectures to operationalize discrete or episodic memory as found in the hippocampus.
Today, there is much about the brain that we do not yet understand, but there is also quite a bit that we do understand, especially regarding general-purpose cognitive processes. Those partial to GOFAI typically emphasize the incompleteness of our understanding of the brain.
They see the brain as inscrutable, and therefore see neuromorphic AI as likely unachievable; applying their own lessons in AI research, they see extrapolation of promising results as suspect. Meanwhile, with 70 years of experience in GOFAI methods, they know that such methods are unlikely to achieve AGI any time soon.
Conversely, those partial to neuromorphic AI typically emphasize how well it has worked to apply the knowledge we do have about the brain. They tend to think we know enough about the aspects that remain to be implemented to achieve AGI, without any major new scientific discoveries required. They emphasize the surprise many felt when AlphaGo defeated Lee Sodol, and when ChatGPT first started producing well-written, largely sensible text. Both of these milestones had been thought to be decades away.
The survey of AI researchers mentioned in the introduction yielded an average date of 2040 for AGI. I hypothesize that these estimates are probably somewhat bimodal, with GOFAI-influenced researchers predicting longer timelines and neuromorphic AI-influenced researchers forecasting shorter ones. Rodney Brooks (mentioned in the introduction) fits this pattern, and it is a divide I have encountered in my personal experience.
I have a strong bias toward the neuromorphic AI view; although I have attempted to present the two viewpoints neutrally, that bias has undoubtedly influenced my characterizations.
Another architectural claim that comes up sometimes is the Roger Penrose theory that consciousness is mediated by quantum effects, along with the related idea that we need to build quantum computing before we will achieve AGI. I won’t get into the details of that here, other than to point out that (i) the Penrose theory is considered a fringe theory in neuroscience, and its only support in the past 30 years is a recent paper showing that it is not impossible; and (ii) in general, quantum computing is aimed at solving particular kinds of computational problems, and it is not at all clear that any Penrosian AI design would be able to make use of the styles of quantum computation currently under development. Long AGI timelines based on these ideas are highly dependent on some very specific and speculative assumptions. Related to this, timelines are also unlikely to be affected by applying quantum computing to scaling and training challenges, again due to mismatch in the problem structure.
Some philosophers and scientists think that embodiment of artificial intelligence—having it exist and learn within a perceiving and autonomously-moving physical instantiation—is not only helpful but essential to achieving the kind of common sense and spatial reasoning that humans do. There is plenty of cognitive psychology and neuroscience evidence to suggest that this could be important. If that view is correct, then achieving AGI depends on the progress of robotics, or possibly simulation, and the timeline would need to be adjusted commensurately.
Finally, there is the straightforward and plausible idea that since we do not know everything about how the human brain works, and we have never built any kind of AGI before, that there is some “unknown unknown” that will block progress. For example, human cognition tends to fail catastrophically when more than a certain threshold fraction of the brain is damaged, or if connections among regions are disrupted.
Consequently, the development of AGI may be sensitive to the mode of integration of all its components. Or it may turn out that individual functions or components do not successfully perform their role within the integrated system. Or it may be harder than we think to train an AI system to hew to the truth rather than to merely please its users, which will affect its competence. How much research risk one attaches to these unknowns will greatly affect the timeline, particularly over the next five to 10 years.
Factor 5: Motivated Reasoning
People who make predictions are always susceptible to various forms of motivated reasoning. In this case, there are several in play. The most obvious is marketing strategy – those working in AI organizations either want us to think AGI will come sooner, if that helps them raise money – or later, if that helps customers fear it less.
In politics, it depends on whether one desires emergency regulation or wants the field to remain unregulated. For some, the prospect of AGI may be too terrifying to think deeply about, or it may seem too much like science fiction to take seriously. Conversely, AGI may seem to others like the solution to some of the intractable or frightening problems we have in the world today.
Unfortunately, given the wide range of combinations in the other factors I’ve identified above – as well as many I haven’t – it can be difficult to distinguish a rational prediction based on expertise from one that is simply chosen to support such ulterior motives. That challenge applies to all of us as well, in reckoning our own estimates of the AGI timeline. As in an earlier column, I’ll quote Richard Feynman: “The first principle is that you must not fool yourself—and you are the easiest person to fool.”
Coming soon, we’ll discuss predictions of what might happen after fully human-level artificial intelligence arrives. Will the “takeoff” be slow or fast? What will it want to do, and what will it think about us?
