The pursuit of Artificial General Intelligence (AGI) is one of the most ambitious goals in modern artificial intelligence research. Achieving AGI—where machines not only perform tasks but understand, learn, and adapt at near-human levels—would represent a monumental shift in technology and society. However, such a feat requires not just incremental progress, but a comprehensive understanding of intelligence itself.
At the forefront of this endeavor is Rich Sutton, a leading figure in reinforcement learning, whose work has shaped the foundational understanding of how machines can learn and make decisions. Alongside Michael Bowling and Patrick Pilarski, Sutton has outlined a roadmap to AGI known as the Alberta Plan. A key component of this plan is the Common Model of the Intelligent Agent, which serves as a blueprint for creating more sophisticated, adaptable AI agents capable of navigating the complexities of the real world. In this post, we will explore the significance of this common model and how it fits into the broader quest for AGI.
The AGI Challenge: A Grand Scientific Prize
AGI is not just a technical achievement; it is a grand scientific challenge, comparable to the breakthroughs of Einstein, Darwin, or Newton. Sutton and his colleagues emphasize that AGI will transform industries, reshape economies, and redefine our understanding of intelligence itself. However, unlike specialized AI systems that excel at narrow tasks, AGI must generalize across different domains, adapting to new environments and learning on the fly—just as humans do.
Sutton estimates that there is a 25% chance of reaching AGI by 2030, thanks in part to the exponential growth of computing power. As Moore’s Law continues to drive down the cost of computation, the pressure to develop scalable, generalizable AI systems grows. But creating AGI is more than just a matter of raw computing power; it requires a deep understanding of how intelligence operates over time and in the face of uncertainty.
The Common Model of the Intelligent Agent: A Base for AGI
At the heart of the Alberta Plan is the Common Model of the Intelligent Agent, also referred to as the base agent. This model represents a universal framework for intelligent decision-making, one that can be applied across a variety of fields, from psychology to economics to AI. The base agent simplifies intelligence into four key components:
- Perception: This is how the agent processes and interprets sensory information, translating raw inputs into an abstract state that can guide decision-making. Essentially, perception is the agent’s representation of past experiences.
- Reactive Policies: These are the actions that the agent takes in response to its current state. A reactive policy might prioritize maximizing rewards, but it could also be designed to optimize for other objectives, such as minimizing risk or balancing long-term and short-term goals.
- Value Functions: These functions evaluate how well the agent is doing based on its current state and action. In reinforcement learning, value functions guide the optimization of reactive policies by attaching expected future rewards to specific actions and states.
- Transition Model: This component predicts the future state of the environment based on the agent’s current state and action. The transition model helps the agent understand the consequences of its actions and adjust its strategies accordingly.
What makes this model so powerful is that it is general. It does not depend on the specifics of any one domain—whether that’s robotics, financial markets, or healthcare. Instead, it provides a flexible structure for decision-making that can be adapted to any environment.
Enriching the Base Agent: Towards Real-World Complexity
While the base agent offers a simplified model of decision-making, Sutton and his team have introduced key enrichments to make it more applicable to real-world challenges:
- Multiple Policies and Value Functions: Rather than a single policy mapping states to actions, the Alberta Plan allows for multiple policies to coexist. This mirrors the complexities of real life, where an individual or agent might have to balance competing goals or adapt strategies based on new information.
- Options in the Transition Model: The traditional transition model predicts the next state based on an action, but Sutton introduces the concept of options—actions with termination conditions. This reflects real-world decision-making, where actions are often contingent on uncertain outcomes, much like decisions in finance or long-term planning.
These enrichments allow the agent to simulate and plan for a wider variety of scenarios, continually learning from its environment while evaluating multiple strategies in parallel.
Signal Processing Over Time: The Foundation of Intelligence
One of the most important ideas in the Alberta Plan is that intelligence is signal processing over time. The base agent is constantly interacting with a complex world, receiving input signals (observations) and sending output signals (actions). To function effectively, the agent must learn to predict and control these signals in a way that maximizes its long-term success.
This concept leads to two critical challenges for AGI: continual learning and meta-learning.
- Continual Learning: Like humans, an intelligent agent must continually integrate new information without needing to pause or reset. This ability to learn on the fly is essential for operating in a dynamic world where circumstances change constantly.
- Meta-Learning: In addition to learning from experience, the agent must also learn how to learn better. Meta-learning involves the development of strategies that help the agent adapt more efficiently to new situations, building on past experiences to tackle novel challenges.
The Alberta Plan emphasizes that these two types of learning are naturally linked. As the agent gathers more experience in a particular environment, it should be able to refine its strategies and learn faster, ultimately becoming more adept at solving a broader range of problems.
The Big World Problem: Why AGI Needs Robust Adaptability
One of the toughest hurdles in AGI research is the Big World Problem—the real world is infinitely more complex than any one agent can fully comprehend. Just like humans, AGI will only be able to form approximations of the world, and those approximations will need to be constantly updated.
Moreover, the world doesn’t stand still. The environment is always changing, and what works today may not work tomorrow. The Alberta Plan seeks to address this by focusing on agents that can make reasonable approximations of their environment and adapt quickly to new challenges. This adaptability is key to building an intelligence that can thrive in uncertainty, just as humans do.
The Roadmap to AGI: A 12-Step Plan
The Alberta Plan outlines 12 steps to create agents that can eventually achieve AGI, beginning with basic tasks like representation learning and progressing towards more sophisticated behaviors like intelligence amplification. Some of the key stages include:
- Representation Learning: The first steps focus on building agents that can continually learn from a fixed set of features and gradually introduce new features.
- Generalized Value Functions (GVFs): Prediction is crucial in reinforcement learning, and agents will learn to predict not just rewards but also more complex aspects of their environment using GVFs.
- Planning and Control: As the agent becomes more capable, it will be introduced to model-based planning, where it can simulate different scenarios and evaluate multiple strategies in parallel.
- Intelligence Amplification: The final stages envision agents that work collaboratively with humans, enhancing human decision-making and capabilities in a symbiotic relationship.
Conclusion: The Path to AGI Starts with the Common Model
The Common Model of the Intelligent Agent is more than just a theoretical construct; it is a practical foundation for building systems that can learn, adapt, and evolve in complex environments. As the Alberta Plan progresses through its twelve steps, it will refine this model and bring AI closer to the goal of AGI—machines that not only solve problems but also define and tackle their own challenges, much like humans do.
By focusing on scalable methods, continual learning, and adaptability, Sutton and his colleagues are laying the groundwork for the next generation of AI. And while the road to AGI may still be long, the Alberta Plan provides a clear path forward, built on a deep understanding of the fundamental principles of intelligence.
The Embodied AI 
Leave a comment