DeepMind Presents a Framework for Measuring Progress Toward AGI
DeepMind has introduced a framework for measuring progress toward AGI and launched a Kaggle hackathon to develop relevant evaluations.
On March 17, 2026, Google DeepMind presented a new framework for measuring progress toward Artificial General Intelligence (AGI), along with the launch of a Kaggle hackathon to develop the necessary evaluations. The approach aims to bridge the gap between the current capabilities of AI systems and AGI goals, providing concrete tools to assess the cognitive abilities of AI models.
DeepMind's Cognitive Framework
The framework proposed by DeepMind is based on decades of research in psychology, neuroscience and cognitive science, identifying ten key cognitive abilities considered fundamental to general intelligence:
- Perception: extracting and processing sensory information from the environment.
- Generation: producing outputs such as text, speech and actions.
- Attention: focusing cognitive resources on what is relevant.
- Learning: acquiring new knowledge through experience and instruction.
- Memory: storing and retrieving information over time.
- Reasoning: drawing valid conclusions through logical inference.
- Metacognition: knowing and monitoring one's own cognitive processes.
- Executive functions: planning, inhibition and cognitive flexibility.
- Problem solving: finding effective solutions to specific problems.
- Social cognition: processing and interpreting social information and responding appropriately in social situations.
To assess these abilities, DeepMind proposes a three-step protocol:
- AI evaluation: testing AI systems on a broad set of cognitive tasks, using separate test sets to avoid data contamination.
- Human baselines: collecting reference data from a representative sample of adults on the same tasks.
- Performance comparison: mapping AI systems' performance against the distribution of human performance for each ability.
The Kaggle Hackathon: An Opportunity for the Community
To put this framework into practice, DeepMind teamed up with Kaggle to launch a hackathon titled "Measuring progress toward AGI: Cognitive abilities". The goal is to involve the community in designing evaluations for the five cognitive abilities with the largest evaluation gaps: learning, metacognition, attention, executive functions and social cognition.
Participants can use Kaggle's Community Benchmarks platform to develop and test their evaluations against a set of state-of-the-art models. The total prize pool is $200,000: $10,000 for each of the top two winners in each of the five categories and $25,000 for the top four overall. Registration is open from March 17 to April 16, with results expected on June 1. To take part, visit the Kaggle website.
Implications for AGI Research and Development
This framework represents a significant step in standardizing the evaluation of AI systems' cognitive abilities, providing a basis for comparing models, assessing risks and measuring progress toward AGI. Collaboration with the community through the Kaggle hackathon underlines the importance of a collective approach to defining and developing effective AGI metrics.
For researchers and developers, this framework offers guidance for identifying areas for improvement and steering future research directions, fostering more targeted and measurable progress in the field of artificial general intelligence.
Takeaways
- Standardizing evaluations: DeepMind's framework provides clear criteria for measuring the cognitive abilities of AI systems, facilitating comparisons and progress toward AGI.
- Community involvement: The Kaggle hackathon encourages active participation from researchers in designing evaluations, promoting collaborative innovation.
- Research direction: The ten identified cognitive abilities offer a map for directing research and development efforts toward key areas for AGI.
- Ethical implications: Standardizing evaluations can help identify and mitigate the risks associated with AGI development, promoting responsible progress.
- Tracking progress: The framework makes it possible to track improvements over time, providing a tangible measure of progress toward AGI.
For more details, see DeepMind's original article.