Visit AISI.gov.uk
Please enable javascript for this website.
A
A
‍

Our priority research areas

We are interested in funding projects to develop mitigations to safety and security risks from misaligned AI systems.

Our research goals

In-scope projects will aim to address either of the following challenges:

  1. How can we prevent AI systems from carrying out actions that pose risks to our collective security, even when they may attempt to carry out such actions?
  2. How can we design AI systems which do not attempt to carry out such actions in the first place? 

Making substantial breakthroughs in these areas is an interdisciplinary effort, requiring a diversity of tools and perspectives. We want the best and the brightest across many fields to contribute to alignment research, so have organised these priority research areas as a set of discipline-specific questions. We suggest clicking ahead to your specific areas of interest, rather than reading linearly. Sections are roughly ordered from most theoretical to most empirical. 

Some of the subfields below have more detail than others about subproblems, recent work, and related work. This should not be read as a signal about which areas we believe are more important: much of the variance is due to areas we or our collaborators have focused on to date. We want to bring other areas up to similar levels of detail, and will attempt to do this in future versions of this agenda.

We’re excited about projects that tackle these questions, even if they aren’t focused on a problem outlined below. Feel free to look at others’ lists and overviews — e.g. Google DeepMind, Anthropic, or Redwood Research — for ideas. If you see connections between your research and these challenges, we encourage you to submit a proposal.

Research areas broken down by discipline

1

Information Theory and Cryptography

Prove theoretical limits on what AI systems can hide, reveal or prove about their behaviour.
2

Computational Complexity Theory

Find Formal guarantees and impossibility results behind scalable oversight protocols.
3

Economic Theory and Game Theory

Find incentives and mechanisms to direct strategic AI agents to desirable equilibria.
4

Probabilistic Methods

Bayesian and rare-event techniques for tail-risk estimation, scientist-AI, and formal reasoning under uncertainty.
5

Learning Theory

Understand how training dynamics and inductive biases shape generalisation.
6

Evaluation and Guarantees in Reinforcement Learning

Stress-test AI agents and prove when they can’t game, sandbag or exploit rewards.
7

Cognitive Science

Map and mitigate the biases and limitations of human supervision.
8

Interpretability

Access internal mechanisms to spot deception.
9

Benchmark Design and Evaluation

Translate alignment's conceptual challenges into concrete, measurable tasks.
10

Methods for Post-training and Elicitation

Refining, probing and constraining model behaviour.
11

Empirical Investigations Into AI Monitoring and Red Teaming

Current alignment methods can't ensure AI systems act safely as they grow more capable, so the field of AI Control focuses on practical techniques—like restricting AI capabilities and using oversight models—to prevent catastrophic outcomes and test systems before they can cause harm.