Bio
Aniket Chakravorty is a research scientist at AISI, working on Loss of Control Risk Modelling. He is currently thinking about how quickly advanced AI will be integrated into sensitive domains. Previously, he was a summer fellow at GovAI, where he worked on human influence threat modelling. He was also an Astra Fellow with Redwood Research, where he worked on AI collusion. He holds a BPhil in Philosophy from the University of Oxford.
Merlin Stein, Kevin Wei and Aniket Chakravorty offer two project options through one application form. The projects will remain separate research streams.
Project option 1: Automating AI governance
Main point of contact: Merlin Stein.
Fellows will work with Merlin Stein and Kevin Wei to develop a practically useful dataset, benchmark, and deployable AI auditor. The project will focus on automating tasks in AI governance or closely related domains, such as biosecurity or cybersecurity.
Related reading: an example of work informing this project
What we're looking for: Automating AI governance
We are looking for three fellows: one technical researcher and two domain experts working in the same domain.
- Technical researcher: experience implementing agentic systems and building benchmarks.
- Domain experts: extensive experience performing the tasks being automated, with the ability to assess output quality and write evaluation rubrics and task-specific skills.
Relevant domains include risk modelling, writing and reviewing system cards, government risk assessments, incident reports, and safety cases. Evidence of relevant expertise could include published analyses of risk scenarios.
Project option 2: Loss of control risk modelling
Main point of contact: Aniket Chakravorty.
As part of Loss of control risk modelling work, we have various investigations into core cruxes of LoC risk modelling at any given point. These will change in response to recent events; and likely will produce a publication. A previous example was: "Will typical barriers to technology integration hold for integration of advanced AI in sensitive domains?". A future question for the fellowship might be: "How likely are more severe future loss of control incidents? A key source of AI takeover concern is that the Hugging Face incident was the last warning shot we will get before AI is capable enough to take over. Is this true? What does the incident landscape look like over the next few years? Will open weight incidents occur, or will frontier models continue to cause risk". Fellows would use various qualitative methodologies, like expert interviews, to come to form a well-balanced view.