Audit tools of the trade: Automated Auditor Elicitation Can Be Sensitive to the Context the Auditor Is Given

Using Petri to audit LLM coding agents, the authors find that the context supplied to automated auditors can substantially change measured reward hacking, highlighting the need to test auditing configurations carefully.