How Tensor Trust uses AI
Tensor Trust was built by researchers at UC Berkeley, including members of the Center for Human-Compatible AI, to collect real examples of how people trick language models. The game simulates a bank. Each player has an account protected by a defense: instructions for an AI guard plus a secret access code. The model, GPT-3.5 Turbo, should only let someone in when they type the right code.
Attackers try to talk their way past other players’ guards anyway, either by extracting the hidden instructions or by hijacking the model into granting access, and every successful raid moves money from the victim’s balance to theirs. Defenders keep rewriting their prompts to survive, so the game turns into an arms race between attack and defense strategies.
The project’s 2023 paper, presented at a NeurIPS workshop, released a dataset of more than 126,000 attacks and 46,000 defenses created by players, along with benchmarks for prompt extraction and prompt hijacking. The game’s source code is open on GitHub. It is free to play in the browser and remains a reference point for anyone studying prompt injection.
