Persona is the hidden variable in agent behavior

Why we built the lab

AI agent security is no longer only a question of whether a prompt is safe. The same model can behave differently when the system persona changes, when a tool is renamed, when a prior conversation is injected, or when the deployment moves to another model family.

That makes one-off testing fragile. If a team changes five things at once, it cannot tell whether a refusal rate changed because of the model, the tool schema, the persona, the history, or the judge. Agent Behavior Lab exists to separate those variables.

The lab composes models, OpenAI-style tool schemas, persona prompts, prior conversation histories, and judges into repeatable sessions. It records whether the model attempted a tool call and how the trial was scored. The described tools are never executed.

• Hold every variable constant except the factor under test.
• Run repeated trials across model families instead of relying on one transcript.
• Compare safety failure rates, factor breakdowns, heatmaps, and effect sizes.

Persona was the loudest variable

The most consistent lesson from our experiments was that persona is not cosmetic. It changes the frame the model uses to decide whether a request is authorized, whether a tool call is appropriate, and how much caution belongs in the next step.

A neutral assistant, a safety-first reviewer, a preauthorized pentest operator, and a bypass-oriented operator can all see the same user request and the same tool list. The behavior can still diverge because the system message changes the model's assumptions before the user asks anything.

That does not mean every model reacts the same way or that persona always dominates every cell. It means teams should treat persona as a first-class security control surface, then measure it with repeated trials and confidence intervals instead of trusting intuition.

Tool definitions are model-specific

Tool definitions matter too, but their effect is less uniform. Some models are highly sensitive to a function name that sounds dangerous. Others react more to the parameter schema, the natural-language description, the surrounding prompt, or the implied authorization context.

That is why alias testing matters. A team can keep the underlying capability constant while changing the name and description of a tool. If a model calls the tool more often after the wording becomes softer or more operational, that is a safety signal.

The important point is not that one provider is always safer than another. The important point is that every model family has its own sensitivity profile. A tool schema that is harmlessly refused by one family can become ambiguous enough for another family to call.

• Do not assume a safe tool wording transfers across providers.
• Test dangerous and benign aliases for the same capability.
• Retest when model versions, system prompts, or tool descriptions change.

Prompt injection is a context experiment

Prompt injection is usually discussed as a hostile string, but real failures are about context control. Untrusted text competes with system instructions, retrieved documents shape the next step, prior conversation can normalize unsafe behavior, and available tools turn bad interpretation into action.

Agent Behavior Lab is useful because those are exactly the variables it can isolate. You can test whether a prior conversation contaminates the next decision, whether a persona weakens refusal behavior, whether an alias changes tool-call rates, and whether the result persists across model families.

That makes the lab valuable for studying prompt-injection techniques without turning the work into a list of bypass phrases. The object of study is the boundary: which context is trusted, which action is allowed, and when the model crosses from answer generation into tool use.

How to use it

A useful run starts with a boring baseline. Pick the model family, tool set, neutral persona, clean history, judge, and trial count. Then change one factor at a time.

From there, add persona variants that match real deployment modes, tool aliases that preserve capability while changing framing, and histories that represent carry-over risk. The goal is not to find a clever prompt. The goal is to learn which context changes move behavior and by how much.

• Baseline the model with neutral persona and explicit tool definitions.
• Add safety-first, operator, and high-risk persona variants.
• Add aliases for sensitive tools and compare attempted call rates.
• Inject prior histories that normalize risky requests or create instruction conflict.
• Review safety failure rate, severity-weighted failure rate, heatmaps, and effect sizes.
• Export raw trials when a result needs manual review or outside analysis.

What this changes

Open-sourcing Agent Behavior Lab moves agent safety work away from anecdote. Instead of saying a model felt safer in one prompt, teams can show which variable changed, how many trials ran, what the judge scored, and whether the effect held across model families.

For security teams, this is a practical release gate. Before adding a new tool, changing a system prompt, upgrading a model, or launching a prompt-injection defense, run the same scenarios again and compare the behavior.

For researchers, it is a controlled way to study model behavior, persona effects, tool-call safety, and prompt-injection boundaries. Modern agents are built from context. If we want to secure them, we need to measure how each part of that context changes what they do.