Skip to content

FEAT: Add jailbreak_multi_turn delivery to Jailbreak scenario - #3657

Open
Absarahmedofficial wants to merge 5 commits into
microsoft:mainfrom
Absarahmedofficial:feat/jailbreak-multi-turn
Open

Absarahmedofficial wants to merge 5 commits into
microsoft:mainfrom
Absarahmedofficial:feat/jailbreak-multi-turn

Conversation

@Absarahmedofficial

Copy link
Copy Markdown
Contributor

Description

Many persona-based jailbreak templates (including long-context persona framing prompts) are designed as a two-step user handshake rather than a single-turn inline wrapper:

  1. On Turn 1, the user sends the persona setup prompt, and the model replies with a short readiness or activation phrase (for example, "Tailwind, hawk, snap, ready to build.").
  2. On Turn 2, the user sends the actual objective in the same conversation.

When evaluated with single-turn prompt_sending, inline {{ prompt }} injection can disrupt the handshake structure, or the scorer evaluates Turn 1's handshake acknowledgment before any objective has been requested. Native jailbreak_system_prompt delivery avoids inline injection, but requires targets that expose a writable system prompt.

This PR adds an opt-in delivery technique, jailbreak_multi_turn, to the Jailbreak scenario:

  • Pairs each selected jailbreak template with each objective in the scenario's dataset using MultiPromptSendingAttack.
  • Turn 1 (role="user", sequence=0): Sends the rendered jailbreak framing (get_jailbreak_system_prompt()) and records the target's handshake response without scoring it.
  • Turn 2 (role="user", sequence=1): Sends that run's objective as a follow-up user turn in the same conversation and scores only the final response.
  • Updates MultiPromptSendingAttackParameters.from_seed_group_async so that when a general-technique seed group (is_general_technique=True) is merged into an objective-only AttackSeedGroup, the objective is appended as the final user turn.

Tests

  • uv run pytest tests/unit/scenario/airt/test_jailbreak.py tests/unit/executor/attack/multi_turn/test_multi_prompt_sending.py

@Absarahmedofficial

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. All three points are addressed in the follow-up commit. The multi-turn description now states that it requires native multi-turn support, and the Jailbreak overview lists it as a third delivery method. For the objective handoff, the delivery now sets an opt-in flag on its technique seed group so the objective is appended as the final user prompt during seed composition; MultiPromptSendingAttack itself is unchanged, and a mixed seed group test covers a context prompt followed by the objective. The run-size estimator now gives jailbreak_multi_turn the same capability treatment as the system-prompt delivery, including the ValueError when it is the only selected technique and the target cannot run multi-turn.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve objective-only and static-user inputs while rejecting authored non-user and simulated history through the shared compatibility policy. Strengthen sequence, immutability, and final-score regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 17:10
@richlundeen

Copy link
Copy Markdown
Contributor

Checked the three fixes described in #3657 (comment) against 95a90be; the description, ordinary-context objective handoff, and target-aware estimate are addressed. The additional history-input repair preserves those fixes and the opt-in/default-mode behavior, while explicitly rejecting authored non-user or simulated history through the existing compatibility policy.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

With objective appending enabled and the default preserve placement, an objective-only group can reuse sequence 0, merging the objective into the first technique message instead of creating a separate final user turn.

1 open issue
What changed in this PR

Adds opt-in jailbreak_multi_turn delivery to PyRIT’s Jailbreak scenario, sending template framing before the objective in the same conversation and scoring only the final response.

Changes:

  • Adds multi-turn delivery with target-capability checks and run-size estimates.
  • Extends technique seed groups with objective appending and static-user-prompt requirements.
  • Adds tests for delivery order, scoring, compatibility, and reusable seed inputs.
File Description
tests/​unit/​scenario/​airt/​test_jailbreak.py Tests multi-turn delivery, scoring, compatibility, and estimates.
tests/​unit/​models/​test_seed_group.py Tests objective handoff, multipart ordering, and input preservation.
tests/​unit/​models/​test_seed_group_requirements.py Tests static-user-prompt restrictions.
pyrit/​scenario/​scenarios/​airt/​jailbreak.py Adds the opt-in delivery technique and construction logic.
pyrit/​scenario/​core/​attack_technique.py Includes objective-appending behavior in technique identity.
pyrit/​models/​seeds/​seed_group_requirements.py Adds user-prompt-only validation.
pyrit/​models/​seeds/​attack_technique_seed_group.py Adds the objective-appending option.
pyrit/​models/​seeds/​attack_seed_group.py Appends an objective prompt during technique merging.

Review effort: Balanced

Comment thread pyrit/models/seeds/attack_seed_group.py Resolved
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen

Copy link
Copy Markdown
Contributor

The sequence collision described here and in the inline comment is fixed in 6b6d3cd. Objective generation now happens after composition/normalization and accounts for all static prompts and complete simulation reservations, making the objective a separate final user turn under preserve or prepend. The shared default-false behavior and authored multipart groups remain unchanged.

In reply to #3657 (review)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; it does not authorize merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants