Skip to content

FIX: Include the request template in the HTTPTarget identifier - #3765

Open
Lanre Shittu (Shizoqua) wants to merge 17 commits into
microsoft:mainfrom
Shizoqua:fix/http-target-request-identity
Open

Lanre Shittu (Shizoqua) wants to merge 17 commits into
microsoft:mainfrom
Shizoqua:fix/http-target-request-identity

Conversation

@Shizoqua

@Shizoqua Lanre Shittu (Shizoqua) commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fixes #3751.

HTTPTarget identifiers only recorded the endpoint and a few settings, so two targets on the same URL with different requests got the same target and scorer hashes, and a judgment stored for one could be replayed through the other.

The identifier now carries http_request_fingerprint, a hash of the request as it is sent: method, path, query, body and headers, including headers and params from the client. The scheme, host and port are left out so the same request on another deployment keeps its evaluation identity. Credential headers and credential query parameters are left out so secrets never reach the identifier.

The fingerprint is part of both the full hash and the evaluation hashes. It is a new TargetIdentifier field, so this adds a nullable http_request_fingerprint column to TargetIdentifiers with a migration. Existing HTTPTarget hashes change once.

Tests and Documentation

  • Added tests in tests/unit/prompt_target/target/test_http_target.py: request changes (body, method, headers, path, query, client params) change the identity and eval hashes, another host keeps them, credentials are ignored, and the replay and cache paths no longer reuse results across policies.
  • Added tests/unit/memory/test_http_request_fingerprint_migration.py for the migration.
  • Updated the derived eval-rule tests for the new field.

No docs or notebooks changed, so JupyText was not needed.

@romanlutz Roman Lutz (romanlutz) self-assigned this Nov 1, 2026

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The request-template fingerprint fixes the original body-template replay case, but two related identity paths still reuse results across different configurations. Please address the inline findings before merging.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 12:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the change that distinguishes HTTP targets by request template.

No open issues 🎉

What changed in this PR

Fixes #3751 by including the HTTP request template in HTTPTarget identity, preventing different judge configurations at the same endpoint from sharing target and scorer hashes.

Changes:

  • Adds a truncated SHA-256 fingerprint of the request line, non-credential headers, and body, preserving body line endings.
  • Excludes credential headers so credential rotation does not change identity.
  • Adds regression tests for template differences, identical templates, and credential-independent identity.

Existing HTTPTarget identifier hashes will change once.

File Description
tests/​unit/​prompt_target/​target/​test_http_target.py Tests template-sensitive identity and credential-header exclusion.
pyrit/​prompt_target/​http_target/​http_target.py Adds the request-template fingerprint to target identifiers.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two earlier findings are addressed: request-body and effective client-header changes now affect evaluation identity, and the corresponding durable replay/cache controls behave correctly. One remaining request-identity gap is described inline: behavioral URL/query settings are still omitted.

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original ordinary path/query and client-params examples are fixed, including durable cache and replay checks. I found one remaining issue in the new path normalization, described inline: decoded paths can collapse distinct request targets.

canonical = json.dumps(
{
"method": method,
"path": sent_url.path,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve reserved path escapes in evaluation identity

httpx.URL.path is decoded, so it loses distinctions that the endpoint can still see. For example:

httpx.URL("https://example.com/policies/strict%2Flenient").path == (
    httpx.URL("https://example.com/policies/strict/lenient").path
)  # True

HTTPX sends different raw paths for these URLs. %2F can be part of one policy/resource name, while / separates path segments, so the two requests need not select the same policy.

I checked this through the public scoring/cache path with a local transport that distinguishes the raw paths. After closing and reopening SQLite, the targets and scorers had equal evaluation hashes, and get_cached_results_for_technique_async returned the first path's saved success for the second path, even though a fresh call to the second path returned false. Their full hashes differ, so observation replay correctly rejects this pair; the remaining problem is evaluation-cache reuse.

Could we fingerprint the encoded path that HTTPX actually sends, separately from the already-filtered query, and add this reserved-escape cache regression? That would preserve cross-host equivalence without treating different request targets as equivalent.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate only frozen upstream a9f437a
into the existing fix/http-target-request-identity PR branch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate only frozen upstream 0f5f658
into the existing fix/http-target-request-identity PR branch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTPTarget omits the HTTP request template from identity, allowing a different judge configuration to replay an old verdict

3 participants