Repository navigation
FIX: Include the request template in the HTTPTarget identifier - #3765
Lanre Shittu (Shizoqua) wants to merge 17 commits into
Conversation
Roman Lutz (romanlutz)
left a comment
There was a problem hiding this comment.
The request-template fingerprint fixes the original body-template replay case, but two related identity paths still reuse results across different configurations. Please address the inline findings before merging.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the change that distinguishes HTTP targets by request template.
No open issues 🎉
What changed in this PR
Fixes #3751 by including the HTTP request template in HTTPTarget identity, preventing different judge configurations at the same endpoint from sharing target and scorer hashes.
Changes:
- Adds a truncated SHA-256 fingerprint of the request line, non-credential headers, and body, preserving body line endings.
- Excludes credential headers so credential rotation does not change identity.
- Adds regression tests for template differences, identical templates, and credential-independent identity.
Existing HTTPTarget identifier hashes will change once.
| File | Description |
|---|---|
tests/unit/prompt_target/target/test_http_target.py |
Tests template-sensitive identity and credential-header exclusion. |
pyrit/prompt_target/http_target/http_target.py |
Adds the request-template fingerprint to target identifiers. |
Review effort: Balanced
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Roman Lutz (romanlutz)
left a comment
There was a problem hiding this comment.
The two earlier findings are addressed: request-body and effective client-header changes now affect evaluation identity, and the corresponding durable replay/cache controls behave correctly. One remaining request-identity gap is described inline: behavioral URL/query settings are still omitted.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Roman Lutz (romanlutz)
left a comment
There was a problem hiding this comment.
The original ordinary path/query and client-params examples are fixed, including durable cache and replay checks. I found one remaining issue in the new path normalization, described inline: decoded paths can collapse distinct request targets.
| canonical = json.dumps( | ||
| { | ||
| "method": method, | ||
| "path": sent_url.path, |
There was a problem hiding this comment.
[P2] Preserve reserved path escapes in evaluation identity
httpx.URL.path is decoded, so it loses distinctions that the endpoint can still see. For example:
httpx.URL("https://example.com/policies/strict%2Flenient").path == (
httpx.URL("https://example.com/policies/strict/lenient").path
) # TrueHTTPX sends different raw paths for these URLs. %2F can be part of one policy/resource name, while / separates path segments, so the two requests need not select the same policy.
I checked this through the public scoring/cache path with a local transport that distinguishes the raw paths. After closing and reopening SQLite, the targets and scorers had equal evaluation hashes, and get_cached_results_for_technique_async returned the first path's saved success for the second path, even though a fresh call to the second path returned false. Their full hashes differ, so observation replay correctly rejects this pair; the remaining problem is evaluation-cache reuse.
Could we fingerprint the encoded path that HTTPX actually sends, separately from the already-filtered query, and add this reserved-escape cache regression? That would preserve cross-host equivalence without treating different request targets as equivalent.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate only frozen upstream a9f437a into the existing fix/http-target-request-identity PR branch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate only frozen upstream 0f5f658 into the existing fix/http-target-request-identity PR branch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Description
Fixes #3751.
HTTPTargetidentifiers only recorded the endpoint and a few settings, so two targets on the same URL with different requests got the same target and scorer hashes, and a judgment stored for one could be replayed through the other.The identifier now carries
http_request_fingerprint, a hash of the request as it is sent: method, path, query, body and headers, including headers andparamsfrom the client. The scheme, host and port are left out so the same request on another deployment keeps its evaluation identity. Credential headers and credential query parameters are left out so secrets never reach the identifier.The fingerprint is part of both the full hash and the evaluation hashes. It is a new
TargetIdentifierfield, so this adds a nullablehttp_request_fingerprintcolumn toTargetIdentifierswith a migration. ExistingHTTPTargethashes change once.Tests and Documentation
tests/unit/prompt_target/target/test_http_target.py: request changes (body, method, headers, path, query, client params) change the identity and eval hashes, another host keeps them, credentials are ignored, and the replay and cache paths no longer reuse results across policies.tests/unit/memory/test_http_request_fingerprint_migration.pyfor the migration.No docs or notebooks changed, so JupyText was not needed.