From 9f48e9e416b5310ce6b0bcfa0eb7adb62f368727 Mon Sep 17 00:00:00 2001 From: Praising Harris Ratnam Date: Wed, 7 Oct 2026 22:35:30 +0530 Subject: [PATCH 1/3] docs: add security methodology and research framing --- docs/THREAT_MODEL.md | 67 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 docs/THREAT_MODEL.md diff --git a/docs/THREAT_MODEL.md b/docs/THREAT_MODEL.md new file mode 100644 index 0000000..13fe691 --- /dev/null +++ b/docs/THREAT_MODEL.md @@ -0,0 +1,67 @@ +# ModelHawk Threat Model + +## Security objective + +Identify code-execution-capable behavior embedded in serialized ML model artifacts without executing the artifact during inspection. + +## Assets + +- Developer/CI host +- Model loading pipeline +- Credentials and filesystem accessible to the loader +- Integrity of model artifacts + +## Attack path + +``` +Attacker + │ + ▼ +malicious model artifact + │ + ▼ +model registry / download + │ + ▼ +torch.load / pickle.load + │ + ▼ +code execution +``` + +ModelHawk inserts a static inspection step before the load boundary. + +## Threats in scope + +- Pickle `GLOBAL` / `STACK_GLOBAL` references to dangerous callables +- `REDUCE`-based execution chains +- Python object-construction gadget indicators +- Unsafe YAML Python tags +- NumPy object-array pickle payloads +- Unsafe model artifacts presented as ordinary ML files + +## Security properties + +The scanner should: + +1. Never deserialize the untrusted artifact. +2. Produce deterministic findings for the same input. +3. Preserve evidence sufficient for manual review. +4. Fail CI at configured severity thresholds. +5. Prefer explicit false-positive tradeoffs over pretending to prove safety. + +## Out of scope + +- Proving arbitrary native extensions are safe +- Detecting every possible gadget chain +- Runtime sandboxing +- Trustworthiness of the model's learned parameters +- Application-specific behavior after a safe load + +## Key limitation + +Static opcode analysis is necessarily bounded by the parser and classification model. A SAFE result means only that supported dangerous patterns were not detected. + +## Defensive recommendation + +Prefer code-free model containers such as safetensors where the deployment stack supports them, and treat untrusted pickle-compatible artifacts as executable content. From 80fe2841ab309924a8739e14349507b805c8a021 Mon Sep 17 00:00:00 2001 From: Praising Harris Ratnam Date: Wed, 7 Oct 2026 22:35:33 +0530 Subject: [PATCH 2/3] docs: add security methodology and research framing --- docs/EVALUATION.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 docs/EVALUATION.md diff --git a/docs/EVALUATION.md b/docs/EVALUATION.md new file mode 100644 index 0000000..95ac951 --- /dev/null +++ b/docs/EVALUATION.md @@ -0,0 +1,58 @@ +# ModelHawk Evaluation Methodology + +## Evaluation objective + +Demonstrate that ModelHawk catches supported malicious serialization patterns while avoiding execution of the inspected payload and limiting false positives on benign model artifacts. + +## Test matrix + +At minimum, evaluate: + +| Category | Positive cases | Negative cases | +|---|---|---| +| Pickle protocol | 2, 4, 5 | benign state dicts | +| Global resolution | `os/nt/posix`, `subprocess`, builtins execution | normal ML globals | +| Invocation | `REDUCE`, object construction paths | non-executing opcode sequences | +| Containers | raw pickle, PyTorch ZIP | safetensors | +| Configuration | YAML Python tags | ordinary YAML | +| Arrays | object-array pickle | numeric NumPy arrays | + +## Safety invariant + +The scanner must not call `pickle.load`, `pickle.loads`, `Unpickler`, or an equivalent deserializer on the inspected bytes. + +This invariant should be tested structurally as well as behaviorally. + +## Metrics + +For a labeled corpus report: + +- TP / FP / TN / FN +- precision, recall and F1 +- false-positive rate +- severity distribution +- scan time by artifact type + +Security claims should include corpus size and exact artifact-generation method. + +## Reproducibility + +Record: + +- ModelHawk commit/version +- Python version +- operating system +- corpus revision +- scanner flags +- artifact hashes + +The malicious fixtures should remain harmless and deterministic. Tests should prove that scanning them does not create their payload side effects. + +## Research questions + +Useful future experiments include: + +1. How much coverage improves with a complete pickle stack-machine model? +2. Which gadget families remain invisible under a module allowlist? +3. What is the false-positive cost of expanding coverage? +4. How does static analysis compare with sandboxed dynamic loading for detection coverage? From f88a8d16e865fea8ec14b0a4dc0cdcc55cb789f2 Mon Sep 17 00:00:00 2001 From: Praising Harris Ratnam Date: Wed, 7 Oct 2026 22:36:03 +0530 Subject: [PATCH 3/3] docs: surface security research methodology --- README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/README.md b/README.md index 980e920..ddbb786 100644 --- a/README.md +++ b/README.md @@ -188,6 +188,15 @@ ModelHawk/ - Python docs — *"the pickle module is not secure; only unpickle data you trust."* - HuggingFace **safetensors** (the safe format) · Protect AI **modelscan** · Trail of Bits **fickling** · **picklescan** + + +## Security research & evaluation + +- [Threat model](docs/THREAT_MODEL.md) +- [Evaluation methodology](docs/EVALUATION.md) + +ModelHawk treats the **no-deserialization invariant** as a first-class security property: the scanner should inspect hostile artifacts without executing them. Benchmarks therefore cover both detection quality and scanner safety. + ## License Released under the [MIT License](LICENSE).