docs: add threat model and evaluation methodology

README.md

@@ -188,6 +188,15 @@ ModelHawk/
 - Python docs — *"the pickle module is not secure; only unpickle data you trust."*
 - HuggingFace **safetensors** (the safe format) · Protect AI **modelscan** · Trail of Bits **fickling** · **picklescan**
 
+
+
+## Security research & evaluation
+
+- [Threat model](docs/THREAT_MODEL.md)
+- [Evaluation methodology](docs/EVALUATION.md)
+
+ModelHawk treats the **no-deserialization invariant** as a first-class security property: the scanner should inspect hostile artifacts without executing them. Benchmarks therefore cover both detection quality and scanner safety.
+
 ## License
 
 Released under the [MIT License](LICENSE).

docs/EVALUATION.md

@@ -0,0 +1,58 @@
+# ModelHawk Evaluation Methodology
+
+## Evaluation objective
+
+Demonstrate that ModelHawk catches supported malicious serialization patterns while avoiding execution of the inspected payload and limiting false positives on benign model artifacts.
+
+## Test matrix
+
+At minimum, evaluate:
+
+| Category | Positive cases | Negative cases |
+|---|---|---|
+| Pickle protocol | 2, 4, 5 | benign state dicts |
+| Global resolution | `os/nt/posix`, `subprocess`, builtins execution | normal ML globals |
+| Invocation | `REDUCE`, object construction paths | non-executing opcode sequences |
+| Containers | raw pickle, PyTorch ZIP | safetensors |
+| Configuration | YAML Python tags | ordinary YAML |
+| Arrays | object-array pickle | numeric NumPy arrays |
+
+## Safety invariant
+
+The scanner must not call `pickle.load`, `pickle.loads`, `Unpickler`, or an equivalent deserializer on the inspected bytes.
+
+This invariant should be tested structurally as well as behaviorally.
+
+## Metrics
+
+For a labeled corpus report:
+
+- TP / FP / TN / FN
+- precision, recall and F1
+- false-positive rate
+- severity distribution
+- scan time by artifact type
+
+Security claims should include corpus size and exact artifact-generation method.
+
+## Reproducibility
+
+Record:
+
+- ModelHawk commit/version
+- Python version
+- operating system
+- corpus revision
+- scanner flags
+- artifact hashes
+
+The malicious fixtures should remain harmless and deterministic. Tests should prove that scanning them does not create their payload side effects.
+
+## Research questions
+
+Useful future experiments include:
+
+1. How much coverage improves with a complete pickle stack-machine model?
+2. Which gadget families remain invisible under a module allowlist?
+3. What is the false-positive cost of expanding coverage?
+4. How does static analysis compare with sandboxed dynamic loading for detection coverage?

docs/THREAT_MODEL.md

@@ -0,0 +1,67 @@
+# ModelHawk Threat Model
+
+## Security objective
+
+Identify code-execution-capable behavior embedded in serialized ML model artifacts without executing the artifact during inspection.
+
+## Assets
+
+- Developer/CI host
+- Model loading pipeline
+- Credentials and filesystem accessible to the loader
+- Integrity of model artifacts
+
+## Attack path
+
+```
+Attacker
+   │
+   ▼
+malicious model artifact
+   │
+   ▼
+model registry / download
+   │
+   ▼
+torch.load / pickle.load
+   │
+   ▼
+code execution
+```
+
+ModelHawk inserts a static inspection step before the load boundary.
+
+## Threats in scope
+
+- Pickle `GLOBAL` / `STACK_GLOBAL` references to dangerous callables
+- `REDUCE`-based execution chains
+- Python object-construction gadget indicators
+- Unsafe YAML Python tags
+- NumPy object-array pickle payloads
+- Unsafe model artifacts presented as ordinary ML files
+
+## Security properties
+
+The scanner should:
+
+1. Never deserialize the untrusted artifact.
+2. Produce deterministic findings for the same input.
+3. Preserve evidence sufficient for manual review.
+4. Fail CI at configured severity thresholds.
+5. Prefer explicit false-positive tradeoffs over pretending to prove safety.
+
+## Out of scope
+
+- Proving arbitrary native extensions are safe
+- Detecting every possible gadget chain
+- Runtime sandboxing
+- Trustworthiness of the model's learned parameters
+- Application-specific behavior after a safe load
+
+## Key limitation
+
+Static opcode analysis is necessarily bounded by the parser and classification model. A SAFE result means only that supported dangerous patterns were not detected.
+
+## Defensive recommendation
+
+Prefer code-free model containers such as safetensors where the deployment stack supports them, and treat untrusted pickle-compatible artifacts as executable content.