Skip to content

Add the 4.9 Rust detection and API migration bridge - #178

Merged
sidmohan0 merged 5 commits into
devfrom
feature/4.9-core-migration
Sep 28, 2026
Merged

sidmohan0 merged 5 commits into
devfrom
feature/4.9-core-migration

Conversation

@sidmohan0

@sidmohan0 sidmohan0 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

DataFog needs an incremental path from its Python API to the shared Rust Core without silently changing current callers. This adds the 4.9 bridge on top of the frozen published 4.8.1 contract; PLAN-4.9.md records scope, ownership, and acceptance criteria.

  • Opt-in backend="rust" for regex scan/redact using the pinned datafog[rust] extra. Python remains the default; native failures propagate and unsupported German requests fail explicitly.
  • datafog.compat.v4 preserves legacy result classes and transformations, while lazy datafog.v5 exports the actual Core schema and APIs.
  • Revised detect/process retirement to 5.0, explicitly replacing the earlier 5.x promise. OCR/Spark remain functional in 4.9 with visible use-time notices and documented 5.0 removal.
  • Exact native parity checks, installed-wheel CI across Linux/macOS/Windows, and reproducible public-call benchmarks. The original 4.8.1 fixture is unchanged; the checker narrowly accounts for four additive backend parameters and revised warnings.

The native corpus has 61 exact matches, two individually reviewed detector differences (checksum-invalid card and embedded SSN), 17 unsupported German cases, and 31 cases outside backend scope. German detector implementation is specified separately and is not implemented in this Python PR. No package is published or final release version stamped.

Validation:

  • Base/CLI regression suite: 782 passed, 18 skipped, 19 existing xfails.
  • Focused native integration: 347 passed against published Core 0.3.1.
  • Base-only Python 3.10 and 3.14: 181 passed each, nine optional native/CLI skips.
  • Clean installed-wheel smoke checks passed with isolated imports.
  • Existing core/hook benchmarks and new public backend comparison completed; timings and methodology are documented.
  • All applicable pre-commit checks passed.
  • Added lazy-export/cache and facade-validation coverage; static-only TYPE_CHECKING imports are excluded from runtime coverage.
  • Updated the live Python guides and marked the bridge unreleased. Sphinx HTML build passed with existing static/autodoc warnings.

The live Mintlify site is sourced from DataFog/datafog-core/docs on main. Companion documentation PR: DataFog/datafog-core#24 (Mintlify validation, link checks, and eight executable migration examples passed).

@sidmohan0
sidmohan0 merged commit 660e3f5 into dev Sep 28, 2026
65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant