Skip to content

fix(rules): keep last downloaded detection rules when an update fails - #182

Open
kphillipsjr1101 wants to merge 5 commits into
CyberDrain:devfrom
VirTechSystems:fix/keep-last-downloaded-rules
Open

kphillipsjr1101 wants to merge 5 commits into
CyberDrain:devfrom
VirTechSystems:fix/keep-last-downloaded-rules

Conversation

@kphillipsjr1101

Copy link
Copy Markdown

Closes #176

Problem
When a rules update fails (GitHub outage, rate limiting, network problems), Check falls back to the bundled rules, which can be several releases behind and cause false alerts. Two paths caused this:

On startup, a cached copy older than the update interval was discarded. MV3 service workers restart often, so an expired cache plus a failed fetch meant bundled rules.

A failed background refresh replaced the in-memory downloaded rules with the bundled ones. The bundled app rules, do not match the latest remote default ruleset, often triggering false positives when Domain Squatting is enabled in the extension if a remote ruleset fails to load.

Changes
The persisted cache only ever holds rules from a successful remote fetch, so it's treated as the last-known-good copy:

Cached rules are loaded at any age. Age still triggers a refresh.
On a failed fetch, the last downloaded rules are kept. Bundled rules are only used if nothing has been downloaded yet.
Responses that aren't a rules file (non-JSON, error objects) are rejected so they can't replace good rules.
After a failure, wait at least 5 minutes before retrying, so an outage or rate limit doesn't trigger a request on every page load.
"Update Rules Now" reports a failed download instead of showing success.
Docs updated (docs/advanced/creating-detection-rules.md, docs/settings/detection-rules.md).
No new settings, so no enterprise/ADMX/schema changes.

Tests
Added tests/detection-rules-manager.test.js (11 tests). 8 fail on the current code and pass with this change.
Fixed tests/helpers/chrome-mock.js to support the callback-style storage API that browser-polyfill.js uses. Without it every storage read hung and the existing suite was cancelled before running. (Some tests in config-persistence.test.js still fail for an unrelated reason: the polyfill binds to the first chrome mock, but each test creates a new one.)
Manual testing
Tested in: Chrome and Firefox on

Windows 11 Pro
Microsoft Edge: Version 155.0.4283.60 (Official build) (32-bit)
Mozilla Firefox: Version 158.0 (64-bit)
Same steps in both browsers:

Extension loads, service worker/background starts, popup and options pages work.
Screenshot 2026-10-16 115502

test-pages/: phishing page blocked, safe page not flagged.
Screenshot 2026-10-16 121942
Screenshot 2026-10-16 120733

A real login page that previously gave a false positive (login.mypurecloud.com) loads with no warning.
Screenshot 2026-10-16 115502

"Update Rules Now" online: rules download and are cached.
Aged the cached rules by 48h (past the 24h interval), went offline, reloaded the extension:
Screenshot 2026-10-16 122020

Downloaded rules kept: version: 1.2.5 | usingFallback: false | downloaded at: <48h ago> | last error: Failed to fetch
A refresh logs Failed to fetch rules from remote URL, then Keeping previously downloaded detection rules until the remote URL is reachable

Detection still works offline.

"Update Rules Now" while offline shows: Failed to update detection rules: Could not download detection rules (Failed to fetch). Still using the previously downloaded rules. After reconnecting, it succeeds.

Fresh install while offline: bundled rules used, as expected.

Unrelated to this change: on load the rogue apps updater logs Failed to update rogue apps: HTTP 403 fetching https://huntresslabs.github.io/rogueapps/rogueapps.json.

Console output (Chrome, offline, cache aged 48h)

version: 1.2.5 | usingFallback: false | downloaded at: 2026-10-14T16:08:59.072Z | last error: Failed to fetch
Fetching detection rules from remote URL: https://raw.githubusercontent.com/CyberDrain/Check/refs/heads/main/rules/detection-rules.json
Failed to fetch rules from remote URL: Failed to fetch
Keeping previously downloaded detection rules until the remote URL is reachable
usingFallback: false

Kevin Phillips added 3 commits October 16, 2026 16:33
browser-polyfill.js calls chrome.storage.* with a callback, but the mock
only returned a promise, so every storage read in a test hung and the
existing suite was cancelled before any test ran. Accept an optional
callback on storage get/set/remove/clear, return a promise from
runtime.sendMessage like the real API, and record sent messages so
tests can assert on broadcasts.
When a rules update failed (GitHub outage, rate limiting, network
problems), the extension fell back to the bundled rules, which can be
several releases behind and cause false alerts. This happened in two
ways: an expired cache was discarded on startup (frequent with MV3
service worker restarts), and a failed background refresh replaced the
in-memory rules with the bundled ones.

The persisted cache only ever holds rules from a successful remote
fetch, so treat it as the last-known-good copy:

- Load cached rules regardless of age; age still triggers a refresh.
- On a failed fetch, keep the last downloaded rules (from memory or
  storage). Bundled rules are used only if nothing was downloaded.
- Reject responses that are not a rules file so they can't replace
  good rules.
- Wait at least 5 minutes between attempts after a failure, so an
  outage or rate limit doesn't trigger a request on every page load.
- Make forceUpdate() report a failed download instead of success.

Closes CyberDrain#176
Copilot AI balanced review requested due to automatic review settings October 16, 2026 16:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Worker restarts bypass retry backoff, and overlapping downloads can incorrectly report successful manual updates as failures.

2 open findings
What changed in this PR

Preserves last-known-good detection rules during update failures, reducing false alerts caused by reverting to older bundled rules.

Changes:

  • Retains expired cached rules and rejects unrelated responses.
  • Adds retry backoff and manual-update failure reporting.
  • Adds regression tests, improves browser mocks, and updates documentation.
File Description
tests/​helpers/​chrome-mock.js Supports storage callbacks and records runtime messages.
tests/​detection-rules-manager.test.js Covers cache retention, failures, retries, and forced updates.
scripts/​modules/​detection-rules-manager.js Implements rule retention, validation, backoff, and failure reporting.
docs/​settings/​detection-rules.md Explains failed-update behavior.
docs/​advanced/​creating-detection-rules.md Documents validation and fallback behavior.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/modules/detection-rules-manager.js Outdated
Comment thread scripts/modules/detection-rules-manager.js Outdated
- Store the time of the last failed fetch so the retry delay survives
  service worker restarts. Previously a restarted worker refetched on
  startup even during an outage.
- Have each fetch report its outcome to its caller instead of through
  a shared field, so an overlapping fetch can't make "Update Rules Now"
  report the wrong result.
@kphillipsjr1101

Copy link
Copy Markdown
Author

Fixed in 45c29f4: the last failure time is now stored and read on startup, so the retry delay survives service worker restarts. Added a test.

Fixed in 45c29f4: each fetch now reports its own result to its caller (no shared error field), so an overlapping fetch can’t change what “Update Rules Now” reports. Added a test.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Restarting without downloaded rules can bypass retry backoff and skip domain-squatting initialization.

1 open finding
2 resolved since last review

🧠 Review effort: Balanced

Comment thread scripts/modules/detection-rules-manager.js
Implement fallback mechanism for cached rules fetching

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Kevin Phillips <k.phillips.jr1101@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants