Skip to content

Log refused economy requests at WARNING instead of ERROR - #4220

Draft
anth-volk wants to merge 2 commits into
masterfrom
fix/economy-refusal-log-level
Draft

anth-volk wants to merge 2 commits into
masterfrom
fix/economy-refusal-log-level

Conversation

@anth-volk

Copy link
Copy Markdown
Collaborator

Fixes #4219

Summary

  • The annual and budget-window economy requests now log a request they refuse at WARNING, with the message Refused economic impact request (or Refused budget-window economic impact request), and every other failure at ERROR with the existing message. Both records keep error_type; an SPMValidationError also records its error_code, such as DATASET_YEAR_UNAVAILABLE.
  • A request counts as refused when:
    • it names a policy its country does not have (PolicyNotFoundError, 404); or
    • request validation raises a ValueError (400): a year outside the country's years (Define each country's years once and refuse society-wide requests for other years #4212), an unknown region, or a dataset, breakdown or SPM setting the API does not offer. Validation ends once the request's setup options are built, before any stored result is read or any job is sent.
  • A ValueError raised after validation stays at ERROR, although the route also answers it with 400. Such errors report a state the API did not expect, such as Unknown impact status, or replay a typed failure the simulation service stored for the request.
  • The new module policyengine_api/services/economy_request_failures.py holds the rule (is_refused_request) and the logging (log_economy_request_failure), used by both except blocks. It is added to ECONOMY_SERVICE_LOGGER_MODULES, so tests that replace the economy loggers replace it too.

Why

The daily telemetry error report groups every record at severity ERROR or above (severity_number >= 500 in .github/scripts/telemetry_error_issues/bigquery.py). A refused request's request.completed record is INFO (outcome client_error), so the service's own ERROR record was the only one that put it in an error group, as a message group on the economy route. #4212 now refuses UK years outside 2024 to 2032, the main source of #4099, #4216, #4217 and #4218, so those refusals would otherwise form a new error group.

Limit

A ValueError raised while the setup options are being built for a reason other than the request, such as a bug in building them, now logs at WARNING. The route already answers it with 400 as a client error.

Testing

  • tests/unit/routes/test_economy_refusal_log_severity.py (new) serves both economy routes from a real service over in-memory caches, through a real observability runtime with the API's logging settings that keeps every record it writes. So it checks every record a request writes, including its request.completed record and standard-library logging.
    • A year outside the UK's years, an unknown UK region and a US policy ID on the UK route each write no record at ERROR or above, and one WARNING refusal record with the exception's type (and the code DATASET_YEAR_UNAVAILABLE for the year).
    • A RuntimeError from the policy lookup (500), and a ValueError from it after validation (400), each still write one ERROR record with the failure message.
  • The first commit adds the tests: the six refusal cases failed on one extra ERROR record each, and the four failure cases passed. The second commit adds the fix; all ten pass.
  • make quality passes.

🤖 Generated with Claude Code

anth-volk and others added 2 commits November 6, 2026 18:50
A request the economy routes refuse with 400 or 404 (a year outside the
country's years, an unknown region, a policy the country does not have)
must add no record at ERROR, which the telemetry error report groups.
Unexpected failures, including a ValueError raised after validation,
must still be logged at ERROR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The annual and budget-window economy requests logged every exception at
ERROR, including the ones the routes answer with 400 or 404. The
telemetry error report groups every record at ERROR or above, so each
refusal formed an error group. A PolicyNotFoundError, or a ValueError
raised before the request's setup options are built, is now logged at
WARNING as a refusal. Any other failure, including a ValueError raised
after validation, is still logged at ERROR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.12%. Comparing base (5aaa1ff) to head (5a341bf).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4220      +/-   ##
==========================================
- Coverage   92.14%   92.12%   -0.03%     
==========================================
  Files         296      297       +1     
  Lines       16696    16714      +18     
  Branches     2722     2724       +2     
==========================================
+ Hits        15384    15397      +13     
- Misses        795      801       +6     
+ Partials      517      516       -1     
Flag Coverage Δ
integration 52.36% <0.00%> (-0.06%) ⬇️
unit 90.18% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop logging refused economy requests at ERROR

1 participant