[dwolfson]

A count that timed out is never reported as a measured total. On the live platform (build 082ecd44) the native count of a supertype (Asset, Process) times out at the client's 30 s limit. overview_metrics._element_count then fell back to len(find), which stops at DEFAULT_CAP (500), and in non-strict mode, the default for count_elements and the demo specs, returned that 500 as the count. The Portal showed it as a measurement: "Total Cataloged Assets = 500", "Assets by Type Breakdown: Processes = 500", and Semantic Grounding "500 links / 100%". Found in the Architect's post-deploy read-back of RS-W2-3 (READBACK-RS-W2-3-portal-deploy-2026-11-08.md §1–2).

Branch fix/capped-count-not-measured off upstream/main 556a2bb (6.2.26). No Egeria writes. No version bump.

Design: None plus a recorded reason, never a structured {"at_least": n} value

When the find fallback reaches the cap, the count is None in both modes. The bound and the reason are recorded as a note, {"count", "reason", "at_least"}, and the report-spec executor puts the notes on its output.

Why not return {"at_least": 500} from count_elements:

• It would break callers. count_elements is a public helper typed Optional[int]. The Overview handler and other routines do arithmetic on it (sums, percentages, term_total - referenced). A dict there is a TypeError. An int subclass marked as a bound would be worse: it serialises as 500, which is the bug again.
• None already has a meaning. None is already "not measured" in strict mode and in fix(overview-metrics): report unmeasured as None, not 0 #531, and every routine's declaration already has a not_measured sentence for it. Adding a second absence shape would give every renderer two things to handle.
• The number still reaches the reader. The bound is not lost: it travels as at_least in the note and as words in content.

How the reason reaches the output:

• overview_metrics.recording_unmeasured() is a context manager over a ContextVar. It collects every note made while it is open.
• exec_report_spec opens it around each analytic routine.
• The notes are per context, so asset_composition's worker threads record into their own cell's block. A cell's reason now names the bound (e.g. "...so there are at least 500").

What a reader sees, per renderer

output	before	after
DICT/JSON, scalar (KPI tile, e.g. Element Count by Type with type_name: Asset)	{"kind":"json","data":500}	{"kind":"absent","content":"The elements were not counted: … Not measured: Asset: the native count timed out, and the find fallback stopped at its 500-element page cap, so there are at least 500.","absence":{state:"not_measured",…},"envelope":{state:"not_established",…},"unmeasured":[{"count":"Asset","at_least":500,"reason":…}]}
DICT/JSON, rows or dict (Breakdown, Semantic Grounding)	the capped values as numbers	the measured values unchanged; each unmeasured one is null; "unmeasured": [...] names each one with its reason and bound
BAR / PIE / SERIES (Vega)	a chart with 500-high bars	kind: "absent", with no chart and the data under detail. A missing bar reads as 0, and a pie without a slice changes every share, so the chart is withheld as ISSUE-140 already does for a non-established state

Changes

• overview_metrics._element_count: a fallback page that reached DEFAULT_CAP returns None in either mode, never the cap. A failed native count is described in words: "timed out", "failed (X)", or "no native count is available". Non-strict failure still returns its historical 0; strict returns None, as before.
• count_relationships: a full get_relationships page returns None plus a note. This is the Semantic Grounding links = 500 case, and also covers License/Exception/feedback/DataFlow counts.
• Count specs ask for strict counts ("strict": True in analytic_spec_params):
  • Element Count by Type
  • Assets by Type Breakdown
  • Count by Property
  • People and Community
  • Usage Context
  • A caller's params still merge over these defaults.
  • sum_type_counts fetches strictly.
  • sum_counts' total is None when any row is (a sum of the measured rows only would read as the whole total).
• Internal callers count strictly: semantic_grounding, orphan_glossary_terms, stale_assets, contextualised_coverage, solution_component_realisation, growth_series (the fix(overview-metrics): report unmeasured as None, not 0 #531 rule).
• exec_report_spec:
  • Adds unmeasured.
  • A scalar not-measured count is absent, not empty ("No results." would read as "nothing found").
  • A chart over one is withheld.
  • A routine's own measured envelope is overridden to not_established when counts inside it were not measured.
  • Identical repeated notes (one per series point) are kept once.
• Declarations:
  • The not_measured sentences for count_elements, count_elements_by_property, counts_by_type and semantic_grounding now name timeouts and the page cap.
  • out shapes are unchanged: every changed field was already an int that can be absent, so no shape changed.
  • completeness and population are untouched.

Demo spec outputs: before (6.2.26) vs after

Snapshot of all 23 demo FormatSets, and 111 exec_report_spec runs (DICT plus every chart type each spec offers). The runs use fake clients in three scenarios: ok (native counts answer), fail (every query raises), and timeout_cap (native count times out; find and get_relationships return a full 500 page).

• FormatSets: 18 identical, 5 changed. Only the description's "Default params" text changed, gaining 'strict': True. The five are Element Count by Type, Assets by Type Breakdown, Count by Property, People and Community, and Usage Context: Supply Chains & Blueprints.
• Runs: 69 identical, 42 changed. Every ok run is byte-identical. All 42 changes are fail or timeout_cap runs.

spec (format)	timeout_cap before → after	fail before → after
Element Count by Type (DICT)	500 → absent, "at least 500"	0 → absent, "find fallback failed"
Count by Property (DICT)	500 → absent, "at least 500"	0 → absent
Assets by Type Breakdown (DICT)	counts 500/500/500 → null + unmeasured	0/0/0 → null + unmeasured
Assets by Type Breakdown (BAR, PIE)	chart of 500s → absent, no chart	chart of 0s → absent
Assets by Type Total (DICT)	total 1500 → null	total 0 → null
Semantic Grounding (DICT)	links 500, pct 100 → null, null + unmeasured (Asset, SemanticAssignment)	unchanged values, gains unmeasured
People and Community (DICT / BAR / PIE)	500s → nulls / absent	0s → nulls / absent
Usage Context (DICT)	500s → nulls	0s → nulls
Certifications and Exceptions (DICT / BAR)	licenses, exceptions 500 → null / chart → absent	unchanged
Feedback Summary (DICT / BAR)	byType 500s, total 2500 → nulls / absent	unchanged
Catalog Growth Trend (DICT / SERIES)	points 500 → null / absent	points 0 → null / absent
Orphan Glossary Terms (DICT)	termTotal 500, orphan 0 → null, null	termTotal 0 → null
Stale Assets (DICT)	assetTotal 500 → null	assetTotal 0 → null
Contextualised Coverage (DICT)	assetTotal 500, pct 100.0 → null, null	gains unmeasured
AI Context Readiness (DICT)	lineage 500 → null	gains unmeasured
Business Value Signals (DICT / BAR)	gains unmeasured (duplicateCount) / chart → absent	gains unmeasured / → absent

Release note (user-visible)

Counts that could not be measured now say so. When Egeria's count of a large type, such as Asset or Process, timed out, a dashboard tile or report used to show the result of a 500-element fallback page as if it were the total ("Total Cataloged Assets: 500"). Such a count is now reported as not measured, with the reason and the lower bound seen ("the native count timed out, and the find fallback stopped at its 500-element page cap, so there are at least 500").

• Report-spec results carry a new unmeasured list.
• A single count that was not measured is returned as kind: "absent" with that sentence.
• Charts are not drawn over a count that was not measured.
• count_elements(...) and count_relationships(...) return None, not 500, when their fallback page is full, in non-strict mode too. Python callers that add these numbers up should treat None as "not measured".
• Use pyegeria.view.overview_metrics.recording_unmeasured() to get the reason.
• Unchanged: in non-strict mode, a count whose query fails outright still returns 0, as before. Pass strict=True to get None.
• Deploy note for the Portal: render kind: "absent" and unmeasured before running on this release (odpi/egeria-workspaces rendering PR), or tiles show raw JSON.

Live timing (iii), read-only, client timeout 120 s

Method:

• MetadataExpert.count_metadata_elements with timeout=120, run one at a time on 2026-11-08.
• Platform https://localhost:9443, view server qs-view-server, user resourceexplorernpa (RE .env).
• Build 082ecd44 per the read-back; not re-checked here.
• Taken after a fresh restart: about 18:05–18:07Z, after egeria-main was heap-dumped and restarted (all servers up by 17:56:58Z). These are fresh-heap numbers, not the 95%-full-heap state of the read-back.
• Body: FindRequestBody, type name, ACTIVE.

type	server default (what the tile sends)	repository (push_down=True)
Asset	52.98 s, 26,994	0.45 s, 27,026
Process	45.09 s, 24,491	0.43 s, 24,491
DataStore	4.29 s, 848	0.40 s, 848
GlossaryTerm	4.66 s, 1,061	0.40 s, 1,061

What this tells the owner:

• The supertype counts do finish, at 45–53 s, which is past the 30 s client timeout. So on this platform the tile's number is unmeasurable at the default timeout, not merely slow.
• The repository-population count of the same types is about 0.4 s. The cost is in the server-default (viewer) path, which evaluates visibility per element.
• That is worth an Egeria performance issue: a viewer-scoped COUNT on a supertype with about 25k instances takes about 50 s.
• The two populations differ for Asset by 32 elements that this user cannot see. So the repository count is not a drop-in replacement for a viewer tile.

The default client timeout is unchanged (30 s) in this PR.

Tests

• uv run -p 3.13 pytest tests/micro-tests/, without the library: 2274 passed, 3 skipped.
• With egeria-trellis-analytics 0.4.0 (scratch venv, editable pyegeria from this branch): 2399 passed, 1 skipped.
• Census unchanged: 437 specs (438 with the library), 2 unrunnable (Asset-Graph, Data-Classes).
• New: test_capped_count_not_measured.py, 20 tests, each executing the branch. They cover:
  • a timeout whose fallback hits the cap → None in both modes, with the "at least 500" note;
  • a timeout whose fallback fails → None (strict);
  • a timeout leaves native enabled for the next count;
  • a page one short of the cap is a measured count;
  • a native count above the cap is trusted;
  • a full relationship page;
  • Semantic Grounding no longer 500/100;
  • sum_counts never adds a bound into a total;
  • an asset-composition cell's reason;
  • exec_report_spec for the tile (absent, at_least 500), a measured tile unchanged, Breakdown rows, BAR/PIE withheld, series notes deduplicated;
  • no rendered output carries 500 as a count.
• Updated: test_issue_123_strict_counts.py::test_strict_returns_none_for_a_capped_fallback pinned the old non-strict behaviour (== DEFAULT_CAP). It is now test_a_capped_fallback_is_none_in_either_mode_never_the_cap.
• ruff --select F,E9 is clean on the new code. The remaining findings in the touched modules pre-exist on upstream/main.

Consumers (not changed here)

• Portal local-dashboards.html renderExecResult (live checkout 3156a8d2, read only):
  • It has no branch for kind: "absent", so such a result falls to its last branch: a <pre> dump of the whole JSON. That is honest (the sentence and the "at least 500" are visible) but not a tile, and it predates this PR: ISSUE-171's absent was never rendered there.
  • In tables, null cells render blank (v == null ? ''), and unmeasured is ignored, so a reader sees an unexplained gap rather than a 500.
  • The Portal's report_specs_handler.py passes the dict through unchanged.
  • A small egeria-workspaces PR is recommended:
    • kind: "absent" → a KPI tile showing "Not measured" plus content, or the first unmeasured[].at_least as "≥ 500 (not measured)";
    • kind: "json" with unmeasured → "not measured" in null cells and a note listing the reasons.
• overview_handler.py calls count_elements non-strict in two places: surveyReports, and dataStoreTotal when stores_capped. It also calls semantic_grounding. These can now get None where they used to get 500. Check that they render None as unmeasured.
• RE curation-coverage tile: not affected. It runs the library's curation_coverage over RE's own ODS registry, not Egeria counts, and curate-coverage.js already shows not_established/not_applicable as the state plus the reason, with no table.

Follow-ups (same defect family, not changed here)

Capped element scans (not counts) still report a page-cap value as a number:

• with a *Capped flag: governedCount, ownershipCount, assetTotal in Business Value Signals, total in AI-Ready Assets;
• with no flag: certifications_summary.active, term_definition_completeness.total, stale_assets.staleCount, contextualised_coverage.contextualisedCount, orphan_glossary_terms.referencedCount.

🤖 Generated with Claude Code