Skip to content

feat(reports): investor graphs receive reports, and a report records its origin - #2084

Merged
jfrench9 merged 4 commits into
mainfrom
feature/entity-report-ingest
Nov 19, 2026
Merged

jfrench9 merged 4 commits into
mainfrom
feature/entity-report-ingest

Conversation

@jfrench9

Copy link
Copy Markdown
Member

Summary

Two foundations for receiving reports. An investor graph now carries RoboLedger's reporting tables, so a report shared into a graph with the roboinvestor extension alone materializes and Portfolio → Position → Security → Entity → Report → Fact resolves there. And a report now records how it arrived (origin), so the rules that protect a received report no longer depend on it having a source graph.

Changes

Graph schema and materialization

  • schemas/extensions/roboinvestor.py: the extension's tables are its portfolio tables plus RoboLedger's reporting section (Report, Fact, FactSet and their edges), the same section the SEC repository loads. None of the general ledger.
  • schemas/runtime/manager.py, schemas/runtime/builder.py: a table named by two extensions is defined once, so a graph with both roboinvestor and roboledger still compiles.
  • operations/extensions/materialize.py: the reporting tables belong to a reporting section, staged on any graph with a receiving extension (roboledger or roboinvestor). Ledger tables still stage only on a ledger graph.

Report origin

  • models/extensions/roboledger/report.py: reports.origin, one of authored, shared, uploaded, NOT NULL with a CHECK, and Report.is_received.
  • Extensions migration 0069: adds the column in public and every tenant schema and backfills shared where source_graph_id is set.
  • operations/roboledger/commands/reports.py: a share stamps its copy shared. Read-only, admin delete and the filing-status exemption read is_received. A received report cannot be shared onward.
  • operations/roboledger/reads/reports.py, views/information_blocks.py: the stored artifact of a received report is never re-derived from rows, and it has no lineage download, keyed on origin.
  • models/api/extensions/reports.py, report_package.py, graphql/types/report_package.py: origin on the report responses.

Nothing writes uploaded yet; the value exists so the import that will use it lands on rules already in place.

Dependencies, demo, docs

  • xbrlkit 0.21 → 0.22.
  • examples/roboinvestor_demo: the fund graph is created with roboinvestor alone.
  • docs/technical: RoboInvestor-Operations and Reporting-and-Rendering say both of the above.

Breaking Changes

None.

  • Behavior change: sharing a received report onward is now refused with a not-authorized error. A shared-in copy could already never be shared (its author is the sender); this makes the rule explicit ahead of reports that arrive another way.
  • Additive API: origin on ReportResponse and the report package (REST and GraphQL). Generated tier, so a client minor on the next regen; the Python client's schema.graphql snapshot will need a refresh.
  • Deploy: extensions migration 0069 runs on the daemon's boot. An existing investor-only graph gains the reporting tables the next time it materializes.

Testing

  • just test-all: 19,359 passed, 44 skipped, 2 xfailed; ruff, format, basedpyright, cfn-lint, actionlint and the dbt tests passed.
  • Migration 0069 applied on the local stack: the column is present in public and all tenant schemas.
  • just demo-roboinvestor against the local stack on this branch, with the fund graph carrying roboinvestor alone: 31/31 checks, including "Portfolio → Position → Security → Entity → Report → Fact resolves" and "traversal attributes facts to the issuer".
  • Not run: the migration's downgrade, and a rebuild of an investor-only graph that existed before this change (the demo graph was created fresh).

Certification

  • I have the right to submit this work under the Apache 2.0 license, and do so. Where any part of it is owned by my employer, I have their permission.

🤖 Generated with Claude Code

Receiving a report needs RoboLedger's reporting tables (Report, Fact, FactSet
and their edges), not its general ledger. The roboinvestor schema now brings
that section the way the SEC repository does, and the materializer stages it
on any graph that can receive, so the traversal from a position to a fact
resolves on an investor-only graph. Bumps xbrlkit to 0.22.
A report carries an origin: authored from the graph's own books, shared in
from another graph, or uploaded as a file. Every rule that protected a
received report (read-only, never re-derived from rows, no lineage download,
admin delete) now reads the origin instead of the source graph, which a report
that arrives as a file does not have. A received report cannot be shared
onward. The origin is on the report responses.
The demo's fund graph now carries the roboinvestor extension alone, so its
traversal check proves the claim; the technical docs say it, and describe a
report's origin.
@github-actions github-actions Bot added the change:normal Normal change: automated review documented on the pull request before merge label Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Change classification: normal — platform or application code changed.

A normal change carries a documented automated review on this pull request before merge, in place of a second human approver. The change-classification job requests that review and passes once it has been posted. Recorded by the change-classification job.

@jfrench9

Copy link
Copy Markdown
Member Author

@claude please review this PR

@claude

claude Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Claude finished @jfrench9's task in 20s —— View job


Review of PR #2084

Verdict: solid, approve with a few follow-ups. The change is coherent and the tests and demo cover the main paths. I read the diff and checked a few call sites. I did not run the test suite.

Findings

  1. refreshable() still keys on source_report_id, not is_received (robosystems/operations/roboledger/commands/report_columns.py:65).

    • The PR moves the other "received" checks (read-only, delete, filing-status exemption, download, information blocks) to origin. This one is missed.
    • Nothing writes uploaded yet, so there is no bug today. An uploaded report has no source_report_id, so refreshable() would return True and the report would be recomputed from local books. That breaks the "never re-derived" rule this PR sets up.
    • Suggested change: if report_def.is_received: return False.
    • Please also grep for other source_report_id and source_graph_id truthiness checks. serialization/bundle.py:541 only copies the field, so it is fine.
  2. Reports with origin='shared' and source_graph_id IS NULL are possible. The backfill and the writer keep them paired, but the DB doesn't enforce it. A second CHECK would make the invariant explicit, for example origin <> 'shared' OR source_graph_id IS NOT NULL. This is optional. Skip it if you expect uploaded or other reports to carry a source id later.

  3. Migration 0069 looks right.

    • It adds the column with a default, backfills, then adds the CHECK, in both public and the tenant schemas.
    • The backfill's AND origin = 'authored' makes it idempotent.
    • The downgrade drops the constraint before the column.
    • Your PR notes say the downgrade was not run. Run it once locally.
  4. Schema dedupe by name silently drops the second definition (manager.py, builder.py).

    • This works because the reporting nodes come from the same REPORTING_NODES objects.
    • If the two extensions ever diverge on a table, the first one wins with no warning. Consider a debug log, or an assertion that a duplicate is identical, so drift doesn't go unnoticed.
    • builder.py and manager.py now carry duplicated dedupe logic. A small shared helper would be cleaner.
  5. Behavior asymmetry in _assert_report_mutable_by. uploaded has its own message ("ingest the corrected file instead"), and shared keeps the existing one. That's fine. The uploaded branch is not exercised by any test, since nothing creates uploaded reports. A one-line unit test that sets origin="uploaded" on a report would lock it in.

  6. Style nits.

    • information_blocks.py _received_model docstring: one line runs past 88 characters after the rewording. ruff format won't wrap docstrings, so reflow it by hand.
    • Report.origin is a free str on the API models (ReportResponse, ReportPackageEnvelope, GraphQL). A Literal["authored","shared","uploaded"] would give typed clients an enum. The trade-off is that the generated clients change shape, and you already note a regen.
    • In report.py, the is_received property sits between column definitions. Moving it below the columns would read better.
  7. Materialize. The REPORTING_SECTION approach is clean. Unknown tables still fall through to base, which keeps the "typo fails loud" behavior. Confirm that existing investor-only graphs pick up the new tables on the next materialize. The PR lists this as untested.

Things I'd confirm before merge

  • Item 1 is fixed.
  • The migration downgrade has been run.
  • An existing investor-only graph rebuild has been tested. This is the one path the demo did not cover.

The column refresh, the close's freeze and the reopen's thaw still told a
received report by its source report id, which a report that arrives as a file
does not have. All three read the origin now, with a database-backed test for
both received origins.
@jfrench9

Copy link
Copy Markdown
Member Author

Review follow-up, pushed:

  • Item 1 (fixed, and wider than reported). refreshable() now reads is_received, and so do the two SQL filters beside it in report_columns.py (the close's freeze and the reopen's thaw), which had the same source_report_id IS NULL test. A database-backed test covers both received origins: a close does not rewrite or freeze them and a reopen does not thaw them.
  • Migration downgrade: run. 0069 → 0068 → 0069 on the local stack; the column is gone from every schema after the downgrade and back in all of them after the upgrade, with the fund graph's shared report backfilled to shared.
  • Existing investor-only graph rebuild: run. A forced materialize of the demo's investor-only fund graph went through the blue-green path (a fresh database built from the current schema, then swapped) and the traversal from a position to the issuer's facts still resolves. That path builds a new database each time, so a graph created before this change gets the reporting tables the same way.
  • Item 5. The uploaded branch of _assert_report_mutable_by is covered by test_regenerate_report_refuses_a_report_received_as_a_file.
  • Items 2, 4, 6: not taken here. The paired-columns CHECK waits until the upload path settles what an uploaded report carries; the dedupe keeps the first definition because both extensions name the same objects.

@jfrench9
jfrench9 merged commit 1bd931f into main Nov 19, 2026
10 checks passed
@jfrench9
jfrench9 deleted the feature/entity-report-ingest branch November 19, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:normal Normal change: automated review documented on the pull request before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant