Repository navigation
feat(roboinvestor): ingest a report from a company that is not on the platform - #2085
Conversation
A holon is read and checked with xbrlkit, stored whole under the report's bundle key, and projected into the tenant as an uploaded report on a named entity. Its concepts and sections go through the reporting-extension command (the library's own concept where it has one, pulled in when the tenant's copy left it out); this writes the report, its fact sets and its facts, each fact keeping its source. A second import for the same entity, period end and kind replaces the first, and deleting an uploaded report takes its sections with it. A file that cannot be read or does not check is refused with its reasons.
Two operations on the RoboInvestor surface, each also an MCP tool. create-issuer adds a company that keeps its books elsewhere as an entity securities and reports attach to. ingest-report loads a report onto an entity from a holon, or from rows (concepts, sections and values with the page or cell each came from) that the server assembles into a holon first, so every report in the graph was loaded from a stored serialization. dry_run checks and says what would load without writing.
…ompany A concept a report brings is a received concept, not one of the graph's accounts, and keeps its qname in the graph. The reports list shows the fund what it received about a counterparty, takes a counterparty to list that company's reports, and shows an uploaded report only the sections it brought.
|
Change classification: normal — platform or application code changed. A normal change carries a documented automated review on this pull request before merge, in place of a second human approver. The |
|
@claude please review this PR |
|
Claude finished @jfrench9's task in 39s —— View job PR review: ingest a report from a company that is not on the platformI read the importer, the RoboInvestor commands, the router registration, the read changes and the API models. I did not run the tests. Your PR description reports The design holds up. Rows are assembled into a holon, which is checked before anything is written and then stored. Vocabulary goes through the existing Should fix1. Replacing a report orphans its stored holon in S3.
Worth addressing2. The holon is uploaded to S3 before the DB commit.
2a. Dry run is not entirely free of side effects, and the check comes after the parse.
3.
4. A numeric concept with a null
5. Concurrent ingests for the same entity, period end and kind can both succeed.
Nits
Checks
Verdict: the data model and checks are solid. I'd merge after fixing #1. The rest can follow. |
A replaced report's file is deleted once the replacement commits, and a report whose transaction is undone takes its own file with it, on every surface. The entity is locked for the import so two for one company cannot both find nothing to replace. A nil value is not loaded, the holon field has a size limit, and rows for an entity outside the graph are refused before anything is built.
|
Review follow-up, pushed:
Affected suites rerun: 2,678 passed. |
Two things a real ledger-authored report carries that a built one does not. A concept the graph already holds in another vocabulary (one a share brought, or a firm's own extension) is linked from the report's section rather than refused by the extension command, which resolves only its own vocabulary and the library. And a line presented twice under one parent is one association.
…latform Two steps. A holding whose issuer keeps its books elsewhere gets an issuer entity and its monthly package ingested from rows, after a mistyped copy is refused with the numbers and a dry run writes nothing. Then the holon of the report the share delivered is downloaded and ingested as a file, and the run asserts it lands the same facts and values as the shared copy. A refusal now writes its amounts out in full.
Summary
A fund can now load a report from a company that keeps its books elsewhere.
create-issueradds the company as an entity, andingest-reportloads its report from a holon or from rows read out of a PDF or a workbook. Rows are assembled into a holon first, so every report in the graph was loaded from a serialization that is stored and can be downloaded. The report is checked before anything is written, and one that does not check is refused with its reasons.Changes
Importer (shared kernel)
operations/roboledger/commands/report_import.py(new):import_reportreads a holon with xbrlkit, runs xbrlkit's report check, stores the file whole under the report's bundle key, and writes anuploadedreport with its fact sets and facts on a named entity. Each fact keeps the source the file gave it, onassertedprovenance.dry_runchecks and reports what would load without writing.commands/reports.py: deleting an uploaded report removes the sections it brought.RoboInvestor operations (each also an MCP tool)
operations/roboinvestor/commands/reports.py(new),routers/extensions/roboinvestor/operations.py,models/api/extensions/investor.py:create-issuerandingest-report, with their request and response models. A duplicate identifier is 409; a refused report is 422 naming each reason; an unknown entity is 404.middleware/mcp/tools/annotations.py: hints for the two new write tools.Reads
operations/roboledger/reads/reports.py: the reports list shows the fund what it received about a counterparty, accepts a counterparty's id to list that company's reports, and gives an uploaded report only the sections it brought.Demo
examples/roboinvestor_demo: two new steps. A holding whose issuer keeps its books elsewhere gets an issuer and its monthly package from rows (a mistyped copy is refused, a dry run writes nothing). Then the holon of the report the share delivered is downloaded and ingested as a file, and the run asserts it lands the same facts and values as the shared copy.Docs
docs/technical/RoboInvestor-Operations.md: a section on receiving a report from a company that is not on the platform, and the operation count.Breaking Changes
None.
create-issuer,ingest-report) and their models, generated tier, so a client minor on the next regen.reportslist now includes reports received about a counterparty entity, andreports(entityId:)accepts a counterparty's id (it returned an entity-not-found error before). Lists that held only authored and shared reports are unchanged.Testing
just test-all, exit 0, at the first four commits: 19,376 passed, 44 skipped, 2 xfailed; the code-quality stages and the dbt tests passed. The later commits were covered by the affected suites (2,678 passed after the review fixes) and by CI.just demo-roboinvestoron the local stack: 45/45 checks, including the route check (62 facts by the share, 62 by the file, the same total).test_report_import_db.py,test_report_ingest_db.py): a package lands with its values, sources, sections and calculation arcs; a curated-out library concept is pulled rather than duplicated; replace; delete; dry run; refusals for a total that does not foot, a missing fact, an unreadable file, an unplaceable row, an unknown entity and a duplicate issuer; listing for the fund and for the company.roboinvestoralone: created an issuer, dry-ran and ingested two months from rows, confirmed a package that does not foot returns 422 with the numbers, added a security and a position, rebuilt the graph, and ranPortfolio → Position → Security → Entity → Report → Factto the ingested facts. Listed the reports over GraphQL both ways, and downloaded the stored holon throughreportDownloadUrl, read it back and re-ran the check on it.reportPackagerenders only the statement sections of an uploaded report, not a custom table.Certification
🤖 Generated with Claude Code