Repository navigation
feat!: settle the public surface of both clients for 0.2.0 - #34
Merged
Merged
Conversation
- typescript: trimEnd in core/text.ts trims trailing characters in one scan and replaces every trailing-trim regex, which ran in quadratic time on long runs (CodeQL js/polynomial-redos). Two tests escape their input in full (js/incomplete-sanitization). - python: the transport builds its own httpcore2 pools with the attempt-deadline backend through public names only, reads proxies from urllib, and drops the <2.16 bounds on httpx2 and httpcore2. - Both clients set answer.timed_out when any part's own reason is timed_out, and expose the party totals a rendering left out as Dataset.party_totals_left_out (partyTotalsLeftOut), saved under dfo.party_totals_left_out. - typescript: Answer.facts and partFacts give the checked facts of each part. - The dashboard journeys check the local-stack seed once and stop with one failure that names the missing readings. - CI installs the lowest permitted versions on Python 3.12 and the newest on 3.14, weekly as well, and tests the apache-arrow peer at 21.0.0 and the newest 21.x. Struct columns that differ between members now stack on pyarrow 15 to 20, and float16 columns compare on pyarrow 15 to 25. polars needs 1.3. Closes #11 Closes #24
Both packages move to 0.2.0. Every breaking change is listed under Unreleased in python/CHANGELOG.md and typescript/CHANGELOG.md. - Errors: every error a public call raises is a DfoError, FormatError and BundleError included, and a damaged file given to dfo.load raises one. Calls that raised ValueError or TypeError still do. Messages end in [<code> trace=<id>]. The old-form shims in Answer.choose and the TypeScript options map are gone. - Timeouts: the client's timeout bounds a whole call unless the call sets its own, and attempt_timeout (attemptTimeout) bounds one attempt. docs/settings.md in each package lists every setting. - entities= takes dcids or what lookup.resolve returns. - fetch is overloaded per target in both languages, and type tests pin the arguments each target refuses. - Datasets: Python gains to_bytes() and load(bytes); TypeScript renames bytes() to toBytes(), gains save(path) on Node, and spells licence as Python does. A cache hit says so in the repr, and saved files record dfo.question and dfo.fetched_at. - TypeScript: Answer.parts types only the members the client checks, isKnownPart checks them, QueryOption is Option, and the root exports the nine names that match Python's top level, with the rest under ./errors and ./types. - Docs: the recipes run without warnings a reader should ignore, and a nightly workflow runs every documented example against dev. Closes #6 Refs #19
tests/test_typing.py runs ty on a forwarded fetch target, so the python-lowest job pins ty from uv.lock beside the other test tools.
pyarrow raises a plain OSError, outside ArrowException, when a page header fails to decode, so dfo.load let it escape DfoError. _open_saved opens a path before parsing, so a missing file still raises FileNotFoundError, and turns any OSError while decoding into BundleError not_a_bundle.
jm-rivera
marked this pull request as ready for review
November 14, 2026 14:35
…Error A deadline taken from the client's timeout now reads "the client's timeout passed", for pin, cards, history, lookups and raw requests as well as fetch. dfo.load takes any bytes-like object, and a read that fails at the disk raises its own OSError where it was reported as a damaged file. A TypeScript fetch of a ref or share refuses entities before it resolves a Match, as Python does.
On pyarrow 15 to 20 the client stacked members whose struct columns differ by round-tripping cells through Python, which cut nanoseconds from time64[ns] fields, dropped field nullability and metadata, and missed structs inside maps. pyarrow 21 promotes structs in concat_tables, so the fallback and its version gate go. A float16 column is cast to float64 before its unique values are taken, which needs no version check.
# Conflicts: # python/tests/datasets/test_dataset.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Settles the public surface of both clients that #19 asks for, and finishes #6 by typing only the part members the TypeScript client checks. Both packages move to 0.2.0, and each
CHANGELOG.mdlists every breaking change under Unreleased.Closes #6
Part of #19
Changes
FormatError,BundleErrorand every bad-input raise on a public path areDfoErrors in both clients, anddfo.loadon a damaged file raises one, while a read that fails at the disk raises its ownOSError.dfo.loadtakes any bytes-like object. Sites that raisedValueErrororTypeErrorin 0.1.0 still do. Messages end[<code> trace=<id>].tests/core/test_errors.pypins the builtin raises that remain.*old_forminAnswer.chooseand the TypeScriptRENAMEDoptions map.timeoutis the default bound on a whole call, a call'stimeoutoverrides it, andattempt_timeout(attemptTimeout, default 30 s) bounds one attempt. A timeout's message names the option that ran out.CLIENT_TIMEOUTandCallTimeoutare indfo.types.python/docs/settings.mdandtypescript/docs/settings.mdlist every setting with its argument, variable, default and precedence.entities=takes dcids,Entityobjects and resolvedMatchobjects fromlookup.resolve. Any other match is refused before a request, with a hint naming a candidate or near match. A ref or share given entities is refused for that first, in both clients.fetchhas one overload per target in both clients, plus a union overload in Python.python/tests/test_typing.pyandtypescript/test/fetch-options.types.tspin the arguments each target refuses.to_bytes(),load(bytes),from_cache,fetched_atandquestion. The cache stores each download's fetch time underfetched/. TypeScript renamesbytes()totoBytes(), addssave(path)through a writerindex.node.tsinstalls, spellslicencethrough a client-ownedCitationSource, and addsquestionandfetchedAt. Saved files carrydfo.questionanddfo.fetched_atas top-level keys.Answer.parts,answer.specsandSpec.resolvedtype only what the client checks, with every other memberunknown.isKnownPartruns the strict-field check, andQueryOptionisOption.@developmentfinance/client/errors, every other name to@developmentfinance/client/types, andtest/package.test.tspins all three lists.docs/run without a warning a reader should ignore, no example silences warnings, and.github/workflows/docs-examples.ymlruns every documented example against the dev website nightly.scripts/check_docs_examples.pyreports each warning a Python block shows.Notes for reviewers
Client(timeout=)changes meaning, from one attempt to the whole call. Code written for 0.1.0 meets adeadline_exceedederror. A slow build cannot fail quietly../errorsand./types, mirroring Python'sdfo.errorsanddfo.types. Settle the public surface of both clients before releasing 0.1.0 #19 asked for one.DfoError.codeis nullable in TypeScript, asErrorCode | Noneis in Python, becauseFormatErrorandBundleErrorcarry no contract code.fetchedAtis the client's clock when the dataset is built, since that client keeps no disk cache. Python'sfetched_atis the download time, absent for cache entries written before 0.2.0.DFO_TIMEOUTandDFO_MAX_RETRIES, thewithout_party_totalspolarity, thereaderkitcache directory, the server'slicence/licensesplit, client codes forFormatError,BundleErrorand the refusals that now borrowinvalid_argument, a scenario expect key forparty_totals_left_out, and the cache'sfetched/tree.entities=takes lookup results only, sofetchmakes no network call to resolve names.Testing: the full CI recipe in both languages (Python 4,357 passed; TypeScript 2,104 passed; ruff, ty, lint-imports, the three codegen checks, eslint, tsc, build, actionlint); the live journeys against the dev website (Python 8 passed and 2 skipped for the inline long build and the crossing, which then passed alone; TypeScript 9 passed and 2 skipped);
scripts/local-stack-journeys.sh both(Python 13 passed, TypeScript 7 passed, crossing passed);scripts/check_docs_examples.pyagainst the dev website (Python 67 passed with 5 deliberate warnings; TypeScript 22 passed, with the twoclient.shareblocks refused by dev's daily share allowance).