Repository navigation
Expand file tree
/
Copy pathllms.txt
More file actions
44 lines (36 loc) · 8.48 KB
/
Copy pathllms.txt
File metadata and controls
44 lines (36 loc) · 8.48 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# development-finance
> Python client for the Development Finance Observatory (DFO) APIs. `dfo.search(question)` returns an `Answer` that says, in Search's sentence, what DFO understood a plain-language question about development finance (ODA, debt, GDP and similar indicators) to mean; `print(answer)` shows the sentence and the next line to type. `answer.fetch()` builds and downloads a versioned table as a `Dataset`. Install with `uv add development-finance` or `pip install development-finance`; import as `dfo`. Python 3.12 or later.
An answer holds readings and no data. When a part of the question has several readings, `print(answer)` and the `dfo.errors.IncompleteAnswerError` that `fetch` raises end with a `Choose:` line such as `answer.choose(0).fetch()`. The order of readings can change between searches, so a script keeps `data.ref` and leaves the index out. A table of flows leaves out group and total rows by default, so a sum counts each party once; `party_totals=True` keeps them. Columns come in reading order: year, parties, value, unit, then the columns that vary between rows, then the constant ones, with `stat_var` last. Rows sort by those columns and then the year, so each series reads in time order. Every error derives from `dfo.errors.DfoError` and carries a `code`. On `FormatError` and `BundleError` the `code` equals their `reason`. The message ends with `[code trace=id]`, or `[code]` when the error has no trace id. The bracket text can change, so read `error.code` and `error.trace_id`.
Main calls:
- `dfo.search(question, *, timeout=CLIENT_TIMEOUT) -> Answer` (60 s unless the client sets `timeout`); `answer.sentence`, `answer.parts`, `answer.specs`, `answer.complete`, `answer.gaps`
- `answer.choose(option, *, part=None) -> Answer`: settle a part with several readings, or a place that matches several, by index, label, or the option's line as `print(answer)` shows it (a candidate's `label · source · unit`); a closing full stop is ignored. One Search request
- `answer.fetch(*, partial=False, window=None, entities=None, facets=None, strict=False, layout="long", party_totals=False, timeout=CLIENT_TIMEOUT, progress=None) -> Dataset`. `window`, `entities` and `facets` narrow every reading as `dfo.fetch` narrows one spec. No deadline unless `timeout=` on the call or the client; Ctrl-C leaves the build running and the same fetch joins it
- `dfo.fetch(target, *, as_of=None, slot=None, window=None, entities=None, facets=None, layout=None, party_totals=None, latest=False, strict=False, timeout=CLIENT_TIMEOUT, progress=None) -> Dataset`. Overloads type each target's options: `window`, `entities`, `facets`, `layout` and `party_totals` go with a `Spec`, `latest` with a `DatasetRef` or `Recipe`, `as_of` and `slot` with a share. `window` is `(start, end)` with years from 1000 to 2200, `"latest"` or `"all"`; `layout="frame"` returns one reading's pipeline frame; `entities` maps a spec's role keys, listed in `spec.roles` (`place`, `origin`, `destination`), to place codes or to the `Match`es `dfo.lookup.resolve` returns; a key the spec does not take raises before any request. `target` is a `Spec`, a `DatasetRef`, a `Recipe`, or a `Share` or its URL
- `Dataset`: `to_pandas()`, `to_polars()`, `to_arrow()`, `pyarrow.table(data)`, `len(data)`, `save(path)`, `ref`, `members`, `sentence`, `citation`, `complete`, `missing`, `stale`
- `dfo.load(path) -> Dataset`: a file `data.save` wrote, or a website bundle
- `dfo.fetch(data.ref)` returns the same versions; `dfo.fetch(data.ref, latest=True)` the newest, and `new.ref != old.ref` tells whether the data changed
- `answer.to_json()` and `dfo.Answer.from_json(text, client=None)`: keep or share an answer and choose later
- `dfo.analysis`: `filter(data, *, keep)`, `aggregate(data, *, by=(), how="sum")`, `growth(data, *, by=(), window, change="relative")`, `rank(data, *, by, window=None, how=None, limit=None)`, `quotient_of(data, other, *, on=None)`, `share_of(data, total=None, *, by=None, on=None)`, `sum_of`, `compare`, `join`, each with `strict=False, timeout=CLIENT_TIMEOUT`, computed on DFO and returned as a derived `Dataset` (`ops_` spec, `ocv_` version). `data` is a `Spec`, `DatasetRef`, `Dataset` or `Recipe`. `on` pairs roles, such as `{"destination": "place"}`. Scales: growth `relative` is a fraction (0.25 is 25%), `difference` is in the data's unit; `share_of` is a percentage (25.0 is a quarter). A derived frame's `time_period` is text, `"2022"` or `"2018/2023"`; `unit` holds a unit id or a composed unit's JSON text
- `dfo.analysis.recipe(start).growth(...).rank(...)` describes a chain without I/O; `dfo.fetch(recipe)` runs it in one request. `dfo.latest_year(offset)` is a year DFO settles on each run, such as `window=(dfo.latest_year(-4), dfo.latest_year())`. `dfo.fetch(derived.ref, latest=True)` computes it again on the newest data
- `data.left_out`: the `LeftOutSeries` an operation dropped for missing data (`coordinates`, `step`, `side`, `reason`, `rows`). Each such fetch issues one `dfo.errors.SeriesLeftOutWarning` (`series_left_out`), raised under `strict=True`
- `dfo.pin(data) -> Dataset`: keep a dataset version fetchable for good. It refuses a derived Dataset
- `dfo.cards.line|bar|table(data, *, window=None, title=None)`, `dfo.cards.stat(data, *, year=None, compare_to=None, change="difference", title=None)` (`percent` is a percentage), `dfo.cards.source(data)`, each `-> Card`; a card draws every series of one version, so narrow with `filter` first
- `dfo.share(cards, *, latest, replace=None, write_token=None) -> Share` publishes cards at a public URL (`share.url`, `share.embed_url`, `share.data_urls`) anyone with the link can open. Keep `share.write_token` (`shw_…`), given only at creation: `replace=`, `dfo.snapshot(share)` and `dfo.unshare(share)` need it. `dfo.history(share)` lists what each slot served, and `dfo.fetch(share, slot=1, as_of=date)` returns what a slot showed then
- `dfo.lookup.resolve(*names, kinds=None, code_system=None, expand_members=False) -> list[Match]`, one per name: `text`, `status` (`resolved`, `ambiguous`, `not_found`, `not_held`), `entity` (`dcid`, `name`, `kind`; set for `resolved` and `not_held`), `candidates`, `near_matches`, `raw`; `print(match)` shows `Germny -> not found; nearest: Germany country/DEU`. `dfo.lookup.node(dcid, *, as_of=None) -> Node`, `dfo.lookup.code(code) -> Code` (`definition`), `dfo.lookup.catalogue(place=None) -> Catalogue` (`counts`, `examples` as question strings); each keeps `raw`. `client.lookup` on a client
- `dfo.Client(*, api_key=None, base_url=None, timeout=None, attempt_timeout=30.0, max_retries=2, cache_dir=None, cache=True, headers=None, header_provider=None)`, a context manager with `search`, `fetch` and `lookup`; `timeout` bounds each whole call that passes none, `attempt_timeout` one request attempt; `client.with_options(timeout=, attempt_timeout=, max_retries=)` returns a copy that shares its connections and cache. `dfo.AsyncClient` mirrors it, and `dfo.configure(...)` replaces the default client
- Modules: `dfo.analysis`, `dfo.cards`, `dfo.errors` (errors and warnings), `dfo.types` (parts, candidates, gaps, members, lookup results, `Recipe`, `LatestYear`, `LeftOutSeries`, `Card`, `Share`, `Served`), `dfo.lookup`, `dfo.auth`
- CLI: `dfo auth login [--with-token]`, `dfo auth status`, `dfo cache clear`
- Environment: `DFO_BASE_URL`, `DFO_API_KEY`, `DFO_PROFILE`, `DFO_CREDENTIALS_FILE`, `DFO_CACHE_DIR`, `BBLOCKS_CACHE_DIR`, `DFO_OFFLINE=1`, `DFO_PROGRESS=0`
## Docs
- [README](README.md): install, the first five lines, and a recipe per task: choose, other years or places, totals, save and load, refresh, share, place codes
- [Answers and choices](docs/answers.md): parts, `choose`, `partial=True`, `SearchWarning` and `strict`, `to_json`, declined answers, gap kinds
- [Datasets and versions](docs/datasets.md): `window`, `entities`, `facets`, layouts, `party_totals`, conversions, refs, retention, `save` and `load`
- [Long builds](docs/long-builds.md): progress, deadlines, Ctrl-C
- [Compute a figure on DFO](docs/compute.md): `dfo.analysis`, recipes, `latest_year`, recomputing, saving a derived result, `left_out`
- [Cards and shares](docs/shares.md): cards, public shares, write tokens, replace, snapshot, unshare, history, `as_of`
- [Errors](docs/errors.md): every error class and code, and warnings
- [Keys and credentials](docs/keys.md): key lookup order, `dfo auth login`, the credentials file
- [The cache](docs/cache.md): layout, `DFO_CACHE_DIR`, `DFO_OFFLINE`, stale reads
- [Settings](docs/settings.md): every option and environment variable, its default and precedence
## Optional
- [Changelog](CHANGELOG.md)