Skip to content

feat: add a unified county argument, like state - #433

Draft
thodson-usgs wants to merge 5 commits into
DOI-USGS:mainfrom
thodson-usgs:feat/county-argument
Draft

thodson-usgs wants to merge 5 commits into
DOI-USGS:mainfrom
thodson-usgs:feat/county-argument

Conversation

@thodson-usgs

@thodson-usgs thodson-usgs commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

TL;DR: Adds a county argument that works like state. It accepts a FIPS code, or a name with its state, and sends each service the fields that service filters on. I checked every conversion against the live services. Scope follows state: Water Data's location and statistics getters, ngwmn.get_sites, and nwdc.get_wateruse, but not WQP, NWIS, or Samples. ADR 0013 and CONTEXT.md now record the rule both arguments follow.

Stacked on #432 (Virgin Islands name fix): its commit e3c3c335 is the first one here. Merge #432 first; review the commits after it.

Changes

  • codes.counties (new; exposes to_county and apply_county). The table holds 3,239 counties and county equivalents. It is a snapshot of the Water Data counties collection for the states and territories that codes.states covers, and a live test compares the two. Accepted inputs:

    • "55025", 55025, or "US:55:025";
    • with state: "025", "Dane County", "dane", or "St Louis County" (case and periods don't matter, and the designation such as County or Parish is optional).
  • Getters. In every getter below, state names the state the counties are in when county is given, instead of adding its own filter. Passing county together with the getter's own state or county parameter raises ValueError.

    Getter Sent as
    waterdata.get_monitoring_locations, get_combined_metadata state_code + county_code; counties in several states become a filter of OR'd pairs
    waterdata.get_stats_por, get_stats_date_range county_code=US:55:025 (repeated)
    ngwmn.get_sites state_name + county_name
    nwdc.get_wateruse countyCd:55025 (the existing county now accepts every form above)
  • Bug fix (NGWMN): get_sites(county_name=[...]) with more than one name returned no rows, because NGWMN matches nothing for a comma-joined county_name. Multi-value sites filters are now POSTed as CQL2.

  • Docs: ADR 0013 gains a clause allowing a unified argument (an argument dataretrieval names for a concept the services spell differently) under four conditions:

    • the service's own parameters stay;
    • the conversion is an exact table, checked live against the service's reference collection;
    • combining the argument with the native parameter raises;
    • the docstring names the field it is sent as.

    CONTEXT.md defines State and County as domain terms and Unified argument as a core term.

Design choices for the maintainer

  1. Names need a state. County names repeat across states (there are 30 Washington Counties), so a name without state raises. A bare name that matches two counties in one state also raises and lists both. This happens 12 times, e.g. Fairfax County and Fairfax City in Virginia.
  2. Counties in several states, Water Data: these are sent as (state_code='55' AND county_code='025') OR …. Separate state_code and county_code lists would also match code 031 in Wisconsin. The request planner already splits a filter at its top-level OR. The catch is that county from more than one state can't be combined with your own filter, and the call raises instead, with a remedy.
  3. NGWMN takes one state per call. OR'ing pairs needs a CQL text filter, and NGWMN's edge blocks those (HTTP 403).
  4. NGWMN and Connecticut. NGWMN still files Connecticut sites under the eight counties the state replaced with planning regions in 2022, and the regions don't map one-to-one onto the old counties. get_sites(county="09110") therefore raises and points to county_name="Hartford County". NGWMN's Municipality of Anchorage is handled with a one-line alias. A live test fails if NGWMN changes either.
  5. NWDC behavior change: get_wateruse(state=..., county=...) used to raise for naming two locations. Now state qualifies the county. An unknown county, such as Connecticut's retired 09003, raises locally instead of returning HTTP 400.
  6. The table is a separate module, _county_table.py, so that the doubled-word hook can skip it ("Walla Walla County").

Verification

  • Conversions (offline): every one of the 3,239 entries round-trips through each form: full name, lower-case name, 3-digit code with state, 5-digit string, US: form, and integer. Each bare name resolves unless it's one of the 12 ambiguous ones.
  • Live, 69 counties: one random county per state and territory, plus edge cases (Virginia independent cities, Baltimore and St. Louis cities, Alaska boroughs and census areas, a Louisiana parish, Puerto Rico municipios, Connecticut planning regions, DC, Guam, American Samoa, the Northern Mariana Islands, and the US Virgin Islands).
    • Every input form round-trips.
    • get_monitoring_locations returns only rows with the requested state_code, county_code, and county_name.
    • ngwmn.get_sites returns only the requested state_name + county_name, or raises for Connecticut.
    • The statistics service and NWDC accept every code.
    • The NWDC wateruse model has no data for Alaska, Hawaii, or the territories, and responds with HTTP 400; state="AK" does the same today.
  • Live, at scale: county= with all 174 Wisconsin and Illinois counties returns the same 5,136 stream sites as state=["WI", "IL"], in 2 requests.
  • Live tests: the 4 new ones pass: the county table against the Water Data counties collection, NGWMN's county names, and the state table against the states collection (moved here from fix(codes): name the Virgin Islands as the services do #432, because ADR 0013's new clause cites it).
  • Offline: 1272 passed, branch coverage 98.9%+. ruff, mypy --strict, import-linter, xenon, and complexipy pass, and so do the pre-commit hooks. The Sphinx build (notebooks not executed) adds no warnings.

thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Oct 2, 2026
Simplify review of DOI-USGS#433:

- apply_county deduplicates the counties before calling render, so the
  three renderers no longer each repeat list(dict.fromkeys(...)).
- Renderers return lists unconditionally. The single-value collapse was
  unneeded: the request builder already sends a one-element list as a
  plain GET parameter (checked live: same URLs and rows).
- The multi-state-with-filter conflict raises through
  _validation.reject_together, in the package's shared wording.
- The Connecticut hint is keyed on the parsed code rather than a second,
  looser parse of the raw value.
- The location getters' county docstrings name the US:55:025 form as the
  others do.
codes.states named FIPS 78 "US Virgin Islands". Water Data and NGWMN
both filter state_name on "Virgin Islands" and match exactly, so
state="VI" returned no rows from either service: 0 rather than 1,092
monitoring locations, and 0 rather than 3 NGWMN sites. The Census name
is still accepted as input.

The territory tests are consolidated to one row per territory, each
checking that its name, postal code, and FIPS code resolve to one
another; the encodings themselves (case, US: prefix) are Test_to_state's.
county accepts a five-digit FIPS code, the statistics service's
US:SS:CCC, or a name or three-digit code with its state, and sends each
service what it filters on: Water Data state_code + county_code,
statistics county_code=US:SS:CCC, NGWMN state_name + county_name, NWDC
countyCd. With county, state qualifies the counties instead of filtering
on its own. Scoped as state is: the location and statistics getters,
ngwmn.get_sites, and nwdc.get_wateruse; not WQP, NWIS, or Samples, which
have no state argument either, and not get_time_series_metadata or
ngwmn.get_providers, which have no county field.

The table is the Water Data counties collection (3,239 counties in the
states and territories codes.states covers), in its own data module so
the doubled-word hook can skip it ("Walla Walla County"). Every entry
round-trips through every input form, and a live test compares the
table with the collection.

A county code repeats across states, so counties in several states go
to the Water Data location collections as a cql-text filter OR-ing
(state_code, county_code) pairs, which the planner splits at top-level
OR: all 174 Wisconsin and Illinois counties returned the same 5,136
stream sites as state=[WI, IL], in 2 requests. NGWMN's edge refuses
cql-text filters (403), so get_sites takes one state per call.

NGWMN names Anchorage differently and still files Connecticut sites
under the counties the state replaced with planning regions in 2022;
the first is aliased, the second raises with a county_name remedy, and
a live test fails if either changes.

Fixes ngwmn.get_sites(county_name=[...]) returning no rows: NGWMN
matches nothing for a comma-joined county_name, so multi-value sites
filters are now POSTed as CQL2 (checked live for state_name,
monitoring_location_id, agency_code, and county_name).
state and now county are package-named arguments for domain concepts
that ADR 0013 otherwise leaves to each service's spelling. Review of
DOI-USGS#422 read state as hiding the APIs' own parameters, and nothing
recorded why it did not contradict the rule.

ADR 0013 gains a clause stating the four conditions both meet: the
native parameters stay, the conversion is an exact table kept against
the service's reference collection, combining the two raises, and the
docstring names what the argument is sent as. CONTEXT.md defines State
and County as domain terms with each service's spelling, and Unified
argument as a core term.
Simplify review of DOI-USGS#433:

- apply_county deduplicates the counties before calling render, so the
  three renderers no longer each repeat list(dict.fromkeys(...)).
- Renderers return lists unconditionally. The single-value collapse was
  unneeded: the request builder already sends a one-element list as a
  plain GET parameter (checked live: same URLs and rows).
- The multi-state-with-filter conflict raises through
  _validation.reject_together, in the package's shared wording.
- The Connecticut hint is keyed on the parsed code rather than a second,
  looser parse of the raw value.
- The location getters' county docstrings name the US:55:025 form as the
  others do.
ADR 0013's unified-argument clause requires each conversion table to be
checked live against the service's reference collection. The county
table is (counties_test.py); this does the same for codes.states, whose
names Water Data and NGWMN match exactly. It would have caught the
Virgin Islands name fixed in DOI-USGS#432.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant