Skip to content

ADR 0014: declare how each service departs from its protocol as data - #448

Draft
thodson-usgs wants to merge 3 commits into
DOI-USGS:mainfrom
thodson-usgs:refactor/date-range
Draft

thodson-usgs wants to merge 3 commits into
DOI-USGS:mainfrom
thodson-usgs:refactor/date-range

Conversation

@thodson-usgs

@thodson-usgs thodson-usgs commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Implements ADR 0014 (proposed, added here) in three commits, one per step. Review commit by commit. This PR replaces the single-branch draft #445, which conflicted with main in 10 files after #442 and #443. It also absorbs #449 and #450, which held steps 2 and 3.

ADR 0014 records how each service departs from its protocol's standard as dialect fields, whose defaults are the standard. Shared code names no service. A date argument is read once and written by each protocol.

Commit 1: read date arguments once and write them per protocol (1b849c83)

  • dataretrieval._dates (a dependency-free leaf) reads a date argument into a DateRange: an instant, a range with None for an open bound, or a duration. Every bound is UTC; a naive value is UTC, as in pandas (fix(ogc): read each side of a "start/end" interval string like the list form #443).
  • dataretrieval._intervals writes a DateRange in the ISO 8601 interval grammar that OGC API Features and STAC share. A DateDialect with two fields adjusts it:
    • open_start: .. or an empty string;
    • durations: whether a lone duration is accepted, False by default.
  • Dialects:
    • OgcDialect.open_start becomes OgcDialect.dates.
    • Water Data sets durations=True.
    • NGWMN sets open_start="".
    • get_ratings uses STAC_DATES and drops its separate duration check.
  • Removed: ogc/dates.py.
  • Behavior change: a lone duration passed to an ngwmn getter raises ValueError before sending, instead of returning HTTP 400. get_ratings still rejects one, now with the shared message.
  • Otherwise unchanged: 1432 inputs compared against main's _format_api_dates under UTC and America/Chicago gave the same output and messages.

Commit 2: declare date parameters on the dialect (a5e12be8)

  • OgcDialect:
    • New field date_params, defaulting to {"datetime"}.
    • New field timed_params.
    • date_only_services is renamed date_only_collections.
    • The Water Data dialect lists its seven date parameters and timed_params={"last_modified"}. Before this, the request builder and the chunk planner named those parameters.
  • When dates are read: prepare_request_args(..., dialect=) reads date parameters when the getter is called. get_ogc_data reads the ones still raw in a get_reference_table(query=...) dict, and raises TypeError for a DateRange under a key the dialect doesn't list. Samples and Statistics pass no dialect.
  • New architecture test: fails if a string a shipped dialect lists appears as a literal in ogc/ or _intervals.py.
  • Behavior: the values sent are unchanged. An unreadable date raises when the getter is called, with the same message as before.

Commit 3: Service and Endpoint (e245f426)

  • dataretrieval._service defines two values:
    • Service: the name settings are resolved by, and the default URL;
    • Endpoint: a service, the URL resolved when a call starts, and a dialect.
  • Service declarations: each adapter declares its Service once. The 18 adapter="<name>" literals and each Configuration.adapter read the name from it, and configuration.service_url(service) resolves its URL.
  • get_ogc_data takes endpoint= in place of base_url=, dialect= and adapter=.
  • New architecture test: fails if a service name appears as a literal outside its declaration and the configuration roster.
  • CONTEXT.md defines Service declaration and Endpoint.
  • Behavior: no change.

Not carried over from #445

  • grouping the result-shaping fields into a ShapingDialect;
  • an Endpoint for the STAC search, or for adapters with no shared protocol code;
  • the ADR's Protocol and Execution rows and its revision log;
  • the DateArgument annotation on the getters, which could be a later follow-up.

Tests

NEWS is dated 10/12; I'll re-date it to the day this lands. ADR 0014 becomes Accepted when this merges.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK

thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Oct 12, 2026
…n entry (ADR 0014)

Step 2 of ADR 0014 (proposed), stacked on step 1 (DOI-USGS#448).

- OgcDialect gains date_params (default {"datetime"}, the standard's one
  date parameter) and timed_params, and date_only_services is renamed
  date_only_collections, since it is keyed by collection. The Water Data
  dialect lists its seven date parameters and timed_params
  {"last_modified"}, which the request builder used to name.
- prepare_request_args(..., dialect=) reads the dialect's date parameters
  into a DateRange when the getter is called and drops an all-open one.
  Water Data's _get_args passes WATERDATA_DIALECT (the Samples and
  Statistics getters pass None, as those APIs are not OGC); NGWMN passes
  NGWMN_DIALECT.
- get_ogc_data reads any date parameter still raw (a verbatim query from
  get_reference_table), and raises TypeError for a DateRange under a key
  the dialect does not list.
- The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange
  is not a list. _DATE_RANGE_PARAMS is removed.
- tests/architecture_test.py fails if a string a shipped dialect lists
  appears as a literal in ogc/ or _intervals.py. It found one: the
  "value" default of _normalize_str_iterable's param_name, now required.

The values sent are unchanged. An unreadable date now raises when the
getter is called rather than when the first request is built; the
message is the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
thodson-usgs added a commit to thodson-usgs/dataretrieval-python that referenced this pull request Oct 12, 2026
…n entry (ADR 0014)

Step 2 of ADR 0014 (proposed), stacked on step 1 (DOI-USGS#448).

- OgcDialect gains date_params (default {"datetime"}, the standard's one
  date parameter) and timed_params, and date_only_services is renamed
  date_only_collections, since it is keyed by collection. The Water Data
  dialect lists its seven date parameters and timed_params
  {"last_modified"}, which the request builder used to name.
- prepare_request_args(..., dialect=) reads the dialect's date parameters
  into a DateRange when the getter is called and drops an all-open one.
  Water Data's _get_args passes WATERDATA_DIALECT (the Samples and
  Statistics getters pass None, as those APIs are not OGC); NGWMN passes
  NGWMN_DIALECT.
- get_ogc_data reads any date parameter still raw (a verbatim query from
  get_reference_table), and raises TypeError for a DateRange under a key
  the dialect does not list.
- The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange
  is not a list. _DATE_RANGE_PARAMS is removed.
- tests/architecture_test.py fails if a string a shipped dialect lists
  appears as a literal in ogc/ or _intervals.py. It found one: the
  "value" default of _normalize_str_iterable's param_name, now required.

The values sent are unchanged. An unreadable date now raises when the
getter is called rather than when the first request is built; the
message is the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
thodson-usgs and others added 3 commits October 12, 2026 16:12
… (ADR 0014)

Step 1 of ADR 0014 (proposed), which replaces the single-branch draft in
DOI-USGS#445. Adds the ADR.

- dataretrieval._dates (a dependency-free leaf) reads a date argument
  into a DateRange: an instant, a range with None for an open bound, or
  a duration. Every bound is UTC; a naive value is UTC, as in pandas.
- dataretrieval._intervals writes a DateRange in the ISO 8601 interval
  grammar that OGC API Features and STAC share, adjusted by a
  DateDialect(open_start, durations). It sits below ogc because STAC is
  not OGC.
- OgcDialect.open_start becomes OgcDialect.dates. Water Data sets
  durations=True; NGWMN sets open_start=""; get_ratings uses STAC_DATES
  (open_start="").
- ogc/dates.py and get_ratings' separate duration check are removed.

Behavior change: a lone duration passed to an NGWMN getter raises
ValueError before sending; the service rejects one with HTTP 400.
get_ratings' duration message is now the shared one. Otherwise the
values sent are unchanged: a comparison of 1432 inputs (each element
type, every pair, interval strings, error cases) against main's
_format_api_dates found no difference in output or message, under UTC
and America/Chicago.

Live tests check each DateDialect field against the Water Data OGC,
NGWMN, and STAC services; tests/_dates_test.py covers each DateRange
shape read_date_range returns.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
…n entry (ADR 0014)

Step 2 of ADR 0014 (proposed).

- OgcDialect gains date_params (default {"datetime"}, the standard's one
  date parameter) and timed_params, and date_only_services is renamed
  date_only_collections, since it is keyed by collection. The Water Data
  dialect lists its seven date parameters and timed_params
  {"last_modified"}, which the request builder used to name.
- prepare_request_args(..., dialect=) reads the dialect's date parameters
  into a DateRange when the getter is called and drops an all-open one.
  Water Data's _get_args passes WATERDATA_DIALECT (the Samples and
  Statistics getters pass None, as those APIs are not OGC); NGWMN passes
  NGWMN_DIALECT.
- get_ogc_data reads any date parameter still raw (a verbatim query from
  get_reference_table), and raises TypeError for a DateRange under a key
  the dialect does not list.
- The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange
  is not a list. _DATE_RANGE_PARAMS is removed.
- tests/architecture_test.py fails if a string a shipped dialect lists
  appears as a literal in ogc/ or _intervals.py. It found one: the
  "value" default of _normalize_str_iterable's param_name, now required.

The values sent are unchanged. An unreadable date now raises when the
getter is called rather than when the first request is built; the
message is the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
…dpoint (ADR 0014)

Step 3 of ADR 0014 (proposed).

- dataretrieval._service (a dependency-free leaf) defines Service (the
  adapter's name, which settings are resolved by, and its default URL)
  and Endpoint (a service, the URL resolved when a call starts, and a
  dialect).
- Each adapter declares its Service once and reads the name from it:
  the Configuration class's adapter attribute, retry and fan-out
  adapter= arguments, and configuration.service_url(service), which
  replaces base_url(adapter=..., default=...) at each URL read site.
  Water Data's SERVICE is in waterdata.endpoints, shared by its four
  APIs.
- get_ogc_data takes endpoint= in place of base_url=, dialect=, and
  adapter=. The request builder and finalizer it binds keep their
  URL and dialect arguments.
- tests/architecture_test.py fails if a service name appears as a
  literal outside its declaration and the configuration roster; the
  read-site test in configuration_test.py looks for <declaration>.name
  or Endpoint(<declaration>, ...).
- CONTEXT.md defines Service declaration and Endpoint, and generalizes
  Dialect.

No behavior change. Checked live: Water Data daily, continuous, ratings,
and a reference table with a date in its query; NGWMN water levels with
an open start; WQP results; and an NgwmnConfiguration base_url redirect.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
@thodson-usgs thodson-usgs changed the title refactor(dates): read date arguments once and write them per protocol (ADR 0014, step 1) ADR 0014: declare how each service departs from its protocol as data Oct 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant