Repository navigation
ADR 0014: declare how each service departs from its protocol as data - #448
Draft
thodson-usgs wants to merge 3 commits into
Draft
thodson-usgs wants to merge 3 commits into
thodson-usgs wants to merge 3 commits into
Conversation
This was referenced Oct 12, 2026
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Oct 12, 2026
…n entry (ADR 0014) Step 2 of ADR 0014 (proposed), stacked on step 1 (DOI-USGS#448). - OgcDialect gains date_params (default {"datetime"}, the standard's one date parameter) and timed_params, and date_only_services is renamed date_only_collections, since it is keyed by collection. The Water Data dialect lists its seven date parameters and timed_params {"last_modified"}, which the request builder used to name. - prepare_request_args(..., dialect=) reads the dialect's date parameters into a DateRange when the getter is called and drops an all-open one. Water Data's _get_args passes WATERDATA_DIALECT (the Samples and Statistics getters pass None, as those APIs are not OGC); NGWMN passes NGWMN_DIALECT. - get_ogc_data reads any date parameter still raw (a verbatim query from get_reference_table), and raises TypeError for a DateRange under a key the dialect does not list. - The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange is not a list. _DATE_RANGE_PARAMS is removed. - tests/architecture_test.py fails if a string a shipped dialect lists appears as a literal in ogc/ or _intervals.py. It found one: the "value" default of _normalize_str_iterable's param_name, now required. The values sent are unchanged. An unreadable date now raises when the getter is called rather than when the first request is built; the message is the same. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
thodson-usgs
added a commit
to thodson-usgs/dataretrieval-python
that referenced
this pull request
Oct 12, 2026
…n entry (ADR 0014) Step 2 of ADR 0014 (proposed), stacked on step 1 (DOI-USGS#448). - OgcDialect gains date_params (default {"datetime"}, the standard's one date parameter) and timed_params, and date_only_services is renamed date_only_collections, since it is keyed by collection. The Water Data dialect lists its seven date parameters and timed_params {"last_modified"}, which the request builder used to name. - prepare_request_args(..., dialect=) reads the dialect's date parameters into a DateRange when the getter is called and drops an all-open one. Water Data's _get_args passes WATERDATA_DIALECT (the Samples and Statistics getters pass None, as those APIs are not OGC); NGWMN passes NGWMN_DIALECT. - get_ogc_data reads any date parameter still raw (a verbatim query from get_reference_table), and raises TypeError for a DateRange under a key the dialect does not list. - The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange is not a list. _DATE_RANGE_PARAMS is removed. - tests/architecture_test.py fails if a string a shipped dialect lists appears as a literal in ogc/ or _intervals.py. It found one: the "value" default of _normalize_str_iterable's param_name, now required. The values sent are unchanged. An unreadable date now raises when the getter is called rather than when the first request is built; the message is the same. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
… (ADR 0014) Step 1 of ADR 0014 (proposed), which replaces the single-branch draft in DOI-USGS#445. Adds the ADR. - dataretrieval._dates (a dependency-free leaf) reads a date argument into a DateRange: an instant, a range with None for an open bound, or a duration. Every bound is UTC; a naive value is UTC, as in pandas. - dataretrieval._intervals writes a DateRange in the ISO 8601 interval grammar that OGC API Features and STAC share, adjusted by a DateDialect(open_start, durations). It sits below ogc because STAC is not OGC. - OgcDialect.open_start becomes OgcDialect.dates. Water Data sets durations=True; NGWMN sets open_start=""; get_ratings uses STAC_DATES (open_start=""). - ogc/dates.py and get_ratings' separate duration check are removed. Behavior change: a lone duration passed to an NGWMN getter raises ValueError before sending; the service rejects one with HTTP 400. get_ratings' duration message is now the shared one. Otherwise the values sent are unchanged: a comparison of 1432 inputs (each element type, every pair, interval strings, error cases) against main's _format_api_dates found no difference in output or message, under UTC and America/Chicago. Live tests check each DateDialect field against the Water Data OGC, NGWMN, and STAC services; tests/_dates_test.py covers each DateRange shape read_date_range returns. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
…n entry (ADR 0014)
Step 2 of ADR 0014 (proposed).
- OgcDialect gains date_params (default {"datetime"}, the standard's one
date parameter) and timed_params, and date_only_services is renamed
date_only_collections, since it is keyed by collection. The Water Data
dialect lists its seven date parameters and timed_params
{"last_modified"}, which the request builder used to name.
- prepare_request_args(..., dialect=) reads the dialect's date parameters
into a DateRange when the getter is called and drops an all-open one.
Water Data's _get_args passes WATERDATA_DIALECT (the Samples and
Statistics getters pass None, as those APIs are not OGC); NGWMN passes
NGWMN_DIALECT.
- get_ogc_data reads any date parameter still raw (a verbatim query from
get_reference_table), and raises TypeError for a DateRange under a key
the dialect does not list.
- The chunk planner's _NEVER_CHUNK lists no date parameter: a DateRange
is not a list. _DATE_RANGE_PARAMS is removed.
- tests/architecture_test.py fails if a string a shipped dialect lists
appears as a literal in ogc/ or _intervals.py. It found one: the
"value" default of _normalize_str_iterable's param_name, now required.
The values sent are unchanged. An unreadable date now raises when the
getter is called rather than when the first request is built; the
message is the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
…dpoint (ADR 0014) Step 3 of ADR 0014 (proposed). - dataretrieval._service (a dependency-free leaf) defines Service (the adapter's name, which settings are resolved by, and its default URL) and Endpoint (a service, the URL resolved when a call starts, and a dialect). - Each adapter declares its Service once and reads the name from it: the Configuration class's adapter attribute, retry and fan-out adapter= arguments, and configuration.service_url(service), which replaces base_url(adapter=..., default=...) at each URL read site. Water Data's SERVICE is in waterdata.endpoints, shared by its four APIs. - get_ogc_data takes endpoint= in place of base_url=, dialect=, and adapter=. The request builder and finalizer it binds keep their URL and dialect arguments. - tests/architecture_test.py fails if a service name appears as a literal outside its declaration and the configuration roster; the read-site test in configuration_test.py looks for <declaration>.name or Endpoint(<declaration>, ...). - CONTEXT.md defines Service declaration and Endpoint, and generalizes Dialect. No behavior change. Checked live: Water Data daily, continuous, ratings, and a reference table with a date in its query; NGWMN water levels with an open start; WQP results; and an NgwmnConfiguration base_url redirect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK
thodson-usgs
force-pushed
the
refactor/date-range
branch
from
October 12, 2026 21:15
585100a to
e245f42
Compare
This was referenced Oct 13, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements ADR 0014 (proposed, added here) in three commits, one per step. Review commit by commit. This PR replaces the single-branch draft #445, which conflicted with
mainin 10 files after #442 and #443. It also absorbs #449 and #450, which held steps 2 and 3.ADR 0014 records how each service departs from its protocol's standard as dialect fields, whose defaults are the standard. Shared code names no service. A date argument is read once and written by each protocol.
Commit 1: read date arguments once and write them per protocol (
1b849c83)dataretrieval._dates(a dependency-free leaf) reads a date argument into aDateRange: an instant, a range withNonefor an open bound, or a duration. Every bound is UTC; a naive value is UTC, as in pandas (fix(ogc): read each side of a "start/end" interval string like the list form #443).dataretrieval._intervalswrites aDateRangein the ISO 8601 interval grammar that OGC API Features and STAC share. ADateDialectwith two fields adjusts it:open_start:..or an empty string;durations: whether a lone duration is accepted,Falseby default.OgcDialect.open_startbecomesOgcDialect.dates.durations=True.open_start="".get_ratingsusesSTAC_DATESand drops its separate duration check.ogc/dates.py.ngwmngetter raisesValueErrorbefore sending, instead of returning HTTP 400.get_ratingsstill rejects one, now with the shared message.main's_format_api_datesunder UTC and America/Chicago gave the same output and messages.Commit 2: declare date parameters on the dialect (
a5e12be8)OgcDialect:date_params, defaulting to{"datetime"}.timed_params.date_only_servicesis renameddate_only_collections.timed_params={"last_modified"}. Before this, the request builder and the chunk planner named those parameters.prepare_request_args(..., dialect=)reads date parameters when the getter is called.get_ogc_datareads the ones still raw in aget_reference_table(query=...)dict, and raisesTypeErrorfor aDateRangeunder a key the dialect doesn't list. Samples and Statistics pass no dialect.ogc/or_intervals.py.Commit 3:
ServiceandEndpoint(e245f426)dataretrieval._servicedefines two values:Service: the name settings are resolved by, and the default URL;Endpoint: a service, the URL resolved when a call starts, and a dialect.Serviceonce. The 18adapter="<name>"literals and eachConfiguration.adapterread the name from it, andconfiguration.service_url(service)resolves its URL.get_ogc_datatakesendpoint=in place ofbase_url=,dialect=andadapter=.CONTEXT.mddefines Service declaration and Endpoint.Not carried over from #445
ShapingDialect;Endpointfor the STAC search, or for adapters with no shared protocol code;DateArgumentannotation on the getters, which could be a later follow-up.Tests
DateDialectfield against the real Water Data OGC, NGWMN and STAC services. All pass.NgwmnConfiguration(base_url=...)redirect.NEWS is dated 10/12; I'll re-date it to the day this lands. ADR 0014 becomes Accepted when this merges.
🤖 Generated with Claude Code
https://claude.ai/code/session_01HBt3Ee7eneKtdKQWq2T3YK