Skip to content

fix(codes): reconcile time-zone table with USGS and WQX code lists - #456

Merged
thodson-usgs merged 2 commits into
DOI-USGS:mainfrom
thodson-usgs:fix/timezone-code-lists
Oct 16, 2026
Merged

thodson-usgs merged 2 commits into
DOI-USGS:mainfrom
thodson-usgs:fix/timezone-code-lists

Conversation

@thodson-usgs

@thodson-usgs thodson-usgs commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Follows up #455 (merged); rebased onto main as a single commit.

Summary

codes.timezones turns the abbreviation beside a local time into the UTC offset used for the *DateTime columns that waterdata.get_samples() and wqp.get_results() append. The table now follows the code lists those services use: the USGS NWIS time-zone list (Samples) and the WQX TimeZone domain (WQP).

  • American Samoa in Samples: Samples writes this zone as ZP-11, which the table lacked, so every timed American Samoa row came back with a NaT datetime. It is now UTC-11.
  • Codes that used to become NaT: 39 more codes from the two lists now resolve: the other ZP* offsets, IDLW/IDLE, NZT, SWT, the retired WQX codes AHST and YST, and others.
  • Behavior change: codes the table read differently from the USGS list now use the USGS meaning: CAST +9:30, CCT +8, EASST +11, EAST +10, NFT -3:30, NZDT +13, WAST +7, WAT -1. FJST moves to +13 too.
  • Where the two lists conflict, WQX wins: SST (Samoa in WQX, Swedish Summer Time in USGS) stays UTC-11, the meaning WQP returns. BST becomes UTC-11. Samples doesn't use either code for Samoa; it writes ZP-11.
  • Snapshots: copies of both lists are in tests/data/, and the module docstring points to them.

Tests

  • tests/timezones_test.py, offline: checks that every code in each list maps to that list's offset, and that WQX_WINS == {"SST", "BST"} covers every conflict between the lists, so any new conflict has to be decided explicitly.
  • tests/timezones_test.py, live: checks that the published lists still match the snapshots, and that Samples/WQP responses for Pacific monitoring locations have no NaT among timed rows.
  • utils_test.py: an end-to-end ZP-11 case through _attach_datetime_columns.
  • With the old table, 50 of the new tests fail; with the new one, all pass.
  • Gates: ruff, mypy, lint-imports, xenon, complexipy; 1757 tests pass; coverage 99.08%.

Live verification (2026-10-16)

Service Monitoring location / query Code Timed rows with NaT
Samples USGS-141037169391201 (Am. Samoa), 566 rows ZP-11 0 (all were NaT before)
Samples USGS-16854500 (Guam), 969 timed GST 0
WQP legacy statecode=US:60, since 2023 SST 0 of 591
WQP legacy statecode=US:66, since 2023 GST 0 of 1099
WQP WQX3 11NPSWRD_WQX-NPSA_F_07TUT_FW SST 0 of 846
WQP WQX3 USGS-16854500 GST 0 of 969

Example: Samples 2011-09-09 09:40 ZP-11 → 2011-09-09 20:40Z.

Not done

  • Silent NaT: an unrecognized code still becomes NaT without any warning.
  • One table for both services: splitting into a separate table per service is deferred until a USGS response is seen using SST or BST with its USGS meaning.

Stress test (2026-10-16)

wqp.what_activities plus _attach_datetime_columns and waterdata.get_samples(service="activities") for all 50 states, DC, and five territories, over four windows (Jan–Mar 1995, Jul 2010, Jun 2024, Jan 2025): about 520,000 timed rows and 25 distinct codes.

  • No timed row became NaT.
  • Every resolved offset falls within ±1 h of its state's zones, apart from UTC/GMT stamps, two Alaska rows stamped AST/ADT, and one Idaho well (USGS-433812111574701) whose ten activities are all stamped WAST. That code is wrong in the source record whatever the table does; the old table put it at +2, the USGS list at +7.
  • WQP passes USGS codes through: ZP-11 appears in WQP results for USGS American Samoa records, so this fix reaches wqp.get_results() too. A live case now covers it, and the docstring and NEWS say so.

…lists

Samples writes American Samoa as ZP-11, which the table lacked, so every
timed American Samoa row from get_samples() had a NaT datetime. Follow the
USGS NWIS list (Samples) and the WQX TimeZone domain (WQP); where they
disagree (SST, BST), WQX wins. Snapshots of both lists back an offline
test, and live tests watch the published lists and responses for Pacific
monitoring locations.
@thodson-usgs
thodson-usgs force-pushed the fix/timezone-code-lists branch from cdf4480 to 6d3824a Compare October 16, 2026 15:50
A live sweep of every state and territory found ZP-11 in WQP responses for
USGS American Samoa records, so the USGS list governs part of WQP too.
@thodson-usgs
thodson-usgs marked this pull request as ready for review October 16, 2026 16:37
@thodson-usgs
thodson-usgs merged commit 776c8b0 into DOI-USGS:main Oct 16, 2026
11 checks passed
@thodson-usgs
thodson-usgs deleted the fix/timezone-code-lists branch October 16, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant