Skip to content

Preserve sync HTTP pools across discovered cluster nodes - #161

Merged
dkropachev merged 7 commits into
mainfrom
fix/sync-host-pool-cache
Oct 27, 2026
Merged

dkropachev merged 7 commits into
mainfrom
fix/sync-host-pool-cache

Conversation

@dkropachev

Copy link
Copy Markdown
Collaborator

Summary

Sync clients and resources could open a fresh TCP connection on every request when cycling through more than ten nodes because botocore leaves urllib3's host-pool cache at its default size. Grow the existing cache to cover seed, routing, and key-affinity nodes, including topology changes and lazily created proxy managers.

Keep max_pool_connections as the per-host connection limit and preserve the existing transport's TLS, proxy, retry, and timeout settings. Regression coverage checks that two passes through eleven HTTP endpoints reuse eleven TCP connections.

Fixes #159. Synchronous discovery pooling is tracked separately in #160.

Botocore leaves urllib3's host cache at ten entries even when Alternator
routes across a larger cluster. Grow each client's existing cache from
its seed, routing and affinity snapshots so revisiting healthy nodes
reuses their connections.

Cover resources, topology growth, lazy proxy managers and TCP reuse
without changing the per-host connection setting.

Fixes #159
@github-actions github-actions Bot added bug Something isn't working docs test labels Oct 26, 2026
@qodo-scylladb

qodo-scylladb Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Code Review by Qodo


Resolved findings
1. Topology changes can evict live pools ✓ Resolved
Description
grow_pools sizes the cache from current node snapshots but does not count the destination already
selected for the request being sent. If discovery replaces an eleven-node snapshot while a request
using its earlier query plan is in flight, that old destination can consume one of the eleven slots
needed by current nodes and evict a reusable pool.
Code

alternator/_sync_sdk.py[R36-39]

+    def grow_pools(**kwargs: object) -> None:
+        hosts = set(config.seed_hosts)
+        hosts.update(manager.nodes.nodes)
+        hosts.update(manager.affinity_nodes.nodes)
Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The before-send cache size excludes a destination selected from a previous topology snapshot. An in-flight request to that destination can evict a pool belonging to the current snapshot.

## Fix Focus Areas
- alternator/_sync_sdk.py[36-51]
- tests/unit/test_sync_connection_pools.py[26-64]

## Recommended Fix
Include the before-send request’s destination in the host set used for sizing. Add a test that changes topology after routing selects a host but before that request is sent.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Concurrent proxy requests can fail ✓ Resolved
Description
grow_pools iterates _proxy_managers.values() without synchronizing with lazy proxy-manager
creation. When concurrent first requests use a proxy, insertion during that iteration can raise
RuntimeError; a manager created after the snapshot can also serve a burst with the default
ten-host cache.
Code

alternator/_sync_sdk.py[42]

+        pool_managers = [transport._manager, *transport._proxy_managers.values()]
Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Proxy managers are created lazily after the before-send callback, while the callback iterates their registry without synchronization. Concurrent initial requests can race with that iteration or use an unsized manager.

## Fix Focus Areas
- alternator/_sync_sdk.py[40-51]
- tests/unit/test_sync_connection_pools.py[26-64]

## Recommended Fix
Arrange for each proxy manager to be sized when it is created, with access to the manager registry synchronized against sizing. Add coverage for concurrent initial proxy requests rather than only a proxy manager created before the event.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗


Powered by Qodo

Describe discovery separately from its connection lifecycle so the data
pool documentation also applies when discovery pooling is introduced.

Refs #159
Count a request's destination when its captured query plan outlives the
current topology. Select and grow its SDK pool manager under a client-local
lock before sending, so concurrent first proxy requests share a sized
manager without iterating the lazy manager registry.

Cover topology handoff and concurrent proxy first use for clients and
resources, and verify reconnects after a server closes a connection.

Refs #159
A captured request plan can visit several retired destinations after a
topology update. Counting only one selected destination still allows those
pools to evict reusable pools belonging to the current snapshots.

At the SDK's actual cache insertion, grow capacity for known nodes and
promote current entries so normal eviction prefers retired pools. Delegate
lookup and disposal to the existing manager under its cache lock, and keep
cached requests on the normal path. Normalize host keys through urllib3.

Cover repeated handoffs and concurrent retired destinations for direct
and HTTPS proxy managers without retaining an unlimited host history.

Refs #159
Comment thread alternator/_sync_sdk.py Fixed
Read topology snapshots only after acquiring the SDK cache lock on a miss,
so an older waiting lookup cannot evict pools for a replacement topology.

Coordinate lazy proxy creation through the SDK's original getter instead
of before-send. This keeps proxy preparation inside the SDK's exception
mapping and standard retry path while retaining its managers and settings.

Add focused regressions for overlapping topology replacement, malformed
unselected hosts, and proxy failure classification and retries.

Refs #159
@dkropachev

Copy link
Copy Markdown
Collaborator Author

In reply to

Include the before-send request’s destination in the host set used for sizing.

Cache misses now account for the selected destination and protect pools for the seed and observed routing/affinity snapshots. Snapshot reads, recency promotion, and insertion share the cache lock. Regressions cover topology replacement after routing, waiting lookups, and repeated retired destinations with direct and HTTPS proxy managers.

Arrange for each proxy manager to be sized when it is created, with access to the manager registry synchronized against sizing.

The original SDK proxy getter now serializes lazy construction and installs the sizing hook before the manager’s first pool lookup, without iterating the registry. Added concurrent first-request coverage for clients and resources, plus a regression preserving SDK proxy-error mapping and standard retries. The per-host connection limit is unchanged.

urllib3 preserves IPv6 zone case while parsing URLs but lowercases host
names in pool keys. Apply the same normalization to protected hosts so
retired destinations cannot evict a current scoped IPv6 pool.

Extend the existing retention regression with this case and remove obsolete
synthetic before-send setup from the pool tests.

Refs #159
Move topology-handoff and concurrent pool setup inside each test's client
cleanup scope. Pool lookups, proxy construction or lock patching can fail
before the assertions, and those failures must still close the client.

Keep barrier and event cleanup scoped to their initialized setup while
preserving the existing assertions and concurrent schedules.

Refs #159
@dkropachev
dkropachev merged commit d625b93 into main Oct 27, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working docs test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant