Skip to content

test: deflake test_stall_waits_for_in_flight_work - #2355

Merged
vdusek merged 2 commits into
masterfrom
test/deflake-stall-in-flight
Oct 26, 2026
Merged

vdusek merged 2 commits into
masterfrom
test/deflake-stall-in-flight

Conversation

@vdusek

@vdusek vdusek commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

test_stall_waits_for_in_flight_work flaked on macOS / 3.11 (run) with DID NOT RAISE PersistentRateLimitError. The test used a 50-100 ms real-time backoff. On a slow runner it ran out before the crawler checked whether the domain was held back, so every attempt counted as an ordinary error, the request failed, and no stall was ever raised.

The test now runs the throttler on a stopped clock, like open_throttler_held_until_reclaim and #2331. Each reclaim moves the clock one backoff forward until the domain has been rate-limited longer than max_domain_stall. After that the clock stays put, so the request waits in its cooldown as a stalled domain and doesn't cycle into MAX_HANDLER_CALLS.

Local runs couldn't reproduce the flake on their own. Delaying is_throttled by 120 ms reproduces the CI failure every time. With that delay the fixed test passes, and it passed 128 of 128 runs under CPU oversubscription. It still fails if the crawler raises a stall while a request is in flight.

✍️ Drafted by Claude Code

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Oct 26, 2026
@vdusek vdusek self-assigned this Oct 26, 2026
@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.14%. Comparing base (449b82b) to head (ab1e102).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2355      +/-   ##
==========================================
+ Coverage   94.10%   94.14%   +0.03%     
==========================================
  Files         184      184              
  Lines       13647    13647              
==========================================
+ Hits        12843    12848       +5     
+ Misses        804      799       -5     
Flag Coverage Δ
unit 94.14% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek requested a review from Mantisus October 26, 2026 07:32
@vdusek
vdusek marked this pull request as ready for review October 26, 2026 07:33
@apify-service-account apify-service-account added the tested Temporary label used only programatically for some analytics. label Oct 26, 2026

@Mantisus Mantisus left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@apify-approval-mirror apify-approval-mirror Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mirroring approval from @Mantisus.

@vdusek
vdusek merged commit 5776553 into master Oct 26, 2026
47 checks passed
@vdusek
vdusek deleted the test/deflake-stall-in-flight branch October 26, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants