Add a refcounted thread pool keep-awake API

[alexreinking]

Adds a refcounted switch that keeps the default thread pool's idle workers spinning instead of sleeping, the shape agreed in maintainer discussion (a refcounted bit rather than a spin count). Without it, back-to-back small parallel loops with serial work between them pay a thread wake-up every time.

• int halide_thread_pool_keep_awake(bool) acquires or releases a reference and returns the new count. While it's held, every thread waiting on the pool's spinning condition variables (A team, B team, and owners waiting on their own loops) keeps spinning instead of sleeping. The A/B team logic is unchanged. Workers blocked on async semaphores still sleep. An unbalanced release is an error.
• Backstop: an idle thread parks anyway after 4096 polls without work (a few ms), which bounds the cost of a leaked holder.
• RAII holders: Halide::Runtime::ThreadPoolKeepAwake (AOT, header-only), Halide::ThreadPoolKeepAwake (JIT) and hl.ThreadPoolKeepAwake (Python, with).
• Behavior is unchanged while no reference is held, and custom thread pools are unaffected.

Future work: the remaining per-loop cost is the work queue's global mutex; a lock-free do_par_for fast path would be a separate change.

Tests: new generator_aot_thread_pool_keep_awake, correctness_thread_pool_keep_awake and python_correctness_thread_pool_keep_awake; performance_thread_pool_keep_awake reports timings.

Fork/join time per call with a 100 µs gap between loops (performance_thread_pool_keep_awake, M3 Max, median of 3):

threads	default	keep-awake
8	10.8 µs	1.5 µs
12	30.7 µs	0.8 µs
16	0.6 µs	0.5 µs

---

Authored by GitHub Copilot

[abadams]

It's not clear to me why any thread pool logic at all needs to change beyond how the spin count is acquired. A too-large A team is very bad for performance, due to thundering herd issues. I don't think the A-team/B-team logic should be touched, I just think the threads waiting on the B team condition variable should be spinning inside the b team condition variable's wait method instead of sleeping. All the condition variables used support spinning, so just let them spin.

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.98%. Comparing base (ee9f17a) to head (534821f).

Files with missing lines	Patch %	Lines
src/JITModule.cpp	90.00%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9526      +/-   ##
==========================================
- Coverage   71.05%   70.98%   -0.08%     
==========================================
  Files         262      262              
  Lines       81084    81114      +30     
  Branches    19770    19777       +7     
==========================================
- Hits        57615    57575      -40     
- Misses      17502    17505       +3     
- Partials     5967     6034      +67     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[alexreinking]

Done in 534821f. The A/B team logic is untouched now. While the count is held, halide_cond_with_spinning::wait just keeps spinning, up to the 4096-poll backstop, instead of sleeping. That applies to every waiter: A team, B team and owners. I dropped the per-team caps and the wake-up broadcast on the first reference, so the only change to the pool is the spin loop's bound. The numbers in the description are re-measured; keep-awake is unchanged at 0.5–1.5 µs per fork/join.

---

Authored by GitHub Copilot