Skip to content

test: Rebalance parallel test scheduling and trim fixture overhead - #2199

Merged
vdusek merged 1 commit into
masterfrom
test/speed-up-unit-suite
Aug 28, 2026
Merged

test: Rebalance parallel test scheduling and trim fixture overhead#2199
vdusek merged 1 commit into
masterfrom
test/speed-up-unit-suite

Conversation

@vdusek

@vdusek vdusek commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

The unit suite spends most of its wall-clock waiting. On a reference CI job (ubuntu-latest / 3.14, run 33047203526) the main pass does 578 worker-seconds of work in 131.8s across 8 workers and burns 476 of them idle. --dist load seeds each worker with a fixed chunk of consecutive tests and never reassigns, so gw4 was still running _playwright/test_utils.py at 125s with the other seven idle for about a minute. Perfect balance would finish at 72.3s.

Same failure mode as apify/apify-client-python#1037.

Changes

  1. --dist worksteal in addopts so an idle worker takes over tests still queued on a busy one. Nothing in the repo uses xdist_group, and --dist is inert without -n.
  2. proxy and disabled_proxy are session-scoped and sync. Each teardown blocks ~1s on proxy.py's hardcoded selector.select(timeout=1); with 17 servers built and destroyed that is ~17 worker-seconds of pure waiting. The server is a stateless forwarder, so one per session is enough. Sync because a session-scoped async fixture is incompatible with asyncio_default_fixture_loop_scope = "function", and neither ever awaited.
  3. Dropped --numprocesses=1 from the run_alone pass - it paid the full xdist worker bootstrap for a 15-test serial suite.
  4. COVERAGE_CORE=sysmon on unit-tests-cov. Measured on a 1397-test subset: default core +14%, sysmon +0%. Coverage falls back with a warning where sys.monitoring is unavailable, so 3.10 and 3.11 are safe.
  5. test_infinite_scroll_on_dynamic_page waits on the page instead of the clock - four wait_for_timeout(1000) sleeps for a loadMore() that awaits 100ms, on the critical-path worker.

Measured

Full suite, both passes, same machine, two reps each: ~114s -> ~74s (-35%). These runs are without coverage, so change 4 is not reflected. 2456 tests pass on both sides; ruff format, ruff check and ty are clean.

Not in this PR

  • from crawlee.crawlers import HttpCrawler costs 3.51s, of which 1.28s is scikit-learn and scipy, imported eagerly by _adaptive_playwright._rendering_type_predictor (behind the optional adaptive-crawler extra). Every worker pays it, as does every crawlee[all] cold start. Not a one-liner - RenderingTypePredictorState needs LogisticRegression at class-creation time. Worth its own issue.
  • 14 of the 15 matrix jobs compute coverage that is never uploaded; skipping it needs a change in apify/workflows.
  • tests_concurrency: "8" may be 2x oversubscription on 4-vCPU runners; worth one run with nproc echoed and an -n4 comparison.
  • test_http_2 still hits the real internet - it needs an HTTP/2 endpoint the local uvicorn server does not speak.

✍️ Drafted by Claude Code

@vdusek vdusek added t-tooling Issues with this label are in the ownership of the tooling team. adhoc Ad-hoc unplanned task added during the sprint. labels Aug 27, 2026
@vdusek vdusek self-assigned this Aug 27, 2026
@github-actions github-actions Bot added this to the 148th sprint - Tooling team milestone Aug 27, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 27, 2026
@vdusek
vdusek marked this pull request as ready for review August 27, 2026 10:18
@vdusek
vdusek requested a review from Pijukatel August 27, 2026 10:18
@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.73%. Comparing base (24a4bc5) to head (79be4fd).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2199      +/-   ##
==========================================
- Coverage   93.73%   93.73%   -0.01%     
==========================================
  Files         181      181              
  Lines       12825    12825              
==========================================
- Hits        12022    12021       -1     
- Misses        803      804       +1     
Flag Coverage Δ
unit 93.73% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek merged commit 1d4007e into master Aug 28, 2026
36 checks passed
@vdusek
vdusek deleted the test/speed-up-unit-suite branch August 28, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants