feat(pyamber): truncate range double keys - #8256
Conversation
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 420 | 0.257 | 23,059/29,121/29,121 us | 🔴 -10.3% / 🔴 +86.9% |
| 🟢 | bs=100 sw=10 sl=64 | 949 | 0.579 | 105,590/123,923/123,923 us | 🟢 -14.9% / 🔴 +15.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,078 | 0.658 | 928,554/1,002,764/1,002,764 us | ⚪ within ±5% / 🟢 -5.4% |
Baseline details
Latest main 50321e4 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 420 tuples/sec | 468 tuples/sec | 787.98 tuples/sec | -10.3% | -46.7% |
| bs=10 sw=10 sl=64 | MB/s | 0.257 MB/s | 0.286 MB/s | 0.481 MB/s | -10.1% | -46.6% |
| bs=10 sw=10 sl=64 | p50 | 23,059 us | 22,434 us | 12,593 us | +2.8% | +83.1% |
| bs=10 sw=10 sl=64 | p95 | 29,121 us | 28,058 us | 15,579 us | +3.8% | +86.9% |
| bs=10 sw=10 sl=64 | p99 | 29,121 us | 28,058 us | 18,786 us | +3.8% | +55.0% |
| bs=100 sw=10 sl=64 | throughput | 949 tuples/sec | 931 tuples/sec | 1,008 tuples/sec | +1.9% | -5.8% |
| bs=100 sw=10 sl=64 | MB/s | 0.579 MB/s | 0.568 MB/s | 0.615 MB/s | +1.9% | -5.9% |
| bs=100 sw=10 sl=64 | p50 | 105,590 us | 102,003 us | 100,701 us | +3.5% | +4.9% |
| bs=100 sw=10 sl=64 | p95 | 123,923 us | 145,609 us | 107,244 us | -14.9% | +15.6% |
| bs=100 sw=10 sl=64 | p99 | 123,923 us | 145,609 us | 116,122 us | -14.9% | +6.7% |
| bs=1000 sw=10 sl=64 | throughput | 1,078 tuples/sec | 1,095 tuples/sec | 1,042 tuples/sec | -1.6% | +3.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.658 MB/s | 0.669 MB/s | 0.636 MB/s | -1.6% | +3.5% |
| bs=1000 sw=10 sl=64 | p50 | 928,554 us | 915,115 us | 981,959 us | +1.5% | -5.4% |
| bs=1000 sw=10 sl=64 | p95 | 1,002,764 us | 956,740 us | 1,023,080 us | +4.8% | -2.0% |
| bs=1000 sw=10 sl=64 | p99 | 1,002,764 us | 956,740 us | 1,051,697 us | +4.8% | -4.7% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,475.81,200,128000,420,0.257,23058.61,29120.74,29120.74
1,100,10,64,20,2108.49,2000,1280000,949,0.579,105589.61,123923.46,123923.46
2,1000,10,64,20,18550.21,20000,12800000,1078,0.658,928553.74,1002764.00,1002764.00|
I am converting this PR to a draft: please follow our PR template and mark it ready for review. |
|
The description now follows the current template, and this is ready for review. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8256 +/- ##
============================================
- Coverage 93.93% 93.90% -0.04%
+ Complexity 4762 4761 -1
============================================
Files 1191 1191
Lines 48308 49053 +745
Branches 5377 5377
============================================
+ Hits 45380 46061 +681
- Misses 1494 1557 +63
- Partials 1434 1435 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What changes were proposed in this PR?
Python range shuffle now truncates finite double keys toward zero before bucket calculation, matching the JVM partitioner. Infinite values retain their existing endpoint routing.
Any related issues, documentation, discussions?
Closes #8255
How was this PR tested?
Direct reproduction after the fix:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex