feat(pyamber): route range NaN keys - #8258
Conversation
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 430 | 0.262 | 22,045/34,785/34,785 us | 🔴 -11.8% / 🔴 +123.3% |
| 🔴 | bs=100 sw=10 sl=64 | 936 | 0.572 | 106,261/119,804/119,804 us | 🔴 +5.8% / 🔴 +11.7% |
| 🟢 | bs=1000 sw=10 sl=64 | 1,106 | 0.675 | 899,419/962,898/962,898 us | 🟢 -8.0% / 🟢 -8.4% |
Baseline details
Latest main 50321e4 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 430 tuples/sec | 486 tuples/sec | 787.98 tuples/sec | -11.5% | -45.4% |
| bs=10 sw=10 sl=64 | MB/s | 0.262 MB/s | 0.297 MB/s | 0.481 MB/s | -11.8% | -45.5% |
| bs=10 sw=10 sl=64 | p50 | 22,045 us | 19,925 us | 12,593 us | +10.6% | +75.1% |
| bs=10 sw=10 sl=64 | p95 | 34,785 us | 31,129 us | 15,579 us | +11.7% | +123.3% |
| bs=10 sw=10 sl=64 | p99 | 34,785 us | 31,129 us | 18,786 us | +11.7% | +85.2% |
| bs=100 sw=10 sl=64 | throughput | 936 tuples/sec | 975 tuples/sec | 1,008 tuples/sec | -4.0% | -7.1% |
| bs=100 sw=10 sl=64 | MB/s | 0.572 MB/s | 0.595 MB/s | 0.615 MB/s | -3.9% | -7.0% |
| bs=100 sw=10 sl=64 | p50 | 106,261 us | 100,422 us | 100,701 us | +5.8% | +5.5% |
| bs=100 sw=10 sl=64 | p95 | 119,804 us | 125,815 us | 107,244 us | -4.8% | +11.7% |
| bs=100 sw=10 sl=64 | p99 | 119,804 us | 125,815 us | 116,122 us | -4.8% | +3.2% |
| bs=1000 sw=10 sl=64 | throughput | 1,106 tuples/sec | 1,084 tuples/sec | 1,042 tuples/sec | +2.0% | +6.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.675 MB/s | 0.662 MB/s | 0.636 MB/s | +2.0% | +6.1% |
| bs=1000 sw=10 sl=64 | p50 | 899,419 us | 911,341 us | 981,959 us | -1.3% | -8.4% |
| bs=1000 sw=10 sl=64 | p95 | 962,898 us | 1,046,764 us | 1,023,080 us | -8.0% | -5.9% |
| bs=1000 sw=10 sl=64 | p99 | 962,898 us | 1,046,764 us | 1,051,697 us | -8.0% | -8.4% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,465.47,200,128000,430,0.262,22044.74,34784.83,34784.83
1,100,10,64,20,2135.93,2000,1280000,936,0.572,106261.02,119804.01,119804.01
2,1000,10,64,20,18076.01,20000,12800000,1106,0.675,899419.28,962897.75,962897.75|
I am converting this PR to a draft: please follow our PR template and mark it ready for review. |
|
The description now follows the current template, and this is ready for review. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8258 +/- ##
============================================
- Coverage 93.93% 93.90% -0.04%
+ Complexity 4762 4761 -1
============================================
Files 1191 1191
Lines 48308 49054 +746
Branches 5377 5377
============================================
+ Hits 45380 46062 +682
- Misses 1494 1557 +63
- Partials 1434 1435 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What changes were proposed in this PR?
Python range shuffle now treats NaN keys as zero before bucket calculation, matching JVM double-to-long conversion. Finite and infinite key behavior remains unchanged.
Any related issues, documentation, discussions?
Closes #8257
How was this PR tested?
Direct reproduction after the fix:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex