feat(pyamber): match Java binary hashes - #8250
Conversation
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8250 +/- ##
============================================
- Coverage 93.93% 93.89% -0.05%
+ Complexity 4762 4761 -1
============================================
Files 1191 1191
Lines 48308 49051 +743
Branches 5377 5377
============================================
+ Hits 45380 46058 +678
- Misses 1494 1558 +64
- Partials 1434 1435 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 362 | 0.221 | 26,047/36,679/36,679 us | 🔴 +9.8% / 🔴 +135.4% |
| 🟢 | bs=100 sw=10 sl=64 | 814 | 0.497 | 122,196/137,588/137,588 us | 🟢 -12.5% / 🔴 +28.3% |
| 🔴 | bs=1000 sw=10 sl=64 | 897 | 0.548 | 1,114,779/1,219,731/1,219,731 us | 🔴 +5.2% / 🔴 +19.2% |
Baseline details
Latest main 50321e4 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 362 tuples/sec | 394 tuples/sec | 787.98 tuples/sec | -8.1% | -54.1% |
| bs=10 sw=10 sl=64 | MB/s | 0.221 MB/s | 0.24 MB/s | 0.481 MB/s | -7.9% | -54.0% |
| bs=10 sw=10 sl=64 | p50 | 26,047 us | 23,713 us | 12,593 us | +9.8% | +106.8% |
| bs=10 sw=10 sl=64 | p95 | 36,679 us | 36,135 us | 15,579 us | +1.5% | +135.4% |
| bs=10 sw=10 sl=64 | p99 | 36,679 us | 36,135 us | 18,786 us | +1.5% | +95.2% |
| bs=100 sw=10 sl=64 | throughput | 814 tuples/sec | 819 tuples/sec | 1,008 tuples/sec | -0.6% | -19.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.497 MB/s | 0.5 MB/s | 0.615 MB/s | -0.6% | -19.2% |
| bs=100 sw=10 sl=64 | p50 | 122,196 us | 119,498 us | 100,701 us | +2.3% | +21.3% |
| bs=100 sw=10 sl=64 | p95 | 137,588 us | 157,222 us | 107,244 us | -12.5% | +28.3% |
| bs=100 sw=10 sl=64 | p99 | 137,588 us | 157,222 us | 116,122 us | -12.5% | +18.5% |
| bs=1000 sw=10 sl=64 | throughput | 897 tuples/sec | 910 tuples/sec | 1,042 tuples/sec | -1.4% | -13.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.548 MB/s | 0.555 MB/s | 0.636 MB/s | -1.3% | -13.8% |
| bs=1000 sw=10 sl=64 | p50 | 1,114,779 us | 1,098,774 us | 981,959 us | +1.5% | +13.5% |
| bs=1000 sw=10 sl=64 | p95 | 1,219,731 us | 1,159,861 us | 1,023,080 us | +5.2% | +19.2% |
| bs=1000 sw=10 sl=64 | p99 | 1,219,731 us | 1,159,861 us | 1,051,697 us | +5.2% | +16.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,552.91,200,128000,362,0.221,26046.80,36678.84,36678.84
1,100,10,64,20,2456.96,2000,1280000,814,0.497,122195.82,137588.34,137588.34
2,1000,10,64,20,22295.08,20000,12800000,897,0.548,1114779.33,1219731.31,1219731.31|
I am converting this PR to a draft: please follow our PR template and mark it ready for review. |
|
The description now follows the current template, and this is ready for review. |
What changes were proposed in this PR?
Python tuple hashing now converts binary field bytes to signed Java byte values before applying the array hash. Existing values below
0x80remain unchanged.Any related issues, documentation, discussions?
Closes #8249
How was this PR tested?
Direct reproduction after the fix:
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Codex