bench: add experimental node:bench module

[jasnell]

A new node:bench module, modeled closely after the node:test architecture.

import { bench, suite } from 'node:bench';

suite('URL', () => {
  const input = 'https://example.com/a?b=c';

  bench('construct', {
    samples: 30,
    params: { input: 'short' },
  }, (b) => {
    const operations = 10_000;

    b.start();
    for (let i = 0; i < operations; i++) {
      new URL(input);
    }
    b.end(operations);
  });
});

node --bench benchmark.mjs

The core functionality is here but there are still improvements that are needed. Specifically, there's a fair amount of overhead interjected by the harness itself that can impact the results. The focus was on minimal functionality, not absolutely optimal functionality. But this gives a solid starting point for improvements.

[nodejs-github-bot]

Review requested:

• @nodejs/loaders
• @nodejs/performance
• @nodejs/startup
• @nodejs/tsc

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/76609/

[codecov]

Codecov Report

❌ Patch coverage is 98.99920% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.16%. Comparing base (0544741) to head (f5af443).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/bench_runner/cli.js	97.95%	13 Missing ⚠️
lib/internal/bench_runner/harness.js	99.49%	1 Missing and 3 partials ⚠️
src/histogram.cc	93.61%	0 Missing and 3 partials ⚠️
lib/internal/bench_runner/benchmark.js	99.61%	2 Missing ⚠️
src/node_options-inl.h	85.71%	0 Missing and 2 partials ⚠️
src/node_options.cc	97.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #65606      +/-   ##
==========================================
+ Coverage   90.07%   90.16%   +0.08%     
==========================================
  Files         751      761      +10     
  Lines      254921   257394    +2473     
  Branches    48129    48686     +557     
==========================================
+ Hits       229627   232068    +2441     
- Misses      16479    16502      +23     
- Partials     8815     8824       +9     

Files with missing lines	Coverage Δ
lib/bench.js	100.00% <100.00%> (ø)
lib/bench/reporters.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/benchmarks_stream.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/reporter/json.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/reporter/spec.js	100.00% <100.00%> (ø)
lib/internal/bench_runner/runner.js	100.00% <100.00%> (ø)
lib/internal/bootstrap/realm.js	96.31% <100.00%> (+0.01%)	⬆️
lib/internal/histogram.js	96.22% <100.00%> (+0.11%)	⬆️
lib/internal/main/bench_runner.js	100.00% <100.00%> (ø)
src/histogram.h	72.72% <ø> (ø)
... and 9 more

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[RafaelGSS]

How is it different from what we (@H4ad) proposed in #50768?

[jsumners-nr]

@RafaelGSS probably nothing except that hopefully now it can progress. My question would be: why not vendor https://www.npmjs.com/package/bench-node?

[jasnell]

... why not vendor ...

I initially considered vendoring one of the existing packages and opted against it in favor of a light weight, minimal, no-new-dependency approach that a more feature-rich tool like bench-node can build/iterate on. I don't consider these either/or options.

Node.js' own benchmarks demonstrate that the full set of features provided by bench-node aren't necessary for a minimal bench suite. node:bench is intended to meet the low end, minimal requirement. These don't need to compete.

	node:bench	bench-node
Role	Minimal built-in benchmark kernel	Full-featured userland benchmark tool
Core features	Samples, warmup, suites, hooks, raw results, reporters, CLI isolation	Automatic warmup, V8 plugins, workers, DCE checks, baselines
Reporting	Structured event stream with spec and JSON reporters	Text, chart, pretty, HTML, JSON, and CSV
Dependencies	No new runtime dependencies	Piscina (installed but not used??)

We can provide stable measurement and structured result primitives without needing to ship the larger user-facing feature set and these existing tools can continue serving their current users while optionally adopting the built-in foundation. Just like node:test hasn't replaced the more sophisticated test suites like vitest.

That said, this is just a starting point. I've never been one to say "It must be done this specific way!" so all options are on the table. What I want is a built-in node --bench ... mechanism, I don't necessarily care how we get there as long as it works.

[jsumners-nr]

Just like node:test hasn't replaced the more sophisticated test suites like vitest.

If I can't do it with node:test I just don't worry about it.

That said, this is just a starting point. I've never been one to say "It must be done this specific way!" so all options are on the table. What I want is a built-in node --bench ... mechanism, I don't necessarily care how we get there as long as it works.

I'm with you. I just think it's easier to deliver what folks would actually want out of the module by bringing something like bench-node in. Look at the logger attempt. Instead of bringing in pino (yes, Matteo and I were/are both against that) a whole new logger was written while trying to keep the learnings from pino. That failed. But the parts undici is used for seem to be working. I think the same approach can be used.

[jasnell]

... what folks would actually want out of the module ...

different folks "actually want" different things. You prove this point yourself with the "If I can't do it with node:test I just don't worry about it" comment. Node.js' own benchmarks don't require any of the extended features of bench-node

[jsumners-nr]

Node.js' own benchmarks don't require any of the extended features of bench-node

Maybe, but aren't standard library modules meant to provide utility to the community, not just the project itself?

[jasnell]

Maybe, but aren't standard library modules meant to provide utility to the community, not just the project itself?

Why not both?

I'm intentionally designing this for both cases. Standalone utility plus primitives that can be used by tools like bench-node.