The log leaves the machine while the boot is still running

[Japabu]

The ThinkPad on the bench answers only through a USB stick and a reboot into
Ubuntu. logd gains a second sink beside its file: a TCP connection through
netd to an address the boot parameter line names, carrying every record as the
exact text line /log gets, in order, the instant it is written, with the
connection's close meaning the boot ended.

How the address gets there. The kernel command line reaches no process, so
params::init copies logstream='s value out of it — before mm::init may
hand that memory out, into a fixed buffer, because there is no allocator yet —
and spawn_init puts it in /system/bin/init's environment. init passes its
own environment to the daemons it starts at boot and clears it for anything its
launcher starts, so logd reads it and a program a user runs does not. It is
information and not authority: the netd connector on logd's manifest row is
what stands between the address and a peer, and no other program in the image
asks for one. No ABI change.

The file stays the sink of record. A line goes to the volume first and is
offered to the stream after, and the offer cannot fail: it either enters a
bounded queue or is counted. The connection is opened and every byte written on
a thread of logd's own, because opening one waits on a DHCP lease, a SYN and a
peer. What the stream could not do — no netd, a refused SYN, a malformed
address, records the queue would not take — goes into the log itself, once, and
is never offered back to the queue: a report a starved queue refuses owes
another report, for the life of the boot.

toyos-logstream is the crate the three parties share (the kernel reads the
parameter and parses no address; logd parses the address and never sees the
parameter; the build system checks the name and boots no guest), and it holds
the queue's whole decision, pure and host-tested. Its bound is derived from the
constants it must hold — BATCH lines at MAX_RECORD_MESSAGE plus what
Tagged renders around a message — with a const assertion and a host test
that fills it with a whole batch of the widest lines. blackbox= and
logstream= are VALUED_PARAMS in src/build.rs: matched as prefixes,
selecting no kernel build, cleared by name before an image is flashed and — for
the one this build can read — parsed there, so a typo is refused by the gate and
not by a machine on the bench.

The two checks

• Negative control, the sink — the whole sink reverted onto this tree
  (Stream::start answering None, nothing else touched): all five
  registrations red, on five distinct signatures. log_stream and log_stream_e1000e on "Boot: complete" never arrived on the stream in 90s: 0 connection(s), 0 line(s),
  log_stream_no_listener on the file carries 0 line(s) about a log stream that never opened, log_stream_unreachable on cost it no record it admits to; the file has 4272 line(s), log_stream_stalled_peer_delivers_whole_records on the stream was still open 20s after the guest went down; 0 line(s) received. The
  parameter path the control does not revert has its own two host tests:
  the_pre_flash_gate_parses_the_address_it_clears over every kind of typo, and
  a_valued_parameter_is_one_the_kernel_claims_by_prefix, which reads
  params::claims's own body.
• Negative control, the netd send path — the bridge_piped hunk reverted
  alone, three runs and six guests counting the harness's re-run-alone: every
  one red on a cut record with a whole one behind it (the stream carries "[…] logstorm t=3 i=739 k=… zabcdefghi…", which /log does not carry after its line N). log_stream_stalled_peer_delivers_whole_records releases the stalled peer while the
  machine is still up, so a byte the machine consumed and did not deliver is
  visible wherever it happened rather than only at the connection's end.
• Mutation — Backlog::admit no longer counting what it refuses: reds
  log_stream_unreachable and log_stream_stalled_peer_delivers_whole_records (a log storm offered to a stream that could not take it cost it no record it admits to) and three of toyos-logstream's ten host tests
  (a_starved_queue_drops_the_newest_and_counts_every_one,
  a_drop_report_is_never_a_line_the_queue_is_offered,
  one_impossible_line_does_not_evict_the_boot).
• Independent oracle — the guest's own log volume, every file this boot
  wrote, read off the FAT partition behind the guest's back and compared with
  what arrived over the wire: 230 lines on virtio and 232 on the e1000e, equal
  in content and in order; on the stalled arm, 636 received lines each /log's
  own in /log's own order. The storm arms' second oracle is the report's own
  arithmetic — the per-run counts must sum to the boot total the last line
  states.

Measured on the dev host

• Boot: complete reached the host 27 ms (virtio) and 35 ms (e1000e) into the
  listener's life, with the guest still running; a job's exit: record 29–32 ms
  after the job ended.
• A peer that accepts and stops reading is what the writer's own backpressure
  needs, and a narrow storm does not reach it: a 2 MiB kernel pipe and netd's
  64 KiB send buffer absorb the 674 KiB (4,213 lines) a log-storm at --smp 8
  produces. log-storm-wide widens each record to nearly a whole message.
• How many lines a stall costs is not a number any arm asserts. It is the
  pipe's size, netd's, and whatever QEMU's user-mode networking holds between
  them; two hosted rounds refused two different estimates of it.
  log_stream_stalled_peer_delivers_whole_records asserts what it forces on any
  host instead: the connection opened, the peer took a storm's worth of records
  while reading nothing and then read again, and every line it received is a
  whole record /log carries in /log's own order, at every position including
  the last. Where the boot does say it refused something, the report is still
  checked against itself. The drop accounting is toyos-logstream's host tests,
  which red under the drop-count mutation with no guest at all, and
  log_stream_unreachable, whose address answers nothing and so must lose
  lines.
• log_stream_unreachable, on the same footing: ~4,075 storm records in /log
  and 3,732–3,762 lines refused across runs, said in 5 lines whose per-run
  counts sum to the last line's boot total.
• cargo test: 337 passed, 337 total. cargo test --lib: 261 passed, 1
  ignored. cargo test --workspace --exclude toyos-build: 1,295 passed, green.
  cargo run -- --clippy: 5 invocations clean.

What this does not do

• The metal half — a listener on the Mac while the T14 boots — is left to build;
  the track file carries what a reader would otherwise re-derive.
• Four of the five are priced and relegated. Run 34185896772's
  test-durations-merged puts log_stream at 24,865 ms, log_stream_e1000e at
  25,219, log_stream_no_listener at 25,182 and log_stream_unreachable at
  25,707 — every one over FAST_CEILING_MS, so each is a Why::Cost row in
  src/tiers.rs and Tier::Nightly in its registration, with the row saying
  what stops being gated per pull request. What costs the price is the image:
  the host picks the listener's port and the port is on the parameter line, so
  each arm builds a boot image nothing can memoize — about 20 s of a 25 s run
  with no assertion behind it, and that is what would return them. The fifth,
  log_stream_stalled_peer_delivers_whole_records, is priced at 34,591 ms from
  run 34189903124 and joins them.
• Compromises found and filed rather than fixed here:
  issues/design-debt/a-boot-with-no-log-volume-streams-nothing.md,
  issues/design-debt/logd-holds-a-netd-connector-on-every-boot.md,
  issues/design-debt/the-parameter-lines-value-is-read-in-two-crates.md, and
  the receive half of
  issues/kernel/netd-drops-what-the-clients-pipe-would-not-take.md. The send
  half of that last one is fixed here: log_stream_stalled_peer_delivers_whole_records
  reproduced it as a record cut in half on the wire — the record stream's own
  oracle failing — and reverting the fix reds that arm.

🤖 Generated with Claude Code