Skip to content

sshd runs a program, moves a file, and is refused without its key - #440

Merged
Japabu merged 4 commits into
mainfrom
wt/toyos-sshd
Sep 8, 2026
Merged

sshd runs a program, moves a file, and is refused without its key#440
Japabu merged 4 commits into
mainfrom
wt/toyos-sshd

Conversation

@Japabu

@Japabu Japabu commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Stage 4 of the network track (issues/hardware/the-t14-answers-only-through-a-usb-stick.md).

Every result from the T14 rides a USB stick because nothing a program prints reaches the harness. sshd is the cable's answer path, and it could carry a shell and nothing else: an exec request ran /system/bin/shell -c, so the diagnostics and the exit status were the shell's; there was no subsystem, so no file moved either way; and the only key file was /home/root/.ssh/authorized_keys, which on a freshly flashed machine is a file nobody can put a key in — /home may be a tmpfs and the machine is unreachable until somebody has already reached it.

The daemon

exec runs the named program, not a shell. SSH carries one string because it has no argument vector; command::split reads it with single quotes, double quotes and backslash, and every other shell metacharacter is an ordinary byte of an argument. shell -c '…' survives the split whole, which is how a client that wants a pipe gets one. stdout is channel data, stderr is the channel's extended data (it was merged into stdout before, so a program's diagnostics were indistinguishable from its output), and the exit status is the channel's exit-status. A line that will not split, a first word that is empty, a name that is neither absolute nor bare, and a spawn the kernel refuses each answer 127 with the reason on stderr — never a channel left open.

The sftp subsystem moves files, draft-ietf-secsh-filexfer-02 version 3 in the profile the bench needs: open, read, write, close, stat and the directory listing. Everything outside it — SETSTAT, FSETSTAT, REMOVE, MKDIR, RMDIR, RENAME, READLINK, SYMLINK, EXTENDED — answers SSH_FX_OP_UNSUPPORTED naming itself, because a SETSTAT that answered OK without setting anything would make a client that checks its own work believe a mode it never got. Nothing acts on a partial packet, and a session that ends on a packet the protocol has no reply for closes with the draft's own SSH_FX_BAD_MESSAGE — 127 is left saying one thing, that this daemon would not run what was asked.

A key installed in the image is read beside the writable one. /system/etc/ssh_authorized_keys is put on ROOT at build time and either file authorizes; /system is the read-only root, so the new file is the less reachable of the two.

A key no file names is refused at the probe. A public-key exchange is an offer carrying the key and no signature, and then — only under USERAUTH_PK_OK — a signature; russh's default answers PK_OK to every offer, which tells a stranger its key would be taken and asks it to prove it holds it. auth_publickey_offered refuses first, so the daemon fails closed at the earliest point the protocol has, and auth_publickey checks again because a client is free to sign without asking.

Nothing this daemon starts outlives the connection that asked for it. That is one rule where the first draft of this branch had four numbers. The session holds a watch sender and every task it spawns holds a subscription: when the connection goes, the input thread ends, the forwarder ends, and the program is killed and waited for before the console says it ended. It replaces EXIT_WAIT — which killed a live program ten seconds after it closed its output — SFTP_IDLE, whose wait ends anyway when the session's sender drops, and a stated inactivity_timeout that restated russh's own default of 600 s and bounded nothing. The one wall clock left is INPUT_STALL, which cannot be an event: the handler offering a program its bytes is the connection's own task, so a program that reads nothing while a client keeps sending would wedge that connection with nothing left able to notice. Its decision is offer, and three host tests judge it.

The harness

tests/ssh-client-host is the client: russh's client half and russh-sftp's, neither of which anybody here wrote, so where the two agree it is because the draft says so. It is its own crate with its own lockfile because russh 0.60 resolves against the RustCrypto release candidates; tests/https-fetch-host is the same shape for the same class of reason, and build_https_hosts is now build_host_judges and builds all three from named rows.

BootOptions::ssh_port is what makes slirp two-way: a hostfwd clause on a free loopback port, and the boot asserts the argv actually carries it, because a harness field that can be silently inert is this suite's worst defect class.

Three names on one tests/sshdcase boot (SSHD_LOGIN), with the key staged into that image:

  • sshd_exececho byte-exact and 0; a missing program 127 with its name on stderr and nothing on stdout; an unterminated quote 127, refused before anything spawns; test_rs_empty_dir_stat, a real guest test binary, run over the cable and judged by its exit status, which is what the metal profile will do without a flash; cat <staged> <missing>, whose file lands on stdout and whose diagnostic lands on stderr with neither carrying the other's bytes; cat reading the channel's own data back, after an env request the guest answers rather than leaving open; and spin, started and then abandoned, which the guest ends and says it ended.
  • sshd_files — the staged authorized_keys file read off the guest and compared with the bytes the build wrote, which never travelled over SFTP; a 1,000-byte file of every byte value out and back; the guest's own listing agreeing about its size; 1 MiB each way.
  • sshd_key_auth — a key no file names refused at its offer, seen two ways: the client offers it through a signer that cannot sign and records being asked, and a guest that refused at the probe never asks; and the console line for the offer is not the console line for a signature. The staged key accepted with its fingerprint beside it, and publickey the only method still on offer after the refusal.

sshd_fail_closed now demands both key files be named as unreadable: a check on only the writable one would pass a machine whose image file was silently never consulted.

Priced from run 34200599143's test-durations-merged: sshd_exec 5110, sshd_files 535, sshd_key_auth 1428, all under FAST_COMMIT_MS, so all three are Tier::Fast.

The two checks

Negative controls — six, each a single thing reverted onto this branch's tip, built and run the same way:

control result
auth_publickey_offered deleted FAIL sshd_key_auth: the machine asked a key no file names to sign, so it answered PK_OK to a stranger's offer instead of refusing it; the other three green
exec_request back to shell -c FAIL sshd_exec: \echo` answered "", FAIL sshd_key_auth: the authorized key got Some(127); sshd_files` green
a child's stderr sent as channel data instead of extended data FAIL sshd_exec: stdout carried "ssh-ed25519 …\n/tmp/no_such_file_for_the_stderr_arm: file not found\n"; the other three green
the child no longer bound to its connection FAIL sshd_exec: a program outlived the connection that started it: STALLED; the other three green
is_authorized returns true FAIL sshd_key_auth, at the offer rather than after the signature; the other three green
the sftp arm of subsystem_request removed FAIL sshd_files: … error starting an sftp session: Timeout; the other three green

Independent oracle — OpenSSH 10.3p1. Not a committed dependency and on no test's path: a temporary arm ran the host's own /usr/bin/ssh and /usr/bin/sftp against the guest through the same forward, then was reverted.

[openssh] ssh exec -> status Some(0) stdout "hello openssh\n" stderr ""
[openssh] ssh missing -> status Some(127) stderr "sshd: cannot run /system/bin/no_such_program_at_all: entity not found\r\n"
[openssh] sftp -> status Some(0);  ls /tmp, get, put, ls /tmp all as expected
[openssh] sftp get of the 1,000-byte file: 1000 bytes, identical=true
[openssh] sftp put of /etc/hosts read back: 214 bytes, identical=true

A mature implementation nobody here wrote read the refusal off stderr and the 127 off the channel's exit status, which is the whole of what the metal profile will judge a test binary by. The draft itself is the other independent reading: the SFTP encode/decode is host-tested against packets built from its own field order, and russh-sftp implements it from the other side.

Issues

Closed: issues/isolation/sshd-accept-path-unexercised.md. It named auth_publickey, auth_publickey_offered and the MethodSet that stops password auth being offered as certified by reading; all three are now exercised — the accepted key, the key refused at its offer, and AuthResult::Failure's remaining_methods, which sshd_key_auth demands be publickey alone.

One commit on this branch, 814020f4, deleted auth_publickey_offered and said no registration could reach it because russh's client signs immediately. That is false: client/encrypted.rs:940-949 sends the key as a probe with no signature — the source's own comment is // This is a probe — and the signature at :1039 follows only under USERAUTH_PK_OK. d301d065 restores the check, says so, and is where the truth is recorded; history is not rewritten.

Filed: issues/isolation/sshd-holds-one-channel-and-does-not-say-so.mdSshSession keeps one channel and one input sender, neither keyed on a ChannelId, so two session channels on one connection alias each other's input. It predates this work and nothing in the tree or in OpenSSH's client reaches it.

No ABI change: nothing under toyos-abi/src, toyos/src or userland/libc/src is touched, and the sysroot claim was never taken.

Measured on the dev host: 29 sshd host tests, 261 toyos-build lib gates, the host workspace (138 suites, 1,337 tests), cargo run -- --clippy clean on all five invocations, and the four sshd names green together in 14.2 s.

🤖 Generated with Claude Code

Japabu and others added 2 commits September 8, 2026 09:41
Every result from the T14 rides a USB stick because nothing a program prints
reaches the harness. sshd is the cable's answer path, and it could carry a
shell and nothing else: an `exec` request ran `/system/bin/shell -c`, whose
diagnostics and exit status are the shell's; there was no subsystem, so no file
moved either way; and the only key file was `/home/root/.ssh/authorized_keys`,
which on a freshly flashed machine is a file nobody can put a key in — `/home`
may be a tmpfs and the machine is unreachable until somebody has already
reached it.

sshd now serves the three things a harness asks of a machine.

**exec runs the named program.** SSH has no argument vector, so the request is
one string; `command::split` is the whole grammar it is read with — single
quotes, double quotes and backslash, and every other shell metacharacter an
ordinary byte of an argument. A client that wants a pipe asks for `shell -c`
by name, and that construct survives the split intact. stdout is channel data,
stderr is the channel's extended data (it used to be merged into stdout, so a
program's diagnostics were indistinguishable from its output), and the exit
status is the channel's `exit-status`. A line that will not split, a name that
is neither absolute nor bare, and a spawn the kernel refuses each answer 127
with the reason on stderr — never a channel left open.

**The `sftp` subsystem moves files**, version 3 of draft-ietf-secsh-filexfer-02
in the profile the bench needs: open, read, write, close, stat, and the
directory listing. Everything outside it — SETSTAT, REMOVE, MKDIR, RENAME —
answers `SSH_FX_OP_UNSUPPORTED` naming itself, because a `SETSTAT` that
answered OK without setting anything would make a client that checks its own
work believe a mode it never got. Nothing acts on a partial packet, and every
length a client chooses is bounded where it is spent.

**A key installed in the image is read beside the writable one.**
`/system/etc/ssh_authorized_keys` is put on ROOT at build time and is what lets
a flashed machine be reached at all; `/home/root/.ssh/authorized_keys` stays
what a person adds a key to afterwards. Either authorizes. Both are named at
startup, and a daemon that can authenticate nobody still exits instead of
holding port 22.

Every wait in the new paths is bounded and refuses by name. Three of them were
unbounded *and* blocking on a single-threaded runtime, so one session could
wedge every other: the write to a child's stdin is now a thread behind a
bounded queue (a program that takes none of it for 30 s gets its input closed,
by name), `child.wait()` is a polled `try_wait` with a 10 s ceiling after both
output pipes have closed, and the SFTP session's filesystem work is a
`spawn_blocking` with a 300 s idle bound on the request stream.

`command::split` and the SFTP wire are host-tested — 25 tests in
`userland/sshd`, against real Ed25519 keys, `ssh-key`'s own parser, and framed
packets built and read back byte by byte.

The harness gets a client, and the gate gets four registrations.

`tests/ssh-client-host` is the other end: `russh`'s client half and
`russh-sftp`'s client, neither of which anybody here wrote, so where the two
agree it is because the draft says so. It is its own crate with its own
lockfile, and that is not tidiness — `russh 0.60` is written against the
release candidates of the RustCrypto `pkcs5`/`pkcs8` line, whose stable
releases renamed a constructor it calls, and sharing the host workspace's
resolution would cost twenty-odd `=` pins in the root manifest.
`tests/https-fetch-host` is the same shape for the same reason, and
`build_https_hosts` is now `build_host_judges` and builds all three.

`BootOptions::ssh_port` is what makes slirp two-way: it adds a `hostfwd` clause
on a free loopback port, and the boot asserts the argv actually carries it,
because a harness field that can be silently inert is this suite's worst defect
class. `QemuInstance::ssh_port` hands it to the members of a shared boot.

Three new names on one `tests/sshdcase` boot (`SSHD_LOGIN`), with the key
staged into that image:

- `sshd_exec` — `echo` byte-exact and 0, a missing program 127 with its name on
  stderr, an unterminated quote 127 refused before anything spawns, and
  `test_rs_empty_dir_stat` — a real guest test binary — run over the cable and
  judged by its exit status. That last one is what the metal profile will do
  without a flash.
- `sshd_files` — the staged `authorized_keys` file read off the guest and
  compared with the bytes the *build* wrote, which never travelled over SFTP; a
  1,000-byte file of every byte value out and back; the guest's own listing
  agreeing about its size; and 1 MiB each way.
- `sshd_key_auth` — a second connection from a key no file names, refused, with
  its fingerprint on the console, and the staged key accepted with its
  fingerprint on the console beside it. Without the second arm a daemon that
  refused everything would pass the first.

`sshd_fail_closed` now demands both key files be named as unreadable: a check
on only the writable one would pass a machine whose image file was silently
never consulted.

Measured on the dev host: four names, two guests, 9.9 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GGQ2H2aCwd1jfiNmjsiUvz
Three gates the first commit had not answered.

`src/sourcegate.rs` keys the dependency bar on the argument text at a
`Command::new`, so the harness's SSH client is declared there beside the reason
it is not a host `ssh` — and `tests/common/ssh.rs` spells the path in one
expression, because a path bound to a name first reaches that scan as nothing.

`src/durations.rs` requires an UNMEASURED marker's provenance to be
`shards=none`: a sentinel that claims a partition took it is a price nobody
measured wearing the shape of one.

And `sshd_key_auth` now reads what the machine still offers after it has
refused a key. `issues/isolation/sshd-accept-path-unexercised.md` named three
things certified by reading — `auth_publickey`, `auth_publickey_offered`, and
the `MethodSet` that stops password auth being offered at all. The first two
are exercised by the accepted and refused keys; the third was not, and russh's
`AuthResult::Failure` carries it: the client reports the remaining methods, and
the arm demands `publickey` and nothing else. A machine that offered `password`
or `keyboard-interactive` there would be offering a credential to guess at, and
no test in the tree could have seen it. That file is deleted, and the paragraph
in `issues/build/there-is-no-network-gate.md` that assigned its client here now
says what was built instead of what was owed.

Filed rather than fixed: `SshSession` keeps one channel and one input sender,
neither keyed on a `ChannelId`, so a client that opens two session channels on
one connection gets one input stream and the second `exec` takes the first
one's. It predates this work — the daemon had the same two fields when its only
request was a shell — and nothing in the tree or in OpenSSH's client does it.
`issues/isolation/sshd-holds-one-channel-and-does-not-say-so.md`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GGQ2H2aCwd1jfiNmjsiUvz
@Japabu
Japabu marked this pull request as ready for review September 8, 2026 08:38
@Japabu
Japabu enabled auto-merge September 8, 2026 08:39
@Japabu
Japabu disabled auto-merge September 8, 2026 08:49
Japabu and others added 2 commits September 8, 2026 11:46
… a test

The review of PR #440 found four timers where one event belonged, three
bounds no test could reach, a stderr split and an offer callback certified
by reading, and 127 answering three things that ran.

The daemon now has one rule where it had four numbers: nothing it starts
outlives the connection that asked for it. The session holds a `watch`
sender; the task carrying a program's output holds a subscription, and
when the session ends the program is killed and waited for before the
console says so. That is what bounds a program that neither writes nor
exits — the case the module doc claimed a bound for and did not have —
and it deletes `EXIT_WAIT`, which used to kill a live program ten seconds
after it closed its output, `SFTP_IDLE`, whose wait ends anyway when the
session's sender drops, and `SESSION_IDLE`, which restated russh's own
default of 600 s (`russh/src/server/mod.rs:122`) and bounded nothing.

`INPUT_STALL` stays, because it cannot be an event: the handler offering
the bytes is the connection's own task, so nothing would be left to notice.
Its decision moved into `offer`, which three host tests judge — a full
queue gives up inside its bound, an empty one takes the chunk, a reader
that has gone is answered at once rather than in thirty seconds.

`auth_publickey_offered` is deleted. russh's client signs immediately
(`russh/src/client/encrypted.rs:1039`), so no registration could ever
reach it, and its console line was byte-identical to `auth_publickey`'s.
`auth_publickey` re-checked every key anyway, so nothing about who gets
in has moved; `issues/isolation/sshd-accept-path-unexercised.md` named
three things and the two that still exist are both exercised.

An SFTP session that ends on a packet this protocol has no reply for now
answers the draft's own `SSH_FX_BAD_MESSAGE`. 127 is left saying one
thing: this daemon would not run what was asked.

`command::split` refuses a first word that is empty. `''` used to reach
`resolve("")` and spawn `/system/bin/` itself, with a refusal that named
nothing.

Three arms join `sshd_exec`, each seeing something a mutation would
otherwise pass: `cat <staged> <missing>` puts the file on stdout and the
diagnostic on stderr, so merging the two streams reds; `cat` with the
channel's data on its input reads it back, and the env request that goes
first is answered rather than left for a client to wait on; `spin` is
started and the connection dropped, and the guest says it ended the
program. `tests/ssh-client-host` grew `feed` and `abandon` for those, and
`tests/sshdcase` stages `cat` and `spin` beside `echo`.

Two host tests cover the SFTP listing: a directory longer than one reply
is handed over in batches that continue where the last left off and each
name arrives once, and one past `MAX_DIR_ENTRIES` is refused naming the
ceiling. `the_image_file_is_read_beside_the_home_one` is deleted: it
asserted a constant against itself.

The three prices are the ones run 34200599143's `test-durations-merged`
measured — sshd_exec 5110, sshd_files 535, sshd_key_auth 1428, all under
`FAST_COMMIT_MS`, so all three stay `Tier::Fast`.

`src/build.rs`'s three judge accessors take a named row instead of an
index, so a row inserted anywhere but the end cannot repoint them.

The prose findings are answered by deletion: the track's story in the
module header, the review answer written into `AUTHORIZED_KEYS`, the
narration at four sites, the how-to-run line copied into two more files,
the 23-line essay above `[package]`, the six-line `why`, the thirteen-line
doc on one `Option<u16>`, and the report paragraph a `track` file had
grown. The `=`-pin count nothing in the tree produces goes with them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GGQ2H2aCwd1jfiNmjsiUvz
… was false

**`814020f4` deleted `auth_publickey_offered` for a reason that is not true,
and this restores it.** That message and the pull request body both said
russh's client "signs immediately (`russh/src/client/encrypted.rs:1039`), so
no registration could ever reach it". Line 1039 is `client_send_signature`,
and reading it alone is the whole of the mistake: it is reached only from the
`USERAUTH_PK_OK` arm at `client/encrypted.rs:142`. What the client actually
sends first is `write_auth_request`'s `Method::PublicKey` at
`client/encrypted.rs:940-949`, which pushes `0` — the source's own comment is
`// This is a probe` — with the algorithm and the public key and no signature.
So the harness's client does offer before it signs, a registration can reach
the callback, and the claim that none could was wrong.

What the deletion left behind was worse than an unexercised callback. With no
`auth_publickey_offered`, russh's default answers `USERAUTH_PK_OK` to every
offer, including a stranger's: the daemon told a key no file names that it
would be taken, and asked it to prove it holds it, before refusing the
signature. The check is restored, so an unauthorized key is turned away at the
earliest point the protocol has one.

**A registration sees it, which is what the callback lacked before.** The two
refusals no longer print the same sentence — the offer says the key "was not
asked to sign", the signature says it "signed" — and `sshd_key_auth` awaits
the first. Beside that, the client now offers the stranger's key through
`authenticate_publickey_with` and a signer that cannot sign and records being
asked: a guest that refuses at the probe never calls it, and a guest that
answers `PK_OK` does. `ssh_refused` hands both facts back, and the gate
demands the machine did not ask.

`issues/isolation/sshd-accept-path-unexercised.md` named three things and
`814020f4` closed it having deleted one of them. All three exist again and all
three are now exercised: the accepted key, the key refused at its offer, and
the `MethodSet` read back off the wire.

Also deleted: the header of `tests/ssh-client-host/Cargo.toml`, which asserted
that the host workspace resolves the stable RustCrypto line. The root
`Cargo.lock` resolves none of those crates, so the sentence was false in the
same way the review's finding 12 said the `=`-pin count was.

`sshd_exec`'s committed price is still the four-arm test's; CI's next artifact
prices the seven-arm one and the tier rule is applied to that.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GGQ2H2aCwd1jfiNmjsiUvz
@Japabu
Japabu enabled auto-merge September 8, 2026 10:44
@Japabu
Japabu added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit 703b39e Sep 8, 2026
24 checks passed
Japabu added a commit that referenced this pull request Sep 8, 2026
Run 29 armed the I219's MSI and stopped at the 32-bit BAR; run 30, the negative
control with the claim pointed at a card this machine does not have, did the
same boot with no netd at all. Both are recorded here.

`boot.lancase.complete_ms` 1257 and `back_secs` 61 are the machine's own numbers
off boots whose claim was refused — facts about the boot rather than about the
claim, so the readings stand. The two `lan.` rows still have none, because netd
never came up on either.

**Run 30 changed what `ping_secs` is for.** The control pinged 192.168.1.46 once
a second for its whole 61-second window and got nothing: this machine's wire
answers no earlier than its ssh, and the loop stops pinging when ssh answers. So
the number that row will hold is not "earlier than Ubuntu" — it is "at all",
because nothing answers at that address unless the boot brings it up. That is a
stronger control than the one the row was written for, and the `ceiling_from`
says so. It still waits on a green reading before it can be tightened to the
span a boot is actually up for.

The track file is brought up to what these three runs established: sshd's half
landed as #440, the I219 driver and DHCP are built and green under QEMU, and
what is left is the laptop — the claim, and the two things that wait on it. Its
constraints list carries three measurements instead of two guesses: the I219 is
an MSI part (`/proc/interrupts`, `msi_irqs/162`), `toyos-t14` resolves to
nothing on this LAN and `t14` resolves to a Tailscale address only Ubuntu holds,
and the 32-bit BAR shares its 2 MiB page with the internal NVMe — which is why
leaving it where firmware put it is not the way out either.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GGQ2H2aCwd1jfiNmjsiUvz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant