Follow-up from #3772, which fixes stdout-only filters silently dropping a tool's stderr. That fix forwards the captured stderr verbatim. Filing the remaining question separately rather than widening that PR.
The gap
RunOptions::stdout_only() hands the filter only stdout, by design — stderr would corrupt the structured parse. After #3772 stderr is forwarded to the user unchanged: no filtering, no truncation, no never_worse guard. The only bound is stream.rs's 10 MiB RAW_CAP.
This is not a regression — it restores what the pre-078ad1d code did (eprintln!("{}", stderr.trim()), equally unbounded) — and rtk stays never-worse than the raw command, since stderr is identical on both sides. But it is a stream rtk passes through rather than compresses, on ~30 call sites: ruff, pytest, rspec, rubocop, prettier, phpunit, phpstan, gh, glab, go test, golangci-lint, tree, wc, psql.
Concrete case: rtk go test ./... on a cold module cache re-emits every go: downloading github.com/… v1.2.3 line. Bulk output with no compression, which is what rtk exists to avoid.
Why the obvious fix is wrong
Capping it with the existing helper does not work. utils.rs:29-39:
pub fn truncate(s: &str, max_len: usize) -> String {
...
format!("{}...", s.chars().take(max_len - 3).collect::<String>())
}
It keeps the head, at passthrough_max_chars: 2000. In the go: downloading case that keeps the noise and cuts whatever error followed it — reintroducing exactly the silent-diagnostic bug #3772 fixes. Any cap here has to be tail-preserving, or head+tail with an elision marker.
Options
- Leave it. Never-worse holds; stderr is the tool's own output; filters wanting stderr compressed should use
RunOptions::default() instead. Zero risk, no benefit.
- Tail-preserving cap. Keep the last N chars (errors usually land last) with a leading elision marker. Bounds the bulk without losing the diagnostic. Needs a new helper —
truncate cannot be reused.
- Head+tail. Keep both ends around an elision marker. Most informative, most code.
- Per-filter stderr handling. Let a filter opt into compressing stderr, e.g. dropping
go: downloading lines in go_cmd. Most precise, least general.
I lean 2 or 4. 2 is a one-place change bounding every caller; 4 is more accurate but has to be repeated per filter, and the noisy-stderr tools are a small set.
Worth noting the benchmark does not currently show a cost — with symmetric measurement (#3774) all 62 rows report 0 negative and 0 fail. The go: downloading case is not represented there because CI's module cache is warm.
Related
Follow-up from #3772, which fixes stdout-only filters silently dropping a tool's stderr. That fix forwards the captured stderr verbatim. Filing the remaining question separately rather than widening that PR.
The gap
RunOptions::stdout_only()hands the filter only stdout, by design — stderr would corrupt the structured parse. After #3772 stderr is forwarded to the user unchanged: no filtering, no truncation, nonever_worseguard. The only bound isstream.rs's 10 MiBRAW_CAP.This is not a regression — it restores what the pre-
078ad1dcode did (eprintln!("{}", stderr.trim()), equally unbounded) — and rtk stays never-worse than the raw command, since stderr is identical on both sides. But it is a stream rtk passes through rather than compresses, on ~30 call sites:ruff,pytest,rspec,rubocop,prettier,phpunit,phpstan,gh,glab,go test,golangci-lint,tree,wc,psql.Concrete case:
rtk go test ./...on a cold module cache re-emits everygo: downloading github.com/… v1.2.3line. Bulk output with no compression, which is what rtk exists to avoid.Why the obvious fix is wrong
Capping it with the existing helper does not work.
utils.rs:29-39:It keeps the head, at
passthrough_max_chars: 2000. In thego: downloadingcase that keeps the noise and cuts whatever error followed it — reintroducing exactly the silent-diagnostic bug #3772 fixes. Any cap here has to be tail-preserving, or head+tail with an elision marker.Options
RunOptions::default()instead. Zero risk, no benefit.truncatecannot be reused.go: downloadinglines ingo_cmd. Most precise, least general.I lean 2 or 4. 2 is a one-place change bounding every caller; 4 is more accurate but has to be repeated per filter, and the noisy-stderr tools are a small set.
Worth noting the benchmark does not currently show a cost — with symmetric measurement (#3774) all 62 rows report 0 negative and 0 fail. The
go: downloadingcase is not represented there because CI's module cache is warm.Related
run_streaming'sCaptureOnlyarm clones the entire child stdout into a field no caller reads #3397 —CaptureOnlyclones the entire child stdout into a field no caller reads; same code path