CLI Test Runtime Hardening Implementation Notes

Summary

• Delivery started from the accepted PLAN.md on branch team/stefan/research-cli-test-hardening, rebased onto origin/main at 53a2166d before implementation.
• T1 completed 18 green profiling commands with identical inventories. Two workers reduced mean ordinary-suite wall time from 348.641s to 170.452s, and full scaffold operations accounted for 94.45%-96.09% of hot-file wall time.
• T2 test-drove explicit bounded ordinary CI. Local evidence was green, but GitHub exposed subprocess starvation; the workflow and T2-specific contract were reverted to serialized ordinary execution.
• T3 introduced a one-shot scoped scaffold operation that prepares either an existing root session or nearest-existing-ancestor authority before inspection and snapshots caller-owned inputs. A distinct absent root is bound only after inspection succeeds. It reduced the stale-skill candidate from two full bundled projections to one. Recording-adapter tests own orchestration/result assertions; a bounded real filesystem witness still owns stale deletion and containment. The final rich plan freezes compiled desired input/state, explicit reset directories, the static deep-owned result, and an opaque one-shot applier, so the exact plan that passes preflight is applied without source traversal or replanning. Final exact-plan coverage is adapter 11/11, output 12/12, and combined 23/23; the earlier adapter 10/10 remains historical evidence.
• T4 routed staged initialization through the same required service. Planning now retains concrete adapter-owned StagePlanState, preserves the selected baseline for stage skills, and separates public ScaffoldCapabilities from the internal filesystem adapter. Stage owns one rich output plan whose additionalSkillSelection freezes selected-first or bundled-fallback skill bytes, aliases, and reset roots; planned settings freezes initialization and baseline metadata. A truly virtual stage test owns orchestration and commit/ rollback assertions; the consolidated real rerun retains mirror, planned-symlink, and containment evidence. Current evidence is adapter 13/13, output 12/12, combined 25/25, focused settings/fallback 3/3, symlink witnesses 2/2, and full stage 49/49; historical T3 remains 11/11, 12/12, and 23/23.
• T5 local convergence passed the full focused, update, release, tooling, and ordinary matrices. Bounded ordinary execution retained the identical 82-file/1,068-test inventory and reduced wall time from 312.63s to 165.75s. GitHub nevertheless exposed subprocess starvation, so T2 scheduling was reverted while retaining the scoped-operation seam.
• Final post-revert local serial validation passed 82 files and 1,068 tests in 316.19s with the same inventory hash. Three corrected GitHub attempts passed serialized ordinary CLI and all four update jobs; only the explicitly excluded wiki projection kept the overall workflow red.
• Private/internal docs ingest updated docs/README.md, docs/runbooks/hi-cli-scaffolding.md, and the existing .agents/notes/2026-08-04-scaffold-parent-identity.md. Routed Wiki projection was intentionally skipped by scope. Stack status showed an independent branch with no PR stack; stack sync --dry-run was unsupported, while the default stack sync preview said the dev stack was current. No apply ran. PR #100 is ready for review and its body was refreshed.
• Run 31067183663 on head 3e81a31c passed cli-tests, all four update jobs, and API, backoffice, and wiki Vercel checks. Behavior Contract was red only at @punks/wiki#check:content, requesting the excluded projection: create routed cli-test-runtime-hardening/PLAN.md and IMPLEMENTATION-NOTES.md, update apps/wiki/content/docs/project/specs/.source-projection.json, and update the routed hi-cli-scaffolding mirror.

Deviations From the Plan

• The source folder has no sibling SPEC.md. The user explicitly directed delivery to start at implementation and accepted PLAN.md as the delivery contract, so this run does not reopen the specification phase.
• T3/T4 candidate ownership was factually inverted in the accepted plan: run.test.ts owns stale regenerated-skill deletion, while stage.test.ts owns exact-mirror and planned-symlink reruns. The task descriptions were corrected without changing scope, dependencies, or accepted test contracts.

Surprises and Decisions

• origin/main advanced after planning. The plan-only branch was rebased before any implementation work began.
• A discarded profiler pre-sample placed TMPDIR inside the worktree and correctly disturbed tests that require a path outside Git. The profiler moved run-owned roots to system temp, removed the invalid sample, and reran cleanly.
• T1's attribution gate passed: repeated full scaffold operations, not direct subprocesses, dominate the selected run and stage assertions.
• Wave 2 review required fresh one-shot plans, inspection-bound stable inputs, and direct typed error propagation before T4. Both adapters and their shared contract now enforce those invariants.
• Wave 3 review rejected a scanner-cause regression, recording-only stage parity, and seeded-once tests mislabeled as idempotency. The scanner diff was removed, stage semantics now share a real/recording contract, and a real two-operation rerun is retained.
• Mandatory closeout review required stable pre-inspection root authority, caller-input snapshots, concrete stage plan state, selected-baseline preservation, an internal filesystem adapter separated from public ScaffoldCapabilities, and a stage orchestration test with no host fixture. Final review then found a deletion/rollback race. The resolved design prepares existing-root or ancestor authority without mutation, creates and binds a distinct absent root only after successful inspection, and leaves failed inspection non-mutating. Mandatory review blockers are resolved. One platform limitation remains parked: Node mkdir followed by lstat cannot atomically prove ownership against an exact intervening replacement without native no-replace publication; the plan does not claim fail-closed for that window.
• The first structured autoreview raised a P2 on exact-plan integrity: apply could still derive output beyond the plan preflight had approved. The correction moved compiled desired input/state, explicit reset directories, the static result including a deep-owned baseline summary, and the opaque one-shot applier into the plan. Tests remove source bytes after planning, exercise a stale reset directory, mutate the original summary, and attempt replay; apply uses the frozen plan, performs no source traversal or replanning, preserves the frozen summary, and rejects replay. Structured autoreview on d58f638d then returned P1 because stage replanned and read live stage-skill source, and P2 because scoped settings read a live baseline version. Stage now owns and consumes one rich output plan: additionalSkillSelection captures selected-first or bundled-fallback bytes, aliases, and exact reset roots for preflight; the old post-commit copy is gone. Planned settings carries initialization and baseline metadata, and scoped settings reads the scaffoldOutput baseline. Both final manual rereviews were clean. The third structured autoreview on head 911a73e1 was CLEAN with no actionable findings and rated the overall patch correct at 0.91. It confirmed retained plans and one-shot lifecycle, root authority, stage-skill snapshotting, settings metadata, rollback boundary, and serialized CI remain consistent.
• The stage-skill correction had a public RED: after planning, the selected skill source was removed. Old apply failed. Exact error: Could not locate baseline asset. The independent scoped-settings RED expected baseline version 3.1.3 but received the mutated-after-plan value. The stage metadata regression became GREEN within the first correction and is not misreported as RED.
• GitHub attempt 1 passed all four update jobs but failed the bounded ordinary job with one Vitest timeout and three spawnSync node ETIMEDOUT failures. The same run caught a forbidden runtime barrel; the public feature root now owns its service contract and Effect-v4 validation passes 94/94.

Sanity Checks

UI Evidence Links

No UI surface is changed by this plan.

Runtime Validation Evidence

Acceptance Criteria Status

Manual Review Checklist

Debug Assessment

Root cause was resource contention and subprocess starvation under the bounded two-worker ordinary GitHub experiment. The fix reverted only T2 ordinary scheduling to serialized execution while retaining four exact update jobs. Existing local and GitHub runtime evidence verifies that outcome. No extra instrumentation was added during debug assessment because captured CI logs had already proven the incident before this phase.

Delivery State

• Delivery is complete with no in-scope blocker. Mandatory review blockers are resolved. The Node mkdir-to-lstat exact replacement window remains a parked platform limitation and is not claimed fail-closed. T4's TDD contract remains partial/not-met because the mandated target was absent during the purported RED run, while its later GREEN evidence is retained. Both manual rereviews are clean. After the non-clean d58f638d review was fixed, structured autoreview on 911a73e1 was CLEAN with no actionable findings and overall patch correctness 0.91. Debug assessment is closed; docs ingest and PR #100 preparation are complete. Run 31067183663 leaves only the explicitly excluded routed Wiki projection as an external blocker. No final branch head is asserted before the parent commit.

Steering