Research
CI Actions Reliability Research
CI Actions Reliability Research
Executive answer
Run 31577448750 has two independent deterministic failures. Neither is a timeout, remote-cache corruption, nor a provider failure.
- The trusted behavior-contract job profiles Bun as Node because the host profiler resolves
nodethroughprocess.execPath. Since the profiler itself runs under Bun, it exports the Bun binary asHI_TEST_NODE_PATH. The CLI baseline builder then launches Oxfmt with Bun 1.3.5, producing 340DataCloneErrorfailures before exiting 2. Sources: failed job, host profiler, baseline formatter launch, and upstream Oxfmt/Bun defect. - The native release fixture makes its fake Git command report the fixture as repository root. Production code correctly derives
<root>/apps/cli, but the fixture never creates that directory or copies its built baseline artifacts there. Node reportsspawnSync git ENOENT; the missing object is the child working directory, not Git. Sources: failed native job, fixture root, and publisher root binding.
The development aggregate contains no third defect. It fails because the ordinary native/ambient dependency failed: aggregate job.
Method
Three readonly lanes inspected the exact failed jobs, the last 31 failed Behavior contract runs from 2026-08-10 through 2026-08-12, and the workflow/task/runtime architecture. The comparison also included successful run 31555413097.
Historical failure clusters
Categories overlap because one run can contain several failed assertions.
| Cluster | Failed runs | Evidence and interpretation |
|---|---|---|
| Vitest time budgets | 15/31 | Repeated 15s, 30s, 45s, and 60s failures. Representative runs: 31575529455, 31553976638, and 31548359116. Commit eb1584e4 removed CLI Vitest test/hook budgets; the current run shows those failures no longer mask later defects. |
| Stale fixtures or contract expectations | 20/31 | Repeated public-wiki legacy wording, pack counts, skill text, and inventory expectations. Representative runs: 31389696706, 31505683365, and 31514288591. These are atomic source/fixture maintenance defects, not runtime flakiness. |
| Baseline digest or fixture identity mismatch | 5/31 | Expected-versus-received SHA evidence in runs including 31409009794 and 31505683365. No inspected log showed remote-cache signature corruption. |
| Formatter/runtime mismatch | 1/31 | Current trusted job. The upstream issue states Oxfmt fails when forced to run under Bun; normal Node execution is the intended boundary. |
| Process-fixture isolation | 1/31 | Current native job. The fixture did not mirror the production repository layout after historical-root binding changed. |
No inspected failure was primarily caused by Docker, npm/GitHub publication APIs, Vercel cache signatures, or OIDC exchange. Gate jobs propagated dependency results as designed.
Architecture facts
run-test-scope.tsexecutes under Bun, profiles the host, and exports exact tool paths into Turbo. Source: scope runner.HI_TEST_NODE_PATHis consumed by build and native test commands, so misidentifying it poisons both execution and cache identity. Source: verification environment.- CLI build is deliberately uncached and rebuilds the full baseline in generic verification. This makes deterministic build defects visible repeatedly. It is amplification, not the cause. Source: Turbo build policy.
- Test and title inventories fail closed with explicit assignment diagnostics. Historical expectation drift does not justify weakening those contracts.
Implementation plan
- Resolve every profiled executable, including Node, from the supplied
PATH; fingerprint and revalidate that exact binary. Add a contract proving a Bun-hosted profiler still selects the real Node executable. - Keep production historical-root behavior. Repair the native publisher fixture by constructing
<fixture>/apps/cli/dist/baselineand copying the exact built artifacts before invoking the publisher. - Run the host-profile contract, the exact failing native tracer, the CLI typecheck/format/diff gates, and the affected behavior-contract route.
- Push the existing PR branch and observe the replacement Actions run. Do not add retries, cache bypasses, or broader time budgets.
Inferences and residual risks
- The Node identity defect is systemic because every
HI_TEST_NODE_PATHconsumer could receive Bun. Fixing the profiler is safer than hardcoding only the baseline formatter. - The native failure is a test-fixture regression. Changing production root binding to accommodate it would weaken exact historical-release verification.
- Root-level Bun behavior-contract tests still contain their own explicit suite limits, while subprocess safety guards remain intentionally bounded. Those were outside the earlier CLI Vitest budget removal and are not causal in the current run.
- Future tool upgrades should retain an execution-boundary assertion: a package with a Node shebang must be launched through the profiled Node path, never through the orchestrator runtime by accident.
Decisions
No product decision remains open for this repair. The accepted direction is to correct executable identity and fixture parity while preserving fail-closed inventories, cache signatures, and production release authority.