Issue 217: CI Cost and Test Signal

Status: evidence-grounded research, extended September 22 with current-code hotspot analysis. Accepted requirements are authoritative in the linked grill status; remaining recommendations are candidates, not measured improvements. No tests, workflows, runner settings, or release policy were changed. The September 22 deep dive below supersedes the original release-recovery build interpretation and reopens technical pruning decisions.

Final accepted scope — R5

The user accepted the consolidated proposals and explicitly removed the 80% target. Optimize actual execution and cache reuse as far as these identified inefficiencies permit, using only 2-vCPU workers and no test sharding. There is no fixed runtime ceiling, monthly minute cap/reserve or test-count deletion quota. Earlier shard experiments and target arithmetic in this historical report are superseded, not acceptance requirements.

The current authority is Issue 217: CI efficiency and test signal, compiled from closed Q1–Q11. The accepted scope covers the full inventory, updater preparation/install waste, minimal real recovery history with full replay on demand, narrower built-CLI proof, precise cache inputs, selected browser provisioning, duplicate root proof, isolated suite infrastructure reuse and wiki runner consolidation. Preserve all identified meaningful safety boundaries, PR/main verification and current release authority. Capability cache tasks remain within existing job topology; no shard fan-out is authorized.

Recommendation

Optimize useful confidence per normalized runner minute. The biggest opportunity is reducing repeated preparation and unnecessary invalidation in the serialized CLI suite. R5 excludes test sharding; bounded execution must respect the two-vCPU resource budget. Deleting registration/prose tests improves test quality but cannot, on present evidence, explain a severe runtime reduction. Sharding alone reduces feedback latency while duplicating setup and usually increasing consumption.

Model the system as a graph from owned invariant → cheapest sufficient proof → actual inputs and capabilities → isolated execution → complete decision. Each proof has one owner and one meaningful public result. Expensive execution should correspond to a real boundary that needs proving: installed executable, filesystem, database, browser, or publication. A test's historical issue number is neither a reason to keep it nor a reason to delete it.

Blacksmith normalizes allowance usage to x64 2-vCPU runners; historical 4-vCPU jobs consume two allowance minutes per wall-clock minute. The user subsequently required only 2-vCPU workers and explicitly rejected a monthly ceiling or reserve. Historical allowance arithmetic below is context, not an accepted budget requirement. Q10 supersedes Q8: no numerical speedup target and no sharding. Compare uncached feedback and total runner work; report warm cache-hit performance separately.

Evidence boundary and method

• Source examined: 7e147b3300c391db979b87e20c4159a5a03d12cb, clean initial working tree. Source paths and line numbers below refer to that revision.
• User request: issue 217, analyzed with Brainstorm, TDD/high-signal-tests, Make TSuite audit, Parallel Research, and Architect Pipeline guidance.
• Four independent readonly lanes: CLI test value; all other test value and selection; execution/caching topology; official Blacksmith/GitHub/Bun/Vitest/Turbo guidance. Coordinator inspected live Actions jobs and consolidated this report.
• Complete source inventory: 151 executable test files, excluding fixtures, snapshots, test support modules, and bundled third-party skill examples: CLI 68; API 29; backoffice 16; wiki 4; web 1; shared packages 21; root/operator tooling 12. This is file inventory, not current executed case count.
• Assertions, fixtures, selectors, and owned seams were inspected. Sensitivity is statically reasoned, not mutation-tested. The full test suite was not rerun; no fresh cold-cache benchmark was commissioned.
• Existing August research was consulted but not copied forward as current fact. That report describes 225 files and a different workflow. Current code already removed much of that topology.

Actual latency and consumption

Completed runs versus misleading samples

GitHub job timestamps were read through the Actions jobs API. Queuing and inter-job scheduling are excluded from individual job durations.

The failed detailed job 105078205569 reports 68 files, 723 cases, 2,612.83s total, including 2,600.01s in test bodies. Transform 1.18s, collection 6.48s, preparation 2.07s are negligible compared with test bodies.

147/723 cases account for 93.1% of test-body time. The issue also reports the longest updater case at 83.652s; its 22-minute total is many cases, so capability-based file splitting can help. The two long recovery workers take 394.860s and 306.165s, creating a separate indivisible floor until their preparation is reduced.

Successful run output is suppressed by --output-logs=errors-only; the passing logs establish overall cost but do not expose fresh per-file durations. Do not present the failed run's breakdown as the exact current passing breakdown.

September usage estimate

Query: repository Actions runs created 2026-09-01 through 2026-09-21, retrieved September 21. There are 184 workflow runs: 146 Behavior Contract, 22 protected release, 13 cache witnesses, 3 external drift. The 146 Behavior Contract runs yielded 402 non-skipped Blacksmith jobs with timestamps: 401 on 4 vCPU and one on 2 vCPU.

The result rows partition the same total; they are not additional consumption. Affected Verification accounts for 97.6%; failed/cancelled workflows account for 67.6%. Failure work is not automatically disposable: it can be valid bug detection. The useful question is how much is avoidable late failure, fixture contention, repeated setup, and superseded work.

This is a GitHub-timestamp estimate, not invoice or current Blacksmith usage. It applies documented CPU weighting, does not assume an undocumented billing-rounding rule, omits previous attempts for two rerun workflows, and excludes other repositories. No authenticated Blacksmith CLI was available. The official usage command below is the remaining financial authority.

Across 44 successful Affected Verification jobs, the median is 89s, but observed green jobs extend to 76m10s. These span revisions and cache states; they are not controlled benchmark repetitions. A blended median is a poor optimization objective. Report warm and forced-execution latency separately, including the tail.

Historical planning example, superseded as a requirement by Q2: at approximately 210 Behavior Contract runs/month, the full 3,000 allowance permits an average 14.3 normalized minutes/run; reserving 20% permits 11.4. The sampled mean is 33.6. The user rejected both a monthly design ceiling and reserve; neither is an acceptance gate.

The existing PR 221 right-sizes the aggregate runner and estimates $0.79/month savings. Even eliminating the entire sampled aggregate cost would recover less than 1% of the total. Its current failed run has not proven the success path on the smaller runner. PR 222 overlaps candidate validation/commit coverage; coordinate any later implementation with it.

Current system and candidate structure

Current graph, from .github/workflows/behavior-contract.yml and scripts/behavior-contract/run-ci-verification.mjs:

The graph already has affected selection, signed task caching, and cancellation of superseded PR work. These are existing capabilities, not new recommendations.

Candidate conceptual structure:

This is a model for task boundaries, not a proposal for another general-purpose proof registry. Reuse package scripts, Turbo, and existing evidence. Avoid test-title inventories or a second scheduler.

Evidence-backed opportunities

1. Eliminate repeated irrelevant preparation. CLI update helper has 69 call sites and no default dependencyProcess replacement (apps/cli/src/update/run.test.ts:256–299). Some cases intentionally execute real installs; many protect other outcomes. Reuse immutable prepared scaffold/toolchain seeds with private copies for filesystem mutations. Keep actual fresh-install and lifecycle-script suppression cases fresh (:2403). Five authoring setup calls and four ordinary-refresh calls perform repeated full-update preparation (:1360). These are source counts, not measured runtime counts.
2. Reduce historical replay, not merely its fixture self-test. September 22 correction: the fixture-only case took 1.226s; exact/resume took 701.025s. Readback-only skips retained publication artifacts but still reconstructs historical authority through isolated installs and CLI builds. The deeper trace below derives approximately 89 such builds across the two scenarios. Keep exact binding, resume, tamper, first-parent and provider readback invariants; Q9 accepts minimal real history in CI and full historical replay only on demand.
3. Separate semantic breadth from production composition. behavioral-portfolio.test.ts contains 25 scaffold call sites and repeated built-process execution (:170). The full issue-215 benchmark runs inside it (:350), with real warm-update, readiness, warnings, and no-write assertions. It is valuable behavior, not a disposable stopwatch. The macOS witness invokes restricted scenarios on a different filesystem; it is not proven duplicate Linux evidence. Move broad combinations to nearer owned seams only with an invariant map, keeping representative built CLI paths and platform-specific safety.
4. Make cache identity match actual capabilities. cache-identity.mjs:162–181 builds an all-host digest including browser, Docker, and executable identities. turbo.json:149 and :291–300 feed it to CLI types/tests. A browser change can invalidate nonbrowser proof. Partition by required capability only after proving the input closure; retain relevant runtime/platform/tool identities, trust authority, source dependencies, and deterministic environment.
5. Audit broad build/task inputs. turbo.json:34–48 uses whole-workspace defaults for CLI build. A test edit can invalidate build; unrelated CLI content can invalidate the one coarse test task. Global inputs also include lint/format config. Exclude irrelevant inputs only after checking build copying and generation; do not trade correctness for attractive cache-hit counts. Existing CLI tests correctly depend on the CLI build (:292).
6. Provision browser only for selected browser work. Chromium install is unconditional (workflow:62) and costs 17–18s in sampled jobs. Resolve selection before provision and coordinate host identity. Native Playwright guidance warns that browser cache restoration may cost as much as download; measure before adding another cache.
7. Give root cache-policy proof one owner. run-ci-verification.mjs:101 and :124 request it in both static and affected jobs. Static lacks shared-cache setup and differs in environment/capabilities. Remove duplicated execution only while preserving the required proof. Keeping cheap static feedback parallel may still be worthwhile: observed static failures did not create material long-running waste in this sample.
8. Reuse one PostgreSQL server within a suite. packages/db/src/baseline-authority-postgres.test.ts starts/removes four independent containers (:78, :149, :224, :295). Use separate databases and migration state on one suite-owned server if equivalent isolation is demonstrable. Do not delete migration/immutability/ledger scenarios. Cross-package server sharing is a larger ownership/concurrency decision.
9. Consolidate wiki test runners. Four wiki files currently start Node, Bun, and Vitest sequentially. One owning runner can retain synchronization, projection, navigation, and route behavior with less ceremony. Probably a small saving beside CLI hotspots.
10. Keep execution bounded without sharding. Organize proof by capability for ownership/cache reuse inside existing verification jobs. R5 rejects the original shard proposal. Historical subprocess starvation explains fileParallelism:false; adding parallelism is not a substitute for removing repeated installs/builds/setup. Monitor total process pressure on two CPUs.

Merely putting the three hot files on separate machines still leaves the 21m42s updater file as the lower bound before setup. Five equally balanced lanes over 43.5 serial minutes have an ideal 8.7-minute average, but setup, imbalance, and the 6.6-minute recovery case remain. This was historical scheduling arithmetic. R5 supersedes all numerical targets and shard experiments; it requires less actual work within existing job topology.

What to remove and what to protect

TDD's criterion is durable owned behavior through the nearest public seam. Do not replace real integration proof with mocks of the same behavior. Do not confuse runtime validation of untrusted data with static typing; owned public compatibility snapshots can be valuable.

Strong full-file removal candidates: three CLI catalog-registration files; CLI improve-mobile-frontend and verification-recovery-skills content tests; current SHA/tag snapshot in sync-skills-repo; release-guidance prose/history locks; API fake-service self-test; backoffice literal cache tags; parked web placeholder/source-copy tests. Partial culls include fixture self-validation, help/registration accounting, source transaction counting, byte-exact Philosophy prose, historical wiki existence/ingest flags, and UI class names. Backoffice auth-forwarding and cosmetic date formatting are medium-confidence candidates requiring consumer/UX confirmation.

Keep especially: user-authored-file preservation; no writes outside owned roots; partial failure/retry truthfulness; no lifecycle-script execution; no credential disclosure; live readiness despite cache hits; artifact tamper/authority rejection; exact historical release and resume; real database constraints and migrations; authentication/session lifecycle; critical browser failure recovery. These buy substantially more confidence than test count alone indicates.

For a merge/move/removal that protects behavior, record actor, authority, input class, seam, observable outcome, side effects, isolation, and retained proof's required gate. Static similarity is insufficient. For no-owned-invariant cases, state that explicitly. New behavior changes require actual RED before production edits; fixture refactoring must preserve semantic fingerprints. Full-file inventory follows.

Complete test inventory

All line references below point to the examined revision. Retain means meaningful evidence exists, not proof of zero duplication. Strengthen/move/merge preserve material behavior until equivalence is demonstrated. Dispositions and sensitivity are static candidates; no deletion or runtime reduction has been verified.

CLI: 68 files

Paths relative to apps/cli/src. All selected by Vitest src/**/*.test.ts through the CLI test task when affected. Generated dist copies are not selected. The old run.test-cases.test.ts exclusion is present in configuration but no such tracked source file exists at this revision.

API: 29 files

Paths relative to apps/api/src. Workspace Vitest selects these when API is affected.

Backoffice: 16 files

Paths relative to apps/backoffice. Vitest selects all except e2e, which belongs to test:browser. Playwright uses one worker and eight cases; server startup is a likely larger concern than browser test sharding.

Wiki and web: five files

Shared packages: 21 files

Paths relative to packages. Workspace test task selects each when affected; config uses Bun, others Vitest.

Root and operator tooling: 12 files

R = ordinary root test:cache-policy; CW = separate cache witness workflow; U = no normal package/Turbo/workflow selector found. U does not mean useless or safe to delete. Deleting U files saves zero normal CI runtime.

Official runner guidance and cost experiments

Sources were read on 2026-09-21; marketing speed claims are not repository benchmarks.

Cost model without assuming rounding:

normalized minutes = sum(job wall minutes × vCPU / 2 × architecture factor)
architecture factor: x64 = 1; ARM = 0.625
sharded work = useful work + repeated setup + transfer + aggregation
latency = longest dependency path + queue/scheduling time

For illustration, four 11-minute 4-vCPU shards consume 88 normalized minutes before extra overhead, comparable to one 44-minute 4-vCPU job. They deliver feedback sooner but do not solve allowance pressure. Reducing work to four 6-minute shards would consume 48; the saving comes from less work, not the number of shards.

ARM may be cheaper if the same-size job takes less than 1.6x the x64 time. Native binaries, package-manager behavior, cache partitioning and required platform coverage must be verified before migration. Do not infer acceptance from public pricing alone.

Release and cache witnesses use GitHub-hosted runners, including macOS; they do not explain Blacksmith minutes. Moving them to Blacksmith would move consumption into this budget. Protected historical build/apply separation remains intentional; current publication eligibility does not require a prior PR Candidate Evidence lookup. Preserve actual current release rules rather than reintroducing historical assumptions.

Candidate experiment sequence and acceptance

1. Restore a valid benchmark starting state: the latest hosted failure is frozen-lockfile mismatch, not test speed. Coordinate with open work; do not weaken --frozen-lockfile.
2. Capture passing per-file/case durations and phase costs on one fixed revision/toolchain/runner: setup/copy/Git/install/build/command/cleanup, CPU/memory and subprocess pressure. Reuse existing runner metrics. A cached timing artifact must retain its original execution provenance and be labelled reused.
3. Cull no-owned-invariant checks and consolidate redundant runner/setup work. For other changes, record semantic-lock mappings before edits. Confirm real cold-install, native filesystem, auth, database and release witnesses remain selected.
4. Introduce capability-level execution/cache boundaries and immutable fixture seeds, each locally runnable. Validate task identity with forced execution, output deletion/restoration, relevant-input misses, irrelevant-input reuse, and live readiness where required.
5. Use historical 4-vCPU results as context and measure comparable 2-vCPU configurations, as required by Q1. Do not add CLI shards; capability groups remain inside existing verification jobs with private mutable resources. Report both critical-path latency and summed runner work, including setup and retries; there is no accepted monthly budget gate.
6. Only then consider more shards or ARM. Retain all expected shard identities, exactly-once selected-file assignment, signed output provenance, and aggregate fail-closed behavior before emitting tested-tree evidence. Do not shard the entire test:ci graph and rebuild everything on every runner.

Measure three distinct modes: task-result reads bypassed with warm dependency stores; clean dependency stores and task caches; normal warm reuse. Benchmark runs must not seed trusted cache evidence under misleading identities. Use several completed green repetitions; include failed/cancelled consumption separately and compare at the same revision. Initial repetitions give distributions, not a statistically robust p95; collect p95 over sustained operation.

Suggested acceptance candidates: warm feedback remains around the current two-minute workflow range; expensive misses improve through removed work, without a fixed percentage target; total normalized usage per comparable change falls; fixture failures and retries do not increase; complete proof remains required. Q10 removes the numerical target and sharding; latency and total work remain separate observations. No improvement is proven by this report.

September 22 technical deep dive

The user reopened requirements to understand concrete test types and pruning, with an aspirational 80% CI-time reduction. Source grounding is now child HEAD 9430c62496aed8b5f9a94c399d877b9f1aa7a39d, above PR #222 head cef8e31e. Three readonly research lanes traced updater tests, release reconstruction, and built CLI journeys. Their findings are consolidated here; no expensive tests or hosted workflows were launched.

The current inventory is 153 executable files, including 70 CLI files. The parent added features/project-settings/service.test.ts and features/scaffold-update/validation-plan.test.ts. The original inventory and timing sample remain historical: 151 total files, 68 CLI. New code anchors below refer to the current child; historical durations do not become current measurements.

Test value and execution cost are different axes

Deleting hundreds of cheap checks while preserving the same repeated installations would barely change latency. Conversely, a test may assert an important invariant at a needlessly expensive execution boundary. Such a test needs a cheaper proof, not an indiscriminate deletion.

Updater: repeated application preparation and real installation

Historical slow-case groups within update/run.test.ts:

These are sums of explicitly printed slow cases, not the complete group case counts. Overall 47 printed slow updater cases explain 1298.719s of its 1301.638s. The largest authored-wiki/starter case took 83.652s.

test fixture
  preparatory updates and authoring completion
  runUpdateWithPreview
    detect repository and compile desired context
    materialize scaffold and run projection subprocess
    compare state and plan reconciliation
    injected lint preview (usually already fake)
    apply each dependency action
      write manifest
      REAL package-manager install unless dependencyProcess supplied
      restore manifest/locks on failure
    completeLocalQualitySetup before receipt persistence
      installation/tool-version/hook work as needed
    verify outputs and publish receipt
  further update/check calls for repair, refusal or repeat

Anchors: apps/cli/src/update/run.test.ts:256 (runUpdateWithPreview), :231 (cleanPreview); update/run.ts:2328 (default process), :2420–2478 (per-dependency apply/install), :6477–6516 (quality setup); scaffold/output.ts:4536 (projection subprocess). A fake general tool runtime does not replace the separate dependency process. cleanPreview does not run candidate installation/lint and does not prevent live application installs.

Static repetition examples, not measured subprocess counts:

• Each of three receipt-recovery variants at run.test.ts:950 traverses five full updates and one preview.
• Each current-handoff variant at :1629 traverses three updates and two previews.
• The ready-validation fixture at :309 performs two preparatory updates plus per-scope authoring completion. It already substitutes installation at :327, so its remaining cost is not solved by removing npm.
• The authored-wiki/starter mega-case at :1925 traverses twelve updates including preparation. Several calls merely classify one file state.
• The real installation journey at :2635 traverses five updates; cold install and lifecycle-script suppression genuinely need the installer boundary, while follow-up policy breadth may use the owning setup capability.

Recommended seam map:

Pure culls include plan-object identity assertions (:509) and a nominal explicit-check preservation test (:491) that only checks no issues/no application without proving commands survived. The latter can instead gain an actual compiled-command assertion if unique update wiring warrants one. Before/after receipt interruption, changed live inputs before publication, symlink/FIFO refusal and independent partial work are distinct safety invariants, not duplicate regressions.

Use controlled installed-state fixtures, not an unconditional no-op that invents healthy tools: subsequent checks may inspect executable versions, lock evidence and tool metadata. Keep private mutable roots. Copying a generated seed requires portability checks for absolute paths, hooks and symlinks; sharing writable roots or copying giant installed trees blindly is not the recommendation.

Some cheaper seams do not yet exist. Flat-route/hidden-placeholder behavior lives in private wikiSemanticState (run.ts:1710–1777). Supplying precomputed obligations to compareObservedState would bypass that behavior. Retain a read-only application witness or separately accept a coherent production refactor; do not call bypassed coverage equivalent.

Release recovery: a nested build farm inside two tests

Historical case durations: exact recovery/tamper 394.860s; partial/completed resume 306.165s; fixture self-test 1.226s. Removing only the fixture self-test cannot materially improve this hotspot.

recovery test → child runner → real historical Git fixture
  createReleasePublicationPlan
    readReleaseRecovery → reconstruct baseline target authority
    each first-parent entry, sequentially
      reconstruct base authority
      reconstruct target authority
      classify changed changelogs
      selected non-readback product → build target again
  validateReleasePublicationPlan
    verify history, notes and artifacts
    readback entries → reconstruct historical authority again

Each default authority reconstruction creates a detached worktree, runs bun install --frozen-lockfile --ignore-scripts, builds that revision's CLI and baseline, assembles artifacts, calculates authority and deletes the worktree/output. Adjacent entries rebuild shared revisions; validation repeats work. Anchors: apps/cli/scripts/classify-release-impact.mjs:265–365; release-publication.mjs:978–983, :1735–1774, :2041; fixture apps/cli/src/scripts/release-recovery-publication.fixture.ts:77–153; runner release-recovery-publication.runner.ts:8–99.

This is a source-derived estimate of the successful fixture paths, not a traced installation count from the old log. Every CLI build also generates a baseline and runs built-command assertions; selected baseline products add nine further baseline generations. Warm dependency archives do not eliminate installation into each isolated tree.

Correction to the original report: readback-only is not build-free. It skips retained publication artifacts after authority reconstruction. Even the completed resume with no entries reconstructs one spent recovery target (release-publication.mjs:751–760). A test that reads frozen recovery declarations without tools does not prove the full planner or validator avoids builds.

Accepted Q9 direction: use a minimal real Git history with real first-parent relationships, local remote tags, current recovery declarations, actual planner/validator and tiny valid outputs to test binding, tampering and partial/completed resume. Keep the full historical replay as an explicit diagnostic. This consciously removes routine compatibility proof for the entire old monorepo/toolchain range; it is not a claim of identical coverage. The user selected full historical replay only on demand; an additional routine historical-builder witness is not required.

Production authority memoization or changelog-first selection could also reduce work, but changes production behavior and requires separately grounded TDD. It is not silently authorized as test-only cleanup. Existing release-publication.test.ts has 61 cases in 2.560s, illustrating the much cheaper semantic layer, although its temporary module with appended private exports deserves a seam cleanup rather than blanket endorsement.

Built CLI: broad policy matrices and a diagnostic benchmark

apps/cli/src/cli/behavioral-portfolio.test.ts:351 launches the issue-215 benchmark without a scenario argument. The default lifecycle runs ten sequential CLI commands: scaffold, adoption, cold, warm, no-op, source-only, managed drift, bypass, clear and human progress. The case took 50.567s historically. It is explicitly diagnostics-only, with no latency/speedup gate (scripts/behavior-contract/issue-215-update-benchmark.mjs:747–766). Several lifecycle samples assert only generic success; human progress is collected without a content/order assertion in this branch.

Move the full diagnostic benchmark out of the normal suite. Retain a focused real cold/warm composition witness. Reducing ten commands to scaffold/adoption/cold/warm removes six launches, not a proven 60% runtime. A valid converged seed could reduce it further, but its preparation must not hide the same work. Existing runtime/scripts.test.ts:1054 already proves non-empty warning replay, source invalidation with installation reuse, missing-tool refusal and uncacheable dynamic configuration through the actual candidate-preview/cache capability.

The generic 11-command matrix (portfolio:184–258, :458–471) accepts any string-valued status or error code and does not constrain process exit status. A newly broken command emitting a typed error can pass. Delete this weak shared matrix and its registry-completeness check; preserve meaningful command-specific refusal semantics in focused tests where required. That avoids twelve CLI invocations and eleven copied/Git-initialized fixtures without pretending generic output shape proved command success.

Further hotspots are narrow policies exercised through complete consumers:

• Missing/old/correct Oxlint catalog: 26.652s, 25.154s and 24.812s; three variants each run two full updates.
• Array workspace globs: 22.819s; incompatible Effect ownership: 22.142s; exact Effect/tsgo catalog versions: 19.820s.
• Prepare adoption/preservation: 17.695s; monorepo Effect prepare lifecycle: 16.657s.
• Four consumer dependency-section variants invoke full monorepo update to vary one manifest field.

Move broad manifest/prepare permutations to real plan/reconcile/file-application seams, retaining one executable witness per distinct composition boundary. scaffold/output-root-dependencies.test.ts:17 already consumes real detection, desired output, materialization and receipts; features/scaffold-state/reconcile.test.ts:249,276 protects catalog-before-dependency ordering. These do not automatically cover every preservation/conflict input: map the missing cases explicitly.

Keep focused built-artifact dispatch, aliases where packaging matters, exact public JSON/exit/stderr semantics, one scaffold/update composition, cold/warm cache wiring, controlled failure with unchanged consumer bytes, and meaningful delegation/transport behavior. Move inexpensive shell-grammar and direct candidate-preview tests out of a file whose initialization requires a built CLI. That also reduces their cache input closure.

Historical 80% arithmetic — superseded by Q10

The historical CLI run was 2612.83s. An 80% reduction means 522.566s, about 8m43s. Other files plus runner overhead consume 193.321s, leaving only 329.245s, about 5m29s, for the three hotspots if execution remains serial. Their combined 2419.509s must fall by 86.4% under that simplifying assumption. The single exact-recovery case already exceeds the entire hotspot allowance.

This is target arithmetic, not a speed prediction on 2-vCPU hardware. The old sample used different hardware/code and failed. Successful expensive workflow samples were roughly 43–46 minutes, implying about 8.6–9.2 minutes for an 80% workflow reduction; other uncached tasks could become the next critical path. Cold dependency stores, forced test execution with warm stores, and normal task-result reuse must be measured separately.

Reduce repeated real operations first, then schedule remaining independent proof. Shards shorten elapsed time but do not eliminate work and duplicate setup; workers within one 2-vCPU VM share that CPU/disk budget. Lower cache invalidation reduces how often work executes, not the cost of a genuine miss. Capability seams help both: a catalog policy test needs manifests/planner/reconciler inputs, while a built CLI smoke legitimately depends on packaged assets. Unrelated wiki prose should not invalidate either; shipped prompts remain inputs to consumers that use them.

Next evidence needed: phase timings for fixture preparation versus assertion work; actual install/build/process counts; one passing comparable 2-vCPU baseline and reduced portfolio; distinct warm/cold measurements; total runner work including retry/setup. This arithmetic is retained only as the rationale for the deeper investigation; Q10 removes the 80% target. No runtime improvement has been measured here.

Current decisions and remaining evidence

Q1–Q7 are accepted in the grill: only 2-vCPU workers, no automated macOS jobs, no monthly ceiling/reserve, cheapest meaningful proof, precise capability caches, PR/main verification and current publication authority retained, all eight unselected operator files reconciled. Existing trigger cancellation semantics remain. Manufactured tests need no replacement when they protect no owned behavior; any affected rule's verification route must remain truthful.

R5 closes shared understanding: Q10 supersedes Q8 with no numerical target and no sharding; Q11 accepts the consolidated proposals. Q9 retains minimal real recovery history in CI and full replay on demand. The compiled specification is authoritative for delivery. Preparation timings, resource pressure, portable fixture isolation and passing 2-vCPU benchmarks remain unmeasured. Blacksmith invoice readback can refine consumption reporting but is not a monthly-cap requirement.

Work log and rule evaluation

Read global/project agent settings before dispatch; inspected .agents/subagents/manifest.mjs and selected planning-discovery ownership/guidance. Four lanes stayed readonly; only coordinator authored this consolidated wiki artifact. No shared skills, source code, tests, workflows, provider settings or external messages changed. No full suite execution was claimed. GitHub issue/PRs were inspected as references only.

Validation: wiki check:content passed before and after synthesis; the owning public-wiki-contract suite passed all six tests. The six inventory tables contain exactly 68 + 29 + 16 + 5 + 21 + 12 = 151 file rows. Mutation-specific production validation and RED/GREEN are not applicable to this research-only change. This log does not claim all retained tests pass or any candidate deletion is semantically proven.

September 22 extension validation: wiki content check and all six public wiki contract tests pass; the four touched research/grill/log files were formatted. Three resumed readonly lanes supplied the updater, release reconstruction and built CLI findings. Coordinator consolidated them, corrected the older build/budget statements and persisted both R4 answers. No timing benchmark or test implementation ran.

Delivery planning refresh — September 22

Three resumed readonly lanes translated the accepted research into implementation kernels: updater preparation and controlled installation; minimal recovery history plus precise CI/cache wiring; and the remaining app/package/wiki portfolio. The current inventory is 153 executable test files after inherited parent changes. Existing public seams support the accepted test-only transformations without production release-planner changes. Each worker must return final selectors, actual input closure, capability needs, semantic fingerprints and proof; root configuration has one dependent owner. Independent plan review resolved T9 dependency ordering and assigned the Bun 1.4 diagnostic assertion to T3. This refresh adds execution detail, not new product decisions or measured savings.