CI Test Suite Pruning Grill Status

Branch Dashboard

• Parked branches: npm release redesign, GitHub billing recovery, release execution, provider mutation, and CI runner topology after suite pruning.

Current Round

• Round: R5
• Current frontier: empty
• Shared-understanding confirmation: confirmed

Glossary

Terms

• High-Signal Test: A test that proves one named capability, public contract, or safety invariant and would fail for a meaningful product defect. Avoid: regression test, coverage test
• Behavioral Test: A test that observes a stable outcome through the nearest public seam, one semantic level above the implementation change. Avoid: integration test when the term only describes technical breadth
• Implementation-Shape Test: A test whose result depends on source text, imports, internal call order, line count, test names, fixture versions, or another internal spelling. Avoid: architecture test, contract test
• Safety Invariant: A rule that prevents corruption, containment escape, secret exposure, partial publication, unrecoverable state, or unsafe cleanup.
• Unique Witness: The smallest test that proves a behavior or safety invariant not already proved by an equal or stronger retained seam.
• CLI Behavioral Test: A Behavioral Test that starts the complete built hi or hint command as an external process and asserts exit status, output, or resulting files. Avoid: native test, module integration test
• Local Provider Substitute: A loopback HTTP server that implements only the external protocol needed by a CLI Behavioral Test. The complete CLI selects it through public configuration such as DP_CONTROL_PLANE_URL. Avoid: live provider, in-process mock
• Focused Test: A fast in-process test of a dense pure domain rule or typed protocol decoder through its exported API. It does not inject internal mocks or inspect implementation shape. Avoid: unit test as a blanket category
• Trusted CI: A run whose code is owned by this repository and may receive an OIDC cache credential and signing key.
• Untrusted Fork CI: A pull-request run executing contributor-owned code that must not receive authority to write trusted cache artifacts.

Relationships

• Every retained High-Signal Test is a Unique Witness for at least one named capability, contract, or Safety Invariant.
• A Behavioral Test observes through the nearest stable public seam.
• An Implementation-Shape Test is never retained as a behavioral witness.
• A genuine architecture invariant may have one static enforcement rule outside the test suite when behavior, compilation, and package boundaries cannot express it.
• A CLI Behavioral Test uses the complete CLI process as its public seam.
• Effect Layers remain the injection mechanism for focused in-process tests. A spawned CLI process reaches a Local Provider Substitute through its public URL configuration and the production HTTP adapter.

Axioms

• Internal rewrites must not break retained tests when behavior is unchanged.
• Test age and historical bug labels do not create retention value.
• Exact strings are asserted only when the string itself is a documented user or protocol contract.
• Deleting a test does not authorize production behavior changes.
• The target suite has no quarantine for low-signal tests.
• Classified low-signal tests are deleted without a witness audit or replacement requirement.
• Pure test deletion is outside RED/GREEN because production behavior does not change.
• Portfolio sufficiency is determined by the capability-and-safety inventory, not numeric coverage.
• Lower-level native tests do not qualify as CLI safety witnesses.
• Each displayed command has one primary full-command witness: check, ensure, init, scaffold, update, operator, report, skills, tools, and upgrade.
• hi is the primary executable. hint has one alias-parity smoke for the whole product.
• Documented JSON behavior is asserted by parsing its meaning. Human text is retained only for the command atlas and one representative failure.
• Full-command tests use real temporary repositories and Local Provider Substitutes. They do not call live providers or replace internal application services.
• Focused Tests remain only when many meaningful pure-rule or protocol-decoding cases would be wasteful or opaque through the full CLI.
• Keep exactly three extra product-wide full-command safety witnesses when publicly reproducible: containment prevents writes outside the repository; a failed write leaves no partial managed state; failures redact secrets.
• Run the primary command suite against built dist. No installed-tarball execution smoke remains; publication performs only a non-test package-integrity inspection.
• Every CI lane must participate in Turbo caching. Cache sharing must preserve the trust boundary between repository-owned code and untrusted fork code.
• Trusted internal pull requests, main, and release verification share one signed Turborepo Remote Cache namespace for identical deterministic tasks.
• Untrusted fork pull requests restore the default-branch GitHub Actions .turbo cache and write only to their isolated pull-request cache scope.
• Every deterministic build, lint, typecheck, test, and package check is a package-owned Turbo task. Root scripts only delegate with turbo run; CI does not invoke test runners or verification suites directly.
• Every deterministic retained task is cacheable, including the CLI build and Behavioral Tests. Only external publication, state mutation, and genuinely nondeterministic operations may opt out.

Flagged Ambiguities

• For CLI behavior, "Behavioral" means executing the complete public command. Native safety fault injection is excluded.
• One purpose-built static rule may enforce a genuine architecture invariant outside the test suite.
• RED/GREEN remains mandatory only when production behavior changes.
• The current code already injects Effect services and HttpClient in focused tests. Existing full-process tests also use real temporary files and loopback servers, but current provider tests do not yet prove the full hi command through DP_CONTROL_PLANE_URL.
• Current trusted and protected jobs use signed Turborepo Remote Cache through GitHub OIDC. Fork jobs have no shared Turbo cache. Root test and test:ci also execute suites before entering Turbo, and @punks/cli#build is explicitly uncached.
• Workflow fan-out, affected-package selection, aggregate job shape, and release triggering are integrated with the CI Workflow Topology grill. No lower-level CLI native safety matrix may survive under topology terminology.