Spec: Lean Pull-Request Verification and Evidence-Gated Publication

Context

Harness currently pays for repeated verification across pull requests, main, schedules, and publication. The retained test surface also contains implementation-shape assertions, literal inventories, version-churn fixtures, duplicate command matrices, and lower-level CLI safety cases that do not represent the public product seam.

Contributors need fast evidence for the exact pull-request revision they are changing. Maintainers need one high-signal portfolio whose scope follows the affected dependency graph. Release operators need npm and baseline publication to trust that exact successful evidence without running the same tests again.

This capability establishes one verification authority: the latest successful prospective pull-request Git tree. It reduces the portfolio to named public capabilities and safety outcomes, makes deterministic verification reusable through Turbo, transports authority as a small Candidate Evidence receipt, and keeps publication limited to release-specific integrity, mutation, reconciliation, and readback.

This spec supersedes conflicting test-preservation, cache-separation, installed-package execution, and release-test replay requirements in earlier CI and release specifications. It does not supersede their accepted release-impact classification, compatibility, credential, publication ordering, or recovery semantics.

Non-Goals

• Adopting Changesets or replacing reviewed repository release intent with generated versions.
• Introducing beta, canary, staged npm publication, or another public release channel.
• Changing the semantic release-impact outcomes none, baseline, npm, and mixed.
• Changing stable-baseline compatibility, publication ordering, external reconciliation, or readback semantics.
• Adding paid GitHub branch protection, required deployment reviewers, or a merge queue.
• Preserving tests because of age, historical regressions, line coverage, title inventories, or a target test count.
• Moving low-signal tests to nightly, quarantine, legacy, or release-only suites.
• Caching publication, external mutation, or genuinely nondeterministic operations.
• Defining delivery files, task names, worker assignments, or implementation order.

User Stories

US-001: Maintain a high-signal verification portfolio

As a maintainer, I want each retained test to prove a named public capability, contract, or Safety Invariant so the suite protects meaningful behavior without preserving implementation history.

US-002: Verify only the affected pull-request revision

As a contributor, I want pull-request CI to run only applicable deterministic verification for my latest prospective merge tree so feedback is fast and superseded work stops consuming runner time.

US-003: Reuse deterministic verification safely

As a contributor, I want identical deterministic work to restore from cache across eligible CI lanes while untrusted fork code cannot write trusted artifacts.

US-004: Publish only an exactly verified tree

As a release operator, I want publication to require successful Candidate Evidence for the exact final Git tree so main can publish without replaying the pull-request suite.

US-005: Keep publication small and recoverable

As a release operator, I want release execution to perform only release-specific integrity checks, mutations, reconciliation, and readback so publishing remains safe without becoming another test pipeline.

US-006: Detect only genuine external drift on schedules

As a maintainer, I want scheduled verification limited to named external behavior that pull-request CI cannot establish so schedules do not become a second full-suite backstop.

Acceptance Criteria

• AC-001: Every retained test maps to at least one named public capability, public contract, or Safety Invariant in a capability-and-safety inventory.
  • Covers: US-001
• AC-002: Tests based on source spelling, import order, internal call order, line count, test titles, fixture package versions, or historical inventory membership are absent from the retained suite.
  • Covers: US-001
• AC-003: A classified low-signal test can be deleted without a replacement witness, mutation proof, unique-witness audit, or RED/GREEN cycle when production behavior does not change.
  • Covers: US-001
• AC-004: The retained CLI portfolio has one primary full-command Behavioral Test for each displayed command: check, ensure, init, scaffold, update, operator, report, skills, tools, and upgrade.
  • Covers: US-001
• AC-005: The primary CLI Behavioral Tests execute the complete built hi process against real temporary repositories; hint receives one product-wide alias-parity smoke instead of a duplicate command matrix.
  • Covers: US-001
• AC-006: Documented JSON behavior is verified by parsing its stable meaning, while exact human text is retained only for the command atlas and one representative failure.
  • Covers: US-001
• AC-007: Full-command provider scenarios select a loopback Local Provider Substitute through public configuration and use production CLI composition rather than internal application mocks or live providers.
  • Covers: US-001
• AC-008: Focused in-process tests remain only for dense pure domain rules or typed protocol decoding exercised through an exported API without internal mocks.
  • Covers: US-001
• AC-009: The retained CLI safety portfolio contains one product-wide full-command witness for repository containment, one for absence of partial managed state after failure, and one for secret redaction when each outcome is publicly reproducible.
  • Covers: US-001
• AC-010: Lower-level CLI symlink, race, permission, rollback, interruption, or similar native fault-injection tests are absent even when the public CLI cannot reproduce them.
  • Covers: US-001
• AC-011: Pull-request verification is the only workflow that runs the retained deterministic test portfolio.
  • Covers: US-002, US-004
• AC-012: A newer revision cancels the superseded pull-request run, and only the latest successful prospective merge tree can become Verification Authority.
  • Covers: US-002, US-004
• AC-013: Affected Verification is selected through the Turbo dependency graph with the lockfile, Turbo configuration, shared TypeScript configuration, tool versions, and other result-affecting root controls declared as global inputs.
  • Covers: US-002, US-003
• AC-014: Applicable lint, typecheck, retained tests, and browser proof contribute to one Stable Aggregate Check whose name remains present when a path-owned proof is intentionally skipped.
  • Covers: US-002
• AC-015: A path-owned proof is skipped only when graph selection establishes that its owned surface and declared global inputs are unaffected.
  • Covers: US-002
• AC-016: Every deterministic build, lint, typecheck, test, browser proof, and repository policy check is owned by a package or justified Turbo root task; root scripts only delegate through Turbo and CI does not run repository test suites directly.
  • Covers: US-002, US-003
• AC-017: Every deterministic retained task, including the CLI build and CLI Behavioral Tests, is cacheable with precise inputs, outputs, environment identity, and dependency edges.
  • Covers: US-003
• AC-018: Trusted internal pull requests, main, and release verification can restore and write one signed remote-cache namespace when deterministic task identities are equal.
  • Covers: US-003
• AC-019: Untrusted fork pull requests can restore eligible default-branch GitHub Actions Turbo cache entries but cannot write a trusted remote-cache artifact; their writes remain isolated to the pull-request cache scope.
  • Covers: US-003
• AC-020: The successful Stable Aggregate Check retains a Candidate Evidence JSON artifact containing schema version, repository, workflow run id, pull-request number, head commit, tested Git tree, and successful conclusion.
  • Covers: US-004
• AC-021: Candidate Evidence is named with its tested tree, retained for 14 days, and contains no package, build output, cache data, credential, or secret.
  • Covers: US-004
• AC-022: Publication accepts only the newest valid successful same-repository, non-fork Candidate Evidence for the exact final main Git tree; missing, expired, failed, fork-originated, or mismatched evidence blocks before production mutation.
  • Covers: US-004
• AC-023: A direct main commit without matching successful Candidate Evidence performs no production mutation.
  • Covers: US-004
• AC-024: Every main push runs a read-only release-impact classifier that resolves exactly one of none, baseline, npm, or mixed and fails closed on unknown impact.
  • Covers: US-004, US-005
• AC-025: A none classification performs no product mutation; another classification enters the protected Publication Workflow only after exact-tree authority succeeds.
  • Covers: US-004, US-005
• AC-026: main and the Publication Workflow do not rerun pull-request lint, typecheck, browser, CLI Behavioral, focused, safety, or package execution suites.
  • Covers: US-004, US-005
• AC-027: Before npm publication, package integrity runs npm pack --json --dry-run and verifies the reviewed version, declared hi and hint binaries, required dist outputs, and bundled baseline files.
  • Covers: US-005
• AC-028: No pull-request, main, publication, or scheduled lane installs and executes a package tarball as a smoke test.
  • Covers: US-001, US-005
• AC-029: npm publication preserves Trusted Publishing through GitHub OIDC for the exact publication workflow and Production environment without a long-lived npm publication token.
  • Covers: US-005
• AC-030: Publication runs serialize without cancellation, external mutations never restore from cache, and a retry reconciles existing external state before advancing to the first incomplete step.
  • Covers: US-005
• AC-031: Successful publication reads back the required npm, baseline, tag, release, alias, and compatibility state defined by the selected release impact without rerunning product tests.
  • Covers: US-005
• AC-032: Scheduled or manual external-drift verification contains only individually named witnesses of current runtime, operating-system, filesystem, or provider behavior that pull-request verification cannot establish.
  • Covers: US-006
• AC-033: No weekly, scheduled, release, quarantine, or legacy workflow executes the complete retained test portfolio or deleted low-signal tests.
  • Covers: US-001, US-006

Constraints

• The repository remains private on GitHub Free; exact Candidate Evidence provides the publication guard that paid merge enforcement cannot provide.
• Pull requests never publish production artifacts.
• Release intent remains reviewed repository state; CI does not invent versions, tags, compatibility ranges, or changelog intent.
• Release-impact classification remains semantic; paths may select comparisons but do not decide product impact.
• Unknown release impact and unavailable authority fail closed.
• Publication credentials and production variables are absent from untrusted and ordinary test jobs.
• Release mutation, external publication, and genuinely nondeterministic operations may opt out of Turbo caching; historical freshness preferences may not.
• Test sufficiency is determined by the capability-and-safety inventory, not line coverage, test count, historical regressions, or the earlier 40–50% reduction estimate.
• One purpose-built static rule may enforce a genuine architecture invariant outside the test suite only when behavior, compilation, and package boundaries cannot express it.
• Existing stable release ordering, compatibility, credential ownership, immutable-version handling, reconciliation, and readback rules remain authoritative unless this spec explicitly supersedes them.

Dependency Readiness

No Stack Required.

Branch/Base Intent

Not applicable.

Accepted Technical Decisions

• Separate pull-request verification, main release intent, and external-drift observation into independent workflow responsibilities.
• Use the prospective pull-request Git tree as Verification Authority and transport it to publication with a small Candidate Evidence JSON receipt.
• Use Turbo's dependency graph and declared global inputs for affected selection instead of a parallel custom path-ownership system.
• Put deterministic repository validation inside the Turbo task graph and make it cacheable by default.
• Share identical signed cache results across trusted lanes while isolating writes from Untrusted Fork CI.
• Build at publication time, inspect npm package contents without executing the tarball, and preserve existing semantic release classification and reconciliation.
• Delete Candidate Evidence transport if a future enforceable merge queue verifies the exact final commit directly.

Accepted Testing Decisions

• Retain one strongest behavioral witness per named public capability or Safety Invariant instead of preserving historical regression breadth.
• Test the CLI through complete built external processes; reserve Focused Tests for dense pure rules and typed protocol decoding.
• Do not duplicate the CLI portfolio across hint, human/JSON presentation modes, commands with equivalent outcomes, or installed package execution.
• Delete implementation-shape tests, literal title inventories, fixture-version churn, duplicate matrices, historical migration assertions, and lower-level CLI native safety cases.
• Delete classified low-signal tests directly; require RED/GREEN only for production behavior changes.
• Keep browser proof only for affected browser-owned behavior.
• Verify package contents immediately before publication without treating package integrity as a test suite.
• Preserve scheduled execution only for named external-drift witnesses.

Verification Seams

• The capability-and-safety inventory exposes every retained witness and its named public outcome or Safety Invariant.
• The complete built hi process exposes CLI exit status, structured output, human output, resulting repository files, containment, atomic failure, and secret redaction.
• Exported pure-domain and protocol-decoder APIs expose eligible Focused Test behavior.
• Turbo task summaries expose affected selection, cache identity, cache hit or execution, restored outputs, and failure status.
• The Stable Aggregate Check exposes the complete applicable pull-request result for one prospective Git tree.
• Candidate Evidence exposes the tested Git tree and successful same-repository pull-request authority without carrying build artifacts.
• The release-impact classifier exposes none, baseline, npm, mixed, and fail-closed errors for the exact main tree.
• npm pack --json --dry-run output exposes reviewed package version, binary declarations, and required package files.
• npm registry metadata, GitHub release/tag state, baseline authority, aliases, compatibility readback, and serialized workflow state expose publication convergence.
• Named scheduled witnesses expose only current external behavior unavailable from pull-request verification.

Parked Decisions

• Beta, canary, or staged npm publication
  • Owner: Stefan.
  • Resume trigger: an explicit request to introduce another publication channel or staged promotion model.
• Merge-queue replacement for Candidate Evidence
  • Owner: repository maintainers.
  • Resume trigger: GitHub provides and the repository adopts enforceable exact-final-commit verification.
• GitHub billing recovery and paid Actions validation
  • Owner: repository administrators.
  • Resume trigger: Actions billing is restored and a paid workflow run can start jobs.
• Provider mutation and release execution
  • Owner: release operators.
  • Resume trigger: implementation is complete, billing permits execution, and an explicitly authorized release is ready.

Decision Log