Spec: Shared Verification Cache and Diff-Classified Automatic Releases

Context

Harness Intelligence already separates signed development cache authority from protected release authority and keeps fork pull requests credential-free. The current task identities and test groupings still prevent much of the replay-safe suite from contributing to one shared local and trusted-CI cache. Several controlled filesystem and subprocess tests remain in broad uncached buckets, while the generic repository test graph has no authenticated remote-cache boundary.

Release behavior has the inverse problem. Pull requests and local development can prove code before merge, but publication is manual and cannot distinguish an unchanged product from a baseline-only, npm-only, or mixed product change. Literal paths and archive bytes are insufficient authority because scaffold inputs cross directory boundaries, generated release identity changes without payload changes, and packaged skills appear in both literal artifacts even though the stable baseline is their intended delivery channel.

Maintainers need replay-safe local work to accelerate trusted pull-request CI. Contributors need the complete release candidate proven before a commit becomes eligible to publish. Release operators need each exact main tree classified and converged automatically without republishing unchanged products or weakening credential, cache, artifact, compatibility, and retry boundaries.

The repository remains private on GitHub Free. Pull-request checks cannot be enforced as merge restrictions, and the project will not buy branch protection. The publication workflow must therefore treat exact successful pull-request candidate evidence as its own fail-closed precondition. A direct, red, cancelled, or unattested main tree may exist but cannot mutate production release state.

This spec extends the implemented cli-release-test-caching contract. It retains signed trust separation, fork isolation, complete inventory, uncached external mutations, and idempotent existing-version recovery. It supersedes that spec only where it limited cross-platform reuse, left broad native/ambient/update groups uncached, or parked release-state reconciliation.

Non-Goals

• Pay for GitHub Team, add branch protection, add a merge queue, or require human review before merge.
• Introduce beta, canary, prerelease, or npm staged-publication channels.
• Add a human production approval gate.
• Publish from a direct, red, cancelled, or otherwise unattested main tree.
• Cache npm publication, npm dist-tag mutation, GitHub release mutation, baseline promotion, or another external release side effect.
• Give fork pull requests authenticated development-cache or publication credentials.
• Let protected publication consume a development-cache artifact.
• Use GitHub path filters, changelog paths, or raw archive digests as release authority.
• Invent release versions, tags, compatibility ranges, or changelog entries in CI.
• Coalesce, skip, overwrite, or reuse an explicitly reviewed product version.
• Split baseline asset upload from stable promotion, add delayed promotion, or create a release-authority service.
• Run the complete release-candidate graph on every local commit.
• Require a bilateral forced-fresh macOS-to-Ubuntu and Ubuntu-to-macOS cache qualification suite.
• Freeze the research report's proposed classifier JSON fields or fingerprint algorithm as the implementation contract.
• Define implementation files, commands, worker assignments, or delivery order.

User Stories

US-001: Build one sound development cache

As a contributor, I want authenticated local verification and trusted same-repository CI to reuse every compatible result so that repeated checks become faster without replaying host-incompatible evidence.

US-002: Catch release-candidate failures before publication

As a contributor, I want fast local feedback and one complete pull-request candidate proof so that product, package, and test-graph failures are visible before any production release can occur.

US-003: Classify the product change accurately

As a maintainer, I want each candidate classified as no release, baseline-only, npm-only, or mixed from semantic product evidence so that unrelated or single-product changes do not publish the wrong artifact.

US-004: Converge stable releases automatically and recoverably

As a release operator, I want an eligible main tree to publish the required stable products in reviewed version order and reconcile partial success on retry so that production converges without a separate release service.

US-005: Preserve complete and compatible consumer artifacts

As a CLI consumer, I want every published npm package and stable baseline to carry complete, verified, mutually compatible content so that installation, scaffold retrieval, and fallback behavior remain usable.

US-006: Keep release authority narrow on GitHub Free

As a repository operator, I want cache and publication credentials confined to their accepted jobs and external systems so that automatic production release does not depend on paid GitHub gates or expose authority to untrusted code.

Acceptance Criteria

• AC-001: Authenticated local development and same-repository pull-request CI can both read and write signed development-cache entries when their complete task identity matches.
  • Covers: US-001
• AC-002: Fork pull-request execution receives no authenticated development-cache or publication credential and can still execute the required verification without a trusted hit.
  • Covers: US-001, US-006
• AC-003: Protected publication uses separate signed authority and rejects every development-cache artifact regardless of matching source or output bytes.
  • Covers: US-004, US-006
• AC-004: A portable task can reuse one result across local macOS and trusted Ubuntu CI only when its behavior is platform-independent and its source, fixtures, dependencies, controlled environment, Node identity, and Bun identity match.
  • Covers: US-001
• AC-005: A test that observes or branches on operating-system or architecture behavior is excluded from the portable class until that behavior is removed or controlled.
  • Covers: US-001
• AC-006: Host-dependent filesystem, subprocess, signal, native, and container verification is cacheable only under a complete capability identity representing every result-affecting host and tool property.
  • Covers: US-001
• AC-007: Every replay-safe case in the current explicit update and ambient groups is reassigned to a capability-keyed task instead of inheriting a group-wide bypass; any retained fresh case satisfies AC-008.
  • Covers: US-001
• AC-008: Fresh execution is limited to the smallest tests whose asserted property is current scheduling, contention, elapsed time, external lifecycle, or external mutation, and every bypass records that reason.
  • Covers: US-001, US-002
• AC-009: Before trusted generic CI receives development-cache authority, repository-wide tests are partitioned so pure tests use portable tasks, host-dependent tests use capability-keyed tasks, and only minimal live-integration witnesses remain fresh.
  • Covers: US-001, US-002
• AC-010: A cacheable build restores its complete declared output inventory; cross-platform reuse is permitted only for proven byte-equivalent output, otherwise the build remains cacheable under a narrower runtime or capability identity.
  • Covers: US-001, US-005
• AC-011: npm publication, dist-tag mutation, GitHub release mutation, baseline publication, and stable promotion always execute rather than resolve from cache.
  • Covers: US-004, US-006
• AC-012: A pre-commit gate selects fast verification from the staged diff through a worktree-aware boundary rather than reusing the existing commit-to-commit selector unchanged.
  • Covers: US-002
• AC-013: A pre-push gate runs the complete release-candidate graph for the exact committed tree with the pinned CI runtime and canonical environment.
  • Covers: US-002
• AC-014: The complete local candidate command is directly callable for diagnosis and is the same behavioral gate used by trusted pull-request CI.
  • Covers: US-002
• AC-015: Pull-request CI retains fast diff-aware feedback and also evaluates the complete prospective merge tree before issuing publication-eligible candidate evidence.
  • Covers: US-002
• AC-016: Complete candidate verification includes the release-shaped environment, full repository verification, baseline assembly, npm package assembly when applicable, and complete artifact inventory.
  • Covers: US-002, US-005
• AC-017: Candidate evidence is bound to the exact prospective tree and assembled artifact identities; a failed, timed-out, cancelled, incomplete, or mismatched run issues no eligible evidence.
  • Covers: US-002, US-004
• AC-018: Every exact candidate resolves to one observable release kind: none, baseline, npm, or mixed.
  • Covers: US-003
• AC-019: Classification is determined by semantic product comparison; changed paths may select comparisons but cannot supply the final product decision.
  • Covers: US-003
• AC-020: Additions, deletions, and renames evaluate both old and new product ownership, and an unknown or unavailable authority blocks classification rather than resolving to none.
  • Covers: US-003
• AC-021: A stable baseline publication is required when installable scaffold payload, scaffold schema, or declared CLI compatibility changes, while release identity or timestamp changes without product change do not require publication.
  • Covers: US-003, US-005
• AC-022: npm publication is required when executable CLI behavior, the reviewed public package surface, or package-production behavior changes.
  • Covers: US-003, US-005
• AC-023: A packaged-skill-only or non-runtime scaffold-payload change classifies as baseline-only even when rebuilding the literal npm tarball would change bundled fallback bytes.
  • Covers: US-003
• AC-024: A CLI-only change classifies as npm-only when the active stable baseline accepts the reviewed CLI version and the stable scaffold payload is unchanged.
  • Covers: US-003, US-005
• AC-025: A change that affects both semantic product surfaces classifies as mixed, and docs-, test-, or release-infrastructure-only changes create no product release unless semantic comparison proves otherwise.
  • Covers: US-003
• AC-026: Every npm package assembly builds and includes a complete bundled baseline snapshot even when stable baseline publication is not required.
  • Covers: US-003, US-005
• AC-027: A pull request classified as npm or mixed contains a reviewed package version and matching npm changelog entry; missing or unchanged npm intent blocks publication eligibility.
  • Covers: US-002, US-003, US-004
• AC-028: A pull request classified as baseline or mixed contains an exact baseline tag, bounded compatibility range, and matching baseline changelog entry; missing baseline intent blocks publication eligibility.
  • Covers: US-002, US-003, US-004
• AC-029: A stable baseline compatibility family begins at its reviewed supported CLI version and ends before the next minor version, for example >=3.1.8 <3.2.0.
  • Covers: US-005
• AC-030: Every CLI patch publication proves the exact reviewed CLI against the active stable baseline consumer matrix before an npm-only release is eligible.
  • Covers: US-003, US-005
• AC-031: A failed compatibility proof or a new CLI minor requires mixed release intent and a new compatible patch family.
  • Covers: US-003, US-005
• AC-032: An unchanged compatible stable baseline can serve later CLI patch releases without another stable baseline publication.
  • Covers: US-003, US-005
• AC-033: A mixed release builds and publishes the stable baseline before publishing npm, including at a next-minor boundary where the accepted temporary interval may reject installed prior-minor CLIs.
  • Covers: US-004, US-005
• AC-034: Every push to main runs the classifier for the exact event tree, including pushes that ultimately perform no production mutation.
  • Covers: US-003, US-004
• AC-035: Production mutation requires successful pull-request candidate evidence for the exact current tree; direct, red, cancelled, or unattested main trees publish nothing.
  • Covers: US-002, US-004, US-006
• AC-036: The main release workflow verifies exact candidate evidence and does not rediscover product-test results by rerunning the complete long verification graph before publication.
  • Covers: US-002, US-004
• AC-037: none records evidence and performs no release mutation; baseline, npm, and mixed outcomes mutate only their required stable products plus required mutable aliases and evidence.
  • Covers: US-003, US-004
• AC-038: Production stable is the only release channel, and successful npm convergence makes latest and next resolve to the same exact production version.
  • Covers: US-004, US-005
• AC-039: Every explicitly reviewed product version is processed serially in version order and is never silently coalesced into a later merge.
  • Covers: US-004
• AC-040: When a reviewed version fails, it stays first; retry reconciles already-completed immutable and mutable external steps, resumes at the first missing step, and later versions wait.
  • Covers: US-004
• AC-041: An npm version already present in the registry is never republished; retry verifies its exact external state and reconciles missing allowed aliases or release records.
  • Covers: US-004, US-005
• AC-042: Every published npm package includes the complete bundled baseline, runtime dependency metadata, pack controls, and declared distribution outputs within one frozen inventory and attestation boundary.
  • Covers: US-005
• AC-043: npm completion evidence records the exact registry tarball, installed package tree, and installed-consumer matrix for the reviewed version; a failed post-publication consumer proof leaves the workflow failed, and retry may repeat that proof without overwriting or republishing the immutable version.
  • Covers: US-004, US-005
• AC-044: Baseline completion evidence verifies the immutable release tag target, manifest and archive identities, compatibility, durable stable promotion, and production readback.
  • Covers: US-004, US-005
• AC-045: GitHub supplies the existing Devpunks Vercel access token, organization ID, and API project ID, and production variables are injected only into the baseline-promotion child.
  • Covers: US-004, US-006
• AC-046: npm publication authenticates through Trusted Publishing bound to the exact workflow and existing case-sensitive Production GitHub environment, with no long-lived npm publication token in the job.
  • Covers: US-004, US-006
• AC-047: Automatic publication lands dormant, Trusted Publishing is configured, a later reviewed change enables it, and one proven OIDC publication precedes revocation of legacy bypass-token authority.
  • Covers: US-004, US-006
• AC-048: The existing Production environment is used as an OIDC trust selector without required reviewers or paid deployment protection, and lack of GitHub branch protection does not weaken the exact-evidence publisher guard.
  • Covers: US-004, US-006
• AC-049: Every verification test belongs to exactly one portable, capability-keyed, or fresh execution group, and a missing or duplicate assignment fails the inventory contract.
  • Covers: US-001, US-002
• AC-050: The accepted complete test inventory and schedule remain represented exactly once across local candidate and trusted CI execution after cache reclassification.
  • Covers: US-001, US-002
• AC-051: Cache acceptance proves forced execution, first-run misses, compatible repeat hits, output restoration after deletion, invalidation for every declared identity change, signed-authority rejection, byte-equivalent outputs, execution of every fresh witness, and a clean repository state.
  • Covers: US-001, US-002, US-005

Constraints

• The repository remains private on GitHub Free and uses only capabilities available on that plan.
• Pull-request checks are publication preconditions, not enforceable merge restrictions.
• Cache policy applies to complete restorable tasks; a replay cannot claim that a current concurrency schedule, external lifecycle, or mutation occurred.
• Every result-affecting input must be controlled or represented in the selected portable or capability identity.
• Development, fork, and protected authority remain separate. Local development never receives protected release authority.
• Remote cache artifacts remain signed and verified within their trust boundary.
• Release-impact paths are routing hints only. Semantic artifact behavior is authoritative and unknown impact fails closed.
• Literal npm tarball drift is evidence distinct from the semantic npm-publication requirement.
• npm versions and immutable release assets are never overwritten or reused.
• Stable baseline compatibility never extends beyond its reviewed minor family.
• Stable baseline promotion remains the existing durable production-authority mutation.
• A bundled baseline build inside npm assembly is independent of stable baseline promotion.
• Publication credentials and production variables are absent from cache producers, test jobs, package builders, and fork execution.
• Release versions, tags, ranges, and changelog intent are reviewed repository inputs rather than CI-generated values.
• The existing no-op completion marker carries no test result and need not become a cache artifact.

Dependency Readiness

No Stack Required.

The following foundations and activation inputs are ready and do not form an upstream delivery stack:

• The signed development/protected cache foundation and protected grouped build-attestation behavior are landed on main; relevant immutable evidence includes c8f6e71f9c988e80de1a410a563360039d198482 and c4dcca3134eace76750fa899f3b581411af094d0.
• The accepted base is main@caee8be43d38bfaaaff4cc3c90cb14c8b82c245a.
• Initial classification research is retained at research/cache-release-diff-classification@7e196b2223d39e8767fb76356f7c56fb0630d966.
• Reopened cache and Actions research is retained at research/cache-release-green-main-gaps@eabdd5f7206f0aeeb3a3aaba4ff31de5dc30d1f8.
• The accepted Vercel dependency exists under scope dev-punks as project harness-intelligence-api, project ID prj_Us1K3pmpyCJiAGLQfiPpAoOBhHGt, organization ID team_0QvyOroTH1I7k8hWMqSRCOqH.
• npm Trusted Publisher configuration is an accepted activation step inside this capability, not an upstream stack dependency.
• Prototype verdict: not applicable because no prototype was produced or required; the capability is grounded in retained repository, release, cache-replay, and Actions evidence.

Branch/Base Intent

• Base: origin/main at caee8be43d38bfaaaff4cc3c90cb14c8b82c245a.
• Parent stack: none.
• Constraint: retain both immutable research refs and the confirmed grill, then use one feature/spec branch into main.

Accepted Technical Decisions

• Model reusable verification as portable, capability-keyed, and minimal fresh-witness classes.
• Make replay-safe verification and build work cacheable by default; require an explicit current-execution rationale for every bypass.
• Separate trusted same-repository generic CI from credential-free fork execution before adding repository-wide development-cache authority.
• Preserve the development/protected signature boundary while allowing authenticated local and trusted PR producers to populate the same development entries.
• Use worktree-aware staged selection for pre-commit and complete exact-tree verification for pre-push.
• Produce exact-tree candidate evidence from the prospective merge tree and complete assembled artifacts.
• Use one semantic classifier for PR intent and main convergence; expose machine-readable outcome and evidence without freezing the research proposal's field layout or fingerprint algorithm.
• Report semantic publication requirements separately from literal artifact-byte drift.
• Keep baseline publication, npm publication, and baseline build for npm assembly as independent facts.
• Preserve stable-baseline delivery for packaged skills even though the npm-bundled fallback refreshes only at the next npm release.
• Preserve one complete npm assembler and attestation boundary after CLI build and bundled-baseline build.
• Keep the simple accepted mixed order: build baseline, publish stable baseline, then publish npm.
• Serialize reviewed versions without cancellation or coalescing and reconcile external state idempotently on retry.
• Treat exact green pull-request evidence as the publication guard available on GitHub Free.
• Inject Vercel production variables only around baseline promotion.
• Bind npm OIDC to the exact publisher workflow and existing Production environment, without an approval gate.

Accepted Testing Decisions

• Cover none, baseline-only, npm-only, mixed, infrastructure-only, and unknown classification outcomes.
• Cover additions, deletions, renames, payload-neutral release identity changes, compatibility-only changes, and dual-surface changes.
• Prove packaged-skills behavior as baseline required, npm not required, and literal npm drift reported separately.
• Prove npm-required intent fails when version or changelog is unchanged or missing, and baseline-required intent fails when tag, range, or changelog is missing.
• Prove portable cache hits only under complete portable identity and capability hits only under complete capability identity.
• Preserve exact-one-group test assignment and fail the inventory contract for every missing or duplicate assignment.
• Preserve the complete accepted test inventory and schedule while changing cache eligibility.
• Prove forced execution, producer misses, compatible replay hits, output restoration, identity invalidation, signature rejection, byte equality, fresh-witness execution, and clean repository state.
• Prove every fresh witness executes and states its unreplayable property.
• Do not require a separate bilateral forced-fresh cross-platform qualification suite.
• Prove staged-diff selection, complete pre-push execution, and the directly callable candidate boundary.
• Prove candidate evidence binds the exact prospective tree and complete package/baseline assembly.
• Prove failed, timed-out, cancelled, incomplete, and artifact-mismatched candidates cannot authorize publication.
• Prove direct and unattested main pushes classify but cannot mutate release state.
• Prove same-minor compatibility, compatibility failure, and next-minor family behavior through exact consumer matrices.
• Prove complete npm package contents, registry tarball identity, installed tree identity, and installed-consumer scenarios.
• Prove baseline release target, immutable asset identities, durable promotion, and production readback.
• Prove existing-version retry, partial baseline/npm success, mutable alias reconciliation, concurrent release serialization, and later-version blocking.
• Prove fork credential isolation, development/protected cache isolation, step-scoped Vercel injection, and npm OIDC without a publication token.

Verification Seams

• Development cache summaries expose task identity, authority, signature validation, hit, miss, restoration, and bypass reason.
• The local pre-commit and pre-push boundaries expose selected scope, exact tree, canonical environment, and pass/fail status.
• Pull-request candidate evidence exposes the exact prospective tree, release intent, release kind, complete verification result, and assembled artifact identities.
• The main publication guard exposes whether exact successful candidate evidence exists for the current tree.
• Classifier output exposes none, baseline, npm, mixed, literal artifact drift, semantic reasons, and fail-closed errors.
• The active stable-baseline consumer matrix exposes compatibility for the exact reviewed CLI version.
• Immutable GitHub release/tag readback and durable control-plane readback expose baseline publication and promotion.
• npm registry metadata, dist-tags, tarball bytes, installed tree, and consumer results expose npm convergence.
• Serialized workflow state and external readback expose the first incomplete release step and prevent later-version advancement.
• GitHub OIDC claims and the selected Production environment expose npm Trusted Publisher identity; child-process environment observation exposes Vercel variable confinement.

Parked Decisions

• Beta release channel
  • Owner: Stefan.
  • Resume trigger: an explicit request to introduce a beta channel.
• Canary release channel
  • Owner: Stefan.
  • Resume trigger: an explicit request to introduce a canary channel.
• npm staged publication or human approval gate
  • Owner: Stefan.
  • Resume trigger: an explicit request to add staged publication or a human approval gate.

Decision Log