Harness Intelligence Wiki
Research

Cache Sharing and Diff-Classified Release Research

Cache Sharing and Diff-Classified Release Research

Executive answer

Yes. The CI graph can share eligible local test results and can choose no release, baseline-only, npm-only, or mixed release after a merge to main. The release decision must come from semantic artifact comparison, not GitHub path filters.

The minimum safe design has three independent facts:

  1. baselinePublicationRequired: the stable scaffold payload or its CLI compatibility changed.
  2. npmPublicationRequired: the executable or public npm package surface changed.
  3. baselineBuildRequiredForNpmAssembly: always true when producing an npm package, because the package contains dist/baseline/**.

This distinction gives the desired policy: a packaged-skill-only change can publish a baseline without publishing npm, while a CLI-code-only change can publish npm without promoting a new stable baseline. Every npm release still builds a bundled baseline snapshot before its complete package inventory and attestation.

Scope and method

Three readonly lanes inspected the current cache/trust graph, the scaffold baseline producer, and the npm producer/retry flow. The coordinator reconciled them against the restored test-graph handoff and the completed 3.1.8 release handoff at repository commit 4a31049a13fb3163849de9c51977eb9a8b81e30e.

This report proposes architecture and acceptance boundaries. It does not change workflows, publish artifacts, or choose the unresolved compatibility policy.

Current facts

The cache graph is already trust-separated, but local and CI identities do not match

  • Same-repository PR jobs receive OIDC remote-cache authority, the development namespace, and the development signature key. Fork jobs receive none of those credentials. Protected main and scheduled jobs use a separate protected namespace and signature key. Sources: workflow development jobs, fork jobs, and protected jobs.
  • The generic behavior-contract job has no remote-cache authority. Therefore repository-wide local-to-CI sharing does not exist; only the credentialed CLI Turbo graph is a candidate. Source: generic behavior-contract job.
  • CLI task hashes include controlled runtime inputs such as CI state, platform, architecture, Node, and Bun. A macOS local run and ubuntu-latest therefore miss even when they use the same remote backend. Native capability identity is intentionally host-shaped and must remain non-portable. Ambient, forced-native, complete, and explicit uncached-update work is not remotely reusable. Sources: runtime environment construction, native and uncached task policy, and development routing.

Path rules cannot correctly classify releases

  • The baseline producer serializes five imported catalogs, selected hook assets, every CLI data script, every CLI subagent, and every packaged skill. Source: baseline producer and payload assembly.
  • Those recursive copies currently make tests and fixtures beneath the copied script/subagent trees real artifact inputs. Classification must retain that behavior unless the baseline producer is deliberately narrowed first.
  • A raw baseline archive digest cannot identify semantic scaffold change. The producer embeds commit, release version, tag, compatibility, and generated time, so every commit can change archive bytes even when the scaffold payload is unchanged. Source: baseline release identity.
  • Skills are literal bytes in both products: the CLI distribution copies them to dist/skills, and the baseline copies them into the scaffold payload. A skill-only diff therefore changes a rebuilt npm tarball even though repository policy permits delivery through a baseline without an npm publication. Source: CLI bundled assets, distribution assembly, and baseline skills assembly.
  • Exact baseline compatibility creates a second trigger independent of payload bytes. The current r1 and r2 changelog entries describe unchanged behavior but rotate compatibility from =3.1.7 to =3.1.8. Source: baseline changelog.

Current publication seams are insufficient for unattended mixed releases

  • npm publication is manual workflow_dispatch; protected verification runs automatically on main, but no baseline publisher runs in CI. Source: protected aggregate and manual publisher.
  • The baseline command builds, creates or reconciles GitHub release assets, and promotes stable authority in one process. Candidate upload and stable promotion cannot currently be sequenced separately. Source: baseline publisher.
  • The npm dispatcher preserves a useful retry contract: it probes the exact version before expensive verification and reconciles an existing version instead of republishing it. Source: release route.
  • The proven npm assembly boundary is build -> baseline:build -> isolated package preparation -> complete inventory/attestation -> npm publish. The current build inventory is written before baseline:build, while baseline:build later adds dist/baseline/**; simply adding that command after current attestation would leave those files outside the evidence boundary. Sources: build inventory timing, baseline task, and protected attestation task pair.

Implement one readonly, versioned classifier used by PR verification and recomputed by the main release workflow. GitHub paths filters may skip obviously unrelated setup work, but they must not be the release authority.

The JSON contract should expose at least:

{
  "schemaVersion": 1,
  "baselinePayloadChanged": false,
  "baselineCompatibilityChanged": false,
  "cliExecutableChanged": false,
  "npmPublicMetadataChanged": false,
  "releaseInfrastructureChanged": false,
  "npmTarballWouldChange": false,
  "baselinePublicationRequired": false,
  "npmPublicationRequired": false,
  "baselineBuildRequiredForNpmAssembly": false,
  "releaseKind": "none",
  "classificationError": null,
  "evidence": {}
}

Baseline predicate

Build the canonical base and head scaffold payloads with fixed release identity, or compare head to the verified active stable payload. Fingerprint ordered path, entry type, executable bit, and bytes. Exclude only baseline.json fields that name the release: version, tag, commit, archive digest, and generated time. Evaluate CLI compatibility separately.

baselinePublicationRequired = baselinePayloadChanged || baselineSchemaChanged || baselineCompatibilityChanged.

This catches imported-catalog, schema, executable-mode, and generated payload changes that path rules miss, while avoiding a baseline release merely because HEAD or timestamp changed. If active stable authority cannot be fetched and verified, classification fails closed.

npm predicate

Compare a normalized production entry bundle and public package surface between base and head. Build the entry bundle with bundled-baseline identity fixed to a sentinel so skill bytes and a regenerated digest do not masquerade as executable behavior. Include reachable workspace production dependencies, package name/version/bin/type/runtime dependencies, build and package-assembly producers, and pack controls. Fail closed on ambiguous CLI-owned or dependency changes.

npmPublicationRequired = cliExecutableChanged || npmPublicMetadataChanged || npmProducerChanged.

Also report npmTarballWouldChange. This is intentionally broader than publication policy: skills-only changes make it true while leaving npmPublicationRequired false. The bundled fallback refreshes at the next npm release; the compatible stable baseline supplies the newer skills meanwhile.

Preliminary path facets

Diff both old and new paths for additions, deletions, and renames, then use these facets to choose which authoritative comparisons to run:

FacetExamplesAuthority
Baseline payloadskills, scaffold catalogs, scripts, subagents, hook assetsNormalized baseline payload digest
CLI executablereachable CLI and workspace production codeNormalized entry-bundle digest
npm metadataversion, bin, runtime dependencies, pack controlsCurated assembled package metadata/tree
Release infrastructureworkflows, Turbo graph, publishers, evidence codeValidation only; never product release by itself
Tests/docstests, wiki, runbooksNo product release by itself

Unknown impact is a classifier failure, not releaseKind: none.

Decision matrix

ChangeBaseline publishnpm publishNotes
Docs/tests onlyNoNoRun affected validation.
Packaged skills onlyYesNonpmTarballWouldChange=true; stable baseline is the delivery channel.
Non-runtime scaffold data/scripts/subagentsYesNoDetermined by normalized payload, not path alone.
Reachable CLI production codeCompatibility-dependentYesWith exact compatibility, a new CLI version also needs compatibility publication.
Catalog/projection code used by both productsYesYesMixed release.
CLI code plus skill payloadYesYesMixed release.
Public npm metadata or package producerNo unless compatibility/payload changesYesAssemble and attest full package.
Release workflow/Turbo/publisher onlyNoNoProtected verification and classifier contract tests required.
Unknown CLI-owned changeBlockBlockRequire classifier ownership update or explicit reviewed override.

CI architecture

Pull requests: classify and prove intent; never publish

  1. Resolve the merge-base and head SHA with full history. Classify old and new paths, then run the canonical comparisons.
  2. Emit a human-readable summary and machine JSON as a required release-impact check.
  3. If baseline publication is required, require an explicit baseline release-intent file or equivalent version/range input plus a matching BASELINE_CHANGELOG.md entry.
  4. If npm publication is required, require a version bump plus a matching CHANGELOG.md entry. Fail if the intended exact version already exists with different bytes.
  5. Run development verification with development cache authority only for same-repository PRs. Forks remain credential-free.

main: recompute state and converge

Use a separate release workflow on push to main, checked out at the exact event SHA. Serialize it with a non-cancelling release concurrency group. Recompute against the current verified stable baseline and npm registry; do not trust a PR artifact or only HEAD^, because queued merges and retries can otherwise skip a required publication.

  • none: record evidence and exit.
  • baseline: build immutable candidate, verify with an already-published compatible CLI, upload/reconcile assets, then promote stable.
  • npm: verify the current stable range accepts the new CLI; assemble build -> baseline:build -> isolated package; inventory and attest the complete tree; publish or reconcile; run exact registry-installed consumer proof.
  • mixed: build/upload the immutable baseline candidate without promotion; assemble and publish npm; run exact registry-installed consumer proof against the candidate; only then promote stable.

The mixed order prevents stable authority from requiring a CLI version that npm failed to publish. It requires splitting the present combined baseline publisher into candidate upload and promotion seams. Existing-version retries remain idempotent: reconcile release/tag state, rerun exact registry proof, and promote only after the full matrix passes.

Create the baseline release tag explicitly at the checked-out GITHUB_SHA and verify it equals the manifest commit. The current gh release create call does not provide --target or verify an existing tag, which is unsafe when serialized work runs after newer merges. Source: current release creation.

Keep publication credentials step-scoped and outside cache producers. Use a protected deployment environment if the repository plan supports the required approval controls. GitHub documents that environment approval gates access to environment secrets, and fork pull-request workflows receive no secrets and a read-only GITHUB_TOKEN: deployment environments and fork pull-request behavior.

Local-to-CI cache architecture

Treat “shared cache” as same backend plus equal, proven task identity—not merely the same Vercel team.

  1. Scope the first delivery to existing cacheable CLI Turbo tasks. Direct root behavior-contract tests have no remote authority and are not independently restorable.
  2. Give local development the same development namespace and signature authority as trusted PR jobs. Never give local machines protected authority.
  3. Pin the same Node and Bun versions locally and in CI.
  4. Preserve host identity for build and native tasks. Native, ambient, complete, and update-uncached remain local executions.
  5. For deterministic tests only, introduce a versioned “deterministic test ABI” after proving macOS/Linux fresh output equality. Hash that ABI instead of raw OS/architecture and canonicalize CI, locale, timezone, Node, and Bun. Until that proof exists, cross-OS misses are correct.
  6. Add acceptance tests for local producer -> trusted PR remote hit, CI producer -> local hit, changed input -> miss, restored outputs, invalid signature rejection, fork no-credential execution, and protected namespace isolation.

Pin the remote-cache setup action by immutable commit before treating it as protected release infrastructure. The workflow currently references the mutable v1.0.0 tag.

Remote artifact signing remains mandatory; it is an integrity check, not permission to reuse a host-shaped result across incompatible runtimes. Source: repository signature setting and Turborepo remote caching documentation.

Conflicts and unresolved decisions

  1. Exact compatibility conflicts with “CLI-only never baseline.” The active policy uses exact ranges. A new CLI version therefore needs a compatibility-only baseline even when scaffold bytes do not change. Choose either tested overlap ranges for compatible patch releases, or accept compatibility-only baseline publications for every CLI version. An overlap range is preferable only when the consumer matrix proves it.
  2. Skills are present in both literal artifacts. The requested semantic policy is sound, but it intentionally allows the npm-bundled fallback to lag until the next npm release. Make that explicit in the contract and runbook.
  3. One npm assembler must become authoritative. The current dispatcher creates sanitized metadata, while the proven isolated preparation preserved the complete package metadata and nested pack control. Converge them and contract-test equality, including runtime dependencies and dist/baseline/**.
  4. Merge-gate capability is not currently proven. On 2026-08-10, GitHub's ruleset and branch-protection API returned a plan-level 403 for this private repository, and PR 120 was observed completing CLI checks after merge. The workflow can expose aggregate checks now, but making them mandatory requires supported branch protection/rulesets or a procedural merge policy. GitHub documents plan/repository visibility availability in About protected branches.
  5. GitHub release concurrency is not a release ledger. Serialize main releases and make every operation reconcilable against external state. Do not cancel an in-progress publisher. GitHub's concurrency behavior is documented at Workflow and job concurrency.

Required contract tests before automation

  • Add/delete/rename handling using both old and new paths.
  • No-op, baseline-only, npm-only, mixed, infrastructure-only, and unknown matrices.
  • Raw baseline identity drift without payload drift.
  • Payload unchanged with compatibility changed.
  • Skills-only: baseline true, npm false, tarball drift true.
  • npm-required with unchanged version fails PR readiness.
  • Complete npm assembly contains and attests dist/baseline/** plus runtime dependencies.
  • Existing-version retry and concurrent publish race reconciliation.
  • Baseline candidate success followed by npm failure leaves stable unchanged.
  • npm success followed by consumer failure leaves stable unchanged and reports that a new npm version is required.
  • Exact registry package plus candidate/stable consumer matrix before promotion.

Planning conclusion

The architecture is feasible and should be specified as two deliverables with one integration point:

  1. A pure release-impact classifier with canonical baseline, executable, and package-surface fingerprints.
  2. An idempotent main release orchestrator with separate baseline candidate and promotion operations and a complete npm assembly/attestation boundary.
  3. Integration into the existing development/protected cache graph, followed by a narrow deterministic cross-OS cache proof.

Before implementation, decide compatibility policy, whether release publication is fully automatic or environment-approved, and whether the repository plan will support mandatory checks. Then route the accepted decisions through requirements/spec planning; the implementation graph can keep classifier, baseline seams, npm assembly, and workflow integration in disjoint ownership lanes.

On this page