CI Suite Reduction and npm Release Research

Executive answer

Harness should cut the CLI test corpus by 40–50%, reduce an internal pull request from 11 active runner jobs to 2–3, and make npm publication a single protected job that builds and publishes an already-reviewed version without replaying the test suite.

The current suite is structurally oversized for the shipped command surface. At origin/main commit 5990a041, the repository has 225 test files and 100,944 test lines. CLI tests alone contain 65,310 lines, exceeding the CLI's 58,117 non-test TypeScript/JavaScript lines. The CLI also carries 43 fixture files totaling 429,340 bytes. These are static source measurements, not runtime estimates.

The cost comes from both corpus size and execution topology. An internal pull request launches 10 active jobs in behavior-contract.yml plus the release guard. The root test:ci preamble duplicates five Effect files, three behavior-contract files, and all 66 cutover/isolation titles before the classified repository chain runs them again. The CLI deterministic policy hardcodes 100 files and starts a separate Vitest process for every file on a cache miss. Sources: workflow, root scripts, repository inventory, and release policy.

The npm problem is narrower. Live registry readback on 2026-08-14 still reports @punks/cli@3.2.0; 3.2.1 and GitHub release v3.2.1 are absent. The last diagnosed publish failure occurred because the nested publish process dropped GITHUB_ACTIONS, causing npm to skip its GitHub OIDC path even though direct token exchange succeeded. Commit 5f5678ec forwards the marker through both release boundaries, but GitHub's failed-payment authorization hold prevented a validating Actions run. This remains an external execution blocker, not evidence that trusted publishing needs another redesign. Sources: npm OIDC detection, authority environment, and convergence environment.

Research coverage

Three readonly lanes inspected the pinned origin/main tree:

1. CI graph, triggers, fan-out, duplicate execution, and cache boundaries.
2. Test value, fixture churn, source-string assertions, duplicated regressions, and native safety boundaries.
3. npm trusted publishing and current TanStack Query, Router, and Table release patterns.

The coordinator also asked Grok to search X for current practitioner guidance, then opened and verified the retained posts directly. X evidence is treated as practitioner context, not primary authority for the repository decisions.

Highest-confidence deletions

Suites to collapse

Update integration matrix

run.test-cases.test.ts is 7,182 lines with 96 update integrations. It rebuilds large filesystem/scaffold contexts for package-manager variants and contains at least 11 full-update Claude file/symlink permutations. One regression named for yes passes write: true, so it does not test yes at all (test).

Keep one end-to-end apply smoke. Table-test package-manager and flag normalization as pure policy. Exercise managed-file state matrices through reconciliation and scoped-filesystem seams. Retain native coverage only for symlink, hardlink, permissions, containment, atomic replacement, rollback, and time-of-check/time-of-use boundaries.

Packaged CLI output

public-output-contract.test.ts is 1,988 lines and repeatedly spawns dist. Root guide, wizard, help, command help, operator markers, JSON actions, and fixtures overlap. Keep one packaged seam smoke per public mode: help atlas, installed version, check --json, one write path, and one redacted failure. Test argv classification, presentation, and operator policy in-process.

Skill and source content

content.test.ts is 1,527 lines and asserts many exact phrases. Reusable skill wording belongs to the canonical wearedevpunks-skills source repository. Harness should verify projection hashes, provenance, schema, referential integrity, and a representative render, not duplicate semantic prose checks.

Workflow and implementation text

packaged-product.test.ts snapshots validator command order and implementation strings. root-suite.test.ts asserts workflow/script text. Replace these with workflow-schema validation, one Turbo dry-run schedule assertion, and observable validator results.

CI topology target

Pull requests

Target two required jobs, with a third only when browser behavior is affected:

1. ci: format/lint/typecheck, affected workspace tests, batched CLI unit tests, and one classified repository-contract path.
2. package-smoke: build/pack/install exact CLI artifact; prove hi, hint, help, version, check --json, and artifact contents.
3. browser-smoke: path-gated to backoffice/browser changes.

Delete the explicit test:ci preamble because the classified repository path already owns that work. Remove the PR release guard because its refusal cases already sit in ordinary deterministic coverage. Add path/diff routing so wiki, docs, backoffice, and non-CLI changes do not launch every CLI update/native lane. Batch pure deterministic files in one Vitest process; isolate only files with proven process-state or RPC hazards. Run one representative update contract on a CLI-affecting PR. Move exhaustive native/fresh permutations to a scheduled diagnostic lane only while they retain demonstrated value.

GitHub supports path filters, job conditions, matrices, and concurrency cancellation in workflow syntax. TanStack Query's current PR workflow cancels superseded work and runs affected checks.

Main and release

Do not launch a second full protected suite on every main push. Branch protection should require the reviewed PR authority before merge; GitHub supports required status checks and merge queues through protected branches.

Publication should consume the reviewed version and perform only release-specific checks:

1. Checkout the exact tag or release commit.
2. Use Node 24 and npm 11.5.1 or newer.
3. Install with the frozen lockfile and no release cache.
4. Build once.
5. Run npm pack --dry-run or the existing package-surface assertion.
6. Assert tag equals package version.
7. Publish with contents: read and id-token: write inside Production.
8. Read back npm and the GitHub release.

npm requires npm 11.5.1+, Node 22.14+, a GitHub-hosted runner, an exact trusted-publisher repository/workflow/environment match, and id-token: write; it automatically emits provenance for public packages from public repositories. Sources: npm trusted publishing and troubleshooting. GitHub environments can require reviewers and restrict deployment branches or tags: environment rules.

TanStack Query and Router are the useful cost models. Their release jobs build and run Changesets publication without replaying PR tests: Query release and Router release. TanStack Table does run test:ci during release, so TanStack itself demonstrates both patterns; Query and Router better fit the stated usage constraint.

Changesets is optional for one publishable package. A reviewed package.json and changelog bump plus a protected v* tag is simpler. If automated version pull requests are desired, Changesets provides that workflow without requiring Harness's reviewed-first-parent candidate queue.

Practitioner evidence from X

The verified X posts reinforce the direction but do not replace repository evidence:

• Bun maintainer Jarred Sumner describes timing-aware shard balancing for genuinely large suites. The measured example reduced the slowest of four shards by 27%, which supports measuring before adding shards: post.
• Nikki Siapno recommends identifying bottlenecks, caching artifacts, and running change-related tests when possible, while warning against overcomplicating selection: post.
• Milan Milanović argues that tests should follow behavior rather than implementation so internal refactors do not break them: post.
• MSW maintainer Artem Zakharchenko recommends flat, granular, self-contained tests and minimal hidden setup: post.

No strong X evidence justified static source-string assertions, test-name inventories, or replaying a full suite during publication.

Conflicts and uncertainty

• npm's example GitHub workflow includes npm test, while its stated trusted-publishing requirements concern runtime, runner, identity matching, and OIDC permissions. TanStack Query and Router omit release-time tests. The example is a safe template, not a requirement; Harness can rely on protected PR checks and keep publication release-specific.
• Test file and line counts are exact for 5990a041; per-job savings are not. The handoff's historical 1,748.8 Linux minutes and 527 non-skipped jobs support the cost direction, but a post-trim Actions run must measure the actual reduction.
• Deleting the Effect migration-closeout suite requires one human confirmation that IP-317 is closed and no longer needs a temporary execution gate.
• Simplifying the reviewed-first-parent release authority changes the security model. The current OIDC environment fix can be validated without making that larger decision.

Recommended delivery order

1. Delete meta-tests, title inventories, the version fixture, the orphan JSON fixture, and source-spelling tests.
2. Remove the duplicate test:ci preamble and PR release guard; batch deterministic unit tests.
3. Collapse packaged CLI and update matrices around observable equivalence classes while preserving native filesystem safety boundaries.
4. Add affected/path routing and reduce PR update verification to one representative contract.
5. Separate a minimal protected npm publish workflow from PR CI, or simplify the existing production job to the same contract.
6. After the GitHub authorization hold is resolved and the user approves one paid attempt, validate the already-merged GITHUB_ACTIONS propagation before redesigning provider authority.

Completion criteria for the delivery plan should be: at least 40% fewer CLI test lines, no test-name inventory, no version-churn fixture, no source-spelling assertions outside a single lint/AST boundary mechanism, at most three active PR jobs, one publish job, preserved native filesystem safety witnesses, and successful installed-tarball hi/hint proof.

Unresolved product decisions

1. Keep the current reviewed-first-parent release authority after the OIDC fix is validated, or replace it with protected PR checks plus a protected release tag.
2. Keep exhaustive native/update permutations as a scheduled diagnostic lane, or delete them once the retained equivalence classes are proven.
3. Adopt Changesets for release pull requests, or retain a direct reviewed version/changelog bump for the single npm package.

No workflow run, publish, billing action, provider setting, release tag, or implementation change was performed during this research.