Comprehensive Behavior Contract Validation

The comprehensive behavior contract is the single local-and-CI signal for public behavior across Harness Intelligence. It protects outcomes at public seams while leaving internal architecture free to change.

Trigger

Run this flow before accepting architecture work that can change CLI output, managed assets, API or browser behavior, persistence, auth, environment validation, scaffold schemas, wiki output, or shared package behavior.

Actors

Implementation owner
Findings-first reviewer
Turborepo and workspace test tasks
Required behavior-contract CI workflow

Execution Topology

bun run test:ci runs the root behavior/static contracts and repository check once. Its generic test wave excludes @punks/cli#test and @punks/api#test; the separate check, check-types, and build wave still runs CLI and API verification once. API tests remain covered exactly once through the @punks/api validate:runtime-product dependency. Trustworthy pull-request refs select affected tasks, while missing or unsafe scope fails closed to the full graph. API runtime validation remains a separate wave outside affected pruning.

GitHub Actions builds CLI plus its Turbo dependencies, then runs CLI Vitest in two isolated matrix shards. Each shard uses the package's serialized test script. For bounded two-worker local feedback, first run bunx turbo run build --filter=@punks/cli --concurrency=2, then bun run --cwd apps/cli test:parallel; the package test script remains serialized for release validation.

Contract Inventory

12 workspaces participate.
11 workspaces own testable public behavior.
packages/config is the sole reason_not_testable workspace and participates through locked static options plus an operating-system temporary TypeScript consumer proof.
20 public seams are classified: 6 byte-for-byte locked, 13 semantically equivalent, and 1 intentionally changed by an accepted requirement.
Eight adapter families share focused deterministic and real contracts: CLI filesystem, CLI process, GitHub provider HTTP, OpenRouter provider HTTP, Postgres persistence, Better Auth session, Resend email, and the backoffice API client.

Flow

Run the root behavior-contract command used by CI, then require both isolated CLI shard jobs from the same workflow. Do not replace either side with a reduced fast lane.
Confirm the repository validator reports all 12 workspace contracts, including the static and consumer proof for packages/config.
Confirm all 20 seams retain a fixture and public-seam test with the accepted classification totals.
Confirm every declared adapter family still runs the same focused contract against deterministic and real implementations.
Exercise supported concurrency, seeded ordering, repetition, and cleanup through the root behavior-contract tooling.
Inspect the repeat report for identical successful runs and valid cleanup. Keep timing evidence descriptive; it is not a threshold.
Run findings-first review, the full local closeout gate, and the required CI workflow before accepting or closing the epic.

Effect V4 Cutover Gates

Architecture cutovers add two root gates around the comprehensive behavior contract:

Run bun run validate:effect-v4 to reject stale Effect dependencies, boundary violations, invalid typed-failure/configuration patterns, and incomplete migration cleanup. Root bun run test executes this gate first.
Run the comprehensive repository tests and static checks to preserve the accepted public behavior classifications.
Run bun run validate:packaged-product to build and install the actual npm tarball, exercise hi and hint, prove packaged API provenance and fail-closed authority, run renderer/catalog/browser/wiki seams, and verify owned-resource cleanup.
Run the CLI lifecycle end-to-end suite (apps/cli/src/registry/lifecycle.e2e.test.ts), which drives hi init, hi update, hi diff, and hi check in temporary repositories against a locally built Registry (HI_REGISTRY_URL=file://...).

The former validate:consumer-repositories and validate:cutover-repeat gates exercised the manifest-driven lifecycle and were deleted with it in CLI 6.0.0.

The current checked-in fixture and the public-seam test that owns it are the authority for accepted behavior. When an accepted release changes that behavior, update the producer, its seam assertions, and the fixture atomically. Migration closeout keeps the durable IP-317 and Effect v4 semantic facts, but it does not freeze a historical fixture inventory or byte-hash snapshot permanently.

CLI filesystem validation failures must remain on the typed error channel through output materialization, project-settings writes, and final system-prompt writing. The public executable seam locks unsafe targets to one validation-failed result instead of an unpresented Effect defect.

Its direct wiki decomposition is ordered: next typegen, fumadocs-mdx, generated-source assertion, native TypeScript with --incremental false, then next build. The generated-source assertion keeps raw sorted path/mode/content digests in complete-run equality. The repository-owned Bun patch for fumadocs-mdx@14.3.2 sorts asynchronous glob results and serializes eager import mutation; remove it only after an upstream release passes repeated normal-threadpool raw-byte generation.

Current Proven Evidence

IP-319 T9 report .reports/behavior-contract/repeat-session-3b3dcb3c-6797-4dbf-9597-9becdc5784a9.json recorded two identical successful runs. Run 1 completed 14 of 14 tasks in 31.427 seconds; run 2 completed 14 of 14 tasks in 30.944 seconds. Internal cleanup was valid with no errors. Standalone cleanup was valid with no discovered, removed, remaining, unknown, or report entries. Final findings-first review returned PASS.

IP-324 T8 replacement evidence exited 0 at HEAD 5e1637f939818e43b69a8a5262c3de012e54518c, with dirty-patch digest 633b1614c0081cd52d7dda086cb241c2fd62f279c2b9d4fe26914e7695faf806. Two complete runs matched at 739c91c0ae08234d6cd7611ffbfda8eed19aa14b1f4355b349d12a546162fe0d; two stress runs matched at dbbed7e6af71a7a1bdf63b9444045743503343e5f86770a4f15cce1c78fd7bac. Every package/consumer lifecycle observed archive SHA-256 cc49bfd06a303ee9f6cf4c5a1a991404c1641de0567c3df77b7032f381d1361; the authoritative 664-entry installed tree matched at 991c96fbcf2e7a0d2b26733704e53944783523e61c47b3fa2319eba48f7f2cb9. Four generated wiki modules matched at fd36d0636ae488eaa5acb37823cf0f4bf6846e0f16452d387312a847be2fe422. Focused coverage passed 31/31 with 214 assertions, static checks passed 12/12, and every complete, stress, injected-failure, and session cleanup path/process/service set was empty.

Closeout Boundary

IP-319 is accepted on draft PR #76, which remains unmerged and undeployed. The last pushed-head behavior-contract CI at fabd4b23 was red from bounded clean-checkout assumptions plus unrelated or stale scaffold baseline/hash drift. Per maintainer direction, no further investigation or global/CI gate was run. The existing T1-T9, final review, and routed docs evidence was accepted, and T10 is complete under that scoped decision.

Outcome

For future architecture work, closeout requires the full local gate plus the CI root contract and both isolated CLI shards to succeed, with valid cleanup, a blocker-free final review, routed documentation, and durable tracker evidence. The scoped IP-319 acceptance above does not weaken that policy.

Comprehensive Behavior Contract Validation

On this page