Spec: Project-Specific Executable Verification Capability

Context

Harness already uses verify-behavior inside implement-spec to prove visibly exercisable acceptance criteria. The portable protocol maps accepted behavior to real user paths and retains evidence, but it does not know how a specific app starts, becomes ready, receives user actions, exposes observable results, isolates a run, or cleans up.

Add a Project Verifier: project-owned executable knowledge consumed through the existing verify-behavior entrypoint. A monorepo owns one Surface Verification Reference per independently runnable app. Each reference co-locates Launch, Doctor, Drive, Evidence, Cleanup, a Feature Map, and optional helpers. The result lets implement-spec reuse real app mechanics instead of rediscovering them for each change.

The primary actors are:

• an implementation orchestrator running implement-spec, filling verification coverage gaps, and proving an accepted scenario;
• contributors and maintainers reading or extending the Project Verifier.

The required outcome is that every app needed by an in-scope verification scenario has a proved, safe, project-owned reference and every selected behavior has a proved executable path before implement-spec classifies the affected acceptance criteria.

Non-Goals

• Replace review-phase or merge Verification with Code Review.
• Turn autoreview into a behavioral-proof system.
• Create a separate verify-<app> skill or a second verification entrypoint.
• Store app-specific drivers in the shared Harness CLI or shared verify-behavior procedure.
• Generate Project Verifier content during hi scaffold, hi update, or their post-command handoffs.
• Regenerate an existing Feature Map from scratch during scaffold, update, creation, or routine maintenance.
• Write or regenerate Feature Maps from docs-ingest-phase.
• Run a full Project Verifier audit after every documentation change.
• Let update-verification-skill repair product code or change expected behavior to hide a product failure.
• Add a review-phase freshness gate, verifier hash, or cross-phase snapshot-binding protocol.
• Define detailed implementation tasks, concrete test files, or command choreography; create-plan owns those choices.

User Stories

US-001: Reuse app-specific verification knowledge

As an implementation orchestrator, I want an accepted scenario to use one stable verification entrypoint and the matching app reference so that behavior proof does not rediscover app mechanics.

US-002: Create missing app coverage on demand

As an implementation orchestrator, I want an Uncovered Surface to create and prove its missing app reference before verification continues so that acceptance never relies on invented instructions.

US-003: Navigate executable product behavior

As a contributor, I want each app's Feature Map to connect documented behavior to real user entry points, driver actions, observable results, and gotchas so that I can verify the correct path.

US-004: Verify cross-app journeys

As an implementation orchestrator, I want one journey to compose the references for every participating app without duplicating app mechanics so that cross-surface behavior remains maintainable.

US-005: Fill missing behavior coverage on demand

As an implementation orchestrator, I want an Uncovered Behavior to add and prove only the executable path required by my selected scenario so that verification can resume without broad regeneration.

US-006: Protect the user's environment and evidence

As a project operator, I want verification to drive only run-owned state, protect credentials and sensitive data, and preserve evidence through cleanup so that proof is safe and auditable.

US-007: Preserve the Code Review boundary

As a reviewer, I want review-phase to remain readonly Code Review while implement-spec owns Verification and its reruns so that the two gates retain distinct authority.

US-008: Preserve project-owned verifier content

As a maintainer, I want scaffold and update follow-through to leave Project Verifier content unchanged so that managed-skill refreshes cannot erase or regenerate project knowledge.

Acceptance Criteria

• AC-001: When implement-spec reaches a visibly exercisable acceptance criterion, it selects the in-scope scenario and app or apps and invokes the single verify-behavior entrypoint.
  • Covers: US-001
• AC-002: When every selected app has a Surface Verification Reference, verify-behavior loads the project index, only the selected app references, only the applicable Feature Map entries, and the applicable Cross-App Journey.
  • Covers: US-001, US-004
• AC-003: App-specific verification knowledge exists only below .agents/skills/verify-behavior/references/; no app-specific SKILL.md or .cursor/skills/verify-<app>/ capability is created.
  • Covers: US-001, US-008
• AC-004: When a selected app has no Surface Verification Reference, implement-spec invokes create-verification-skill before attempting to classify the affected acceptance criteria.
  • Covers: US-002
• AC-005: create-verification-skill interviews the selected app's current repository evidence and routed wiki specifications, behavior pages, flows, and relevant domain knowledge before writing its reference.
  • Covers: US-002, US-003
• AC-006: A newly created app reference contains complete Launch, Doctor, Drive, Evidence, Cleanup, Feature Map, and applicable helper guidance with no unresolved placeholders.
  • Covers: US-002, US-003
• AC-007: A newly created app reference becomes ready only after one Reference Smoke Proof launches the app, passes Doctor, drives one mapped user action, retains evidence, cleans run-owned state, and confirms that the evidence remains.
  • Covers: US-002, US-006
• AC-008: When creation or its Reference Smoke Proof cannot complete, the original scenario and every affected acceptance criterion remain blocked with the exact blocker.
  • Covers: US-002
• AC-009: Verification does not silently replace a missing app reference or drive path with improvised automation and report the scenario verified.
  • Covers: US-001, US-002
• AC-010: Every Feature Map entry states the user-facing behavior, how the user reaches it, how the selected driver exercises it, which observable end state proves it, and applicable variants, side effects, prerequisites, and gotchas.
  • Covers: US-003
• AC-011: Feature Map entries link to applicable product or domain authority for meaning and requirements rather than duplicating that authority.
  • Covers: US-003
• AC-012: Initial creation preserves existing app references and Feature Map entries and adds only the missing app coverage required by the selected scenario.
  • Covers: US-002, US-003, US-008
• AC-013: The Project Verifier index identifies every known app reference and Cross-App Journey without repeating app-specific launch or driver instructions.
  • Covers: US-003, US-004
• AC-014: A Cross-App Journey composes two or more app references through verify-behavior and retains one observable scenario result without copying the referenced app mechanics.
  • Covers: US-004
• AC-015: docs-ingest-phase never creates, edits, regenerates, or maintains Feature Maps.
  • Covers: US-005
• AC-016: When the selected app reference exists but the required behavior has no Feature Map entry or executable drive path, implement-spec invokes update-verification-skill before classifying the affected acceptance criteria.
  • Covers: US-005
• AC-017: update-verification-skill reads applicable wiki specifications, behavior pages, flows, domain knowledge, current code, and runtime entry points, then adds or repairs only the executable knowledge required by the selected scenario.
  • Covers: US-005
• AC-018: A ready updater result proves the added drive path and returns control to the original verification scenario; a blocked result keeps that scenario and its affected acceptance criteria blocked with the exact blocker.
  • Covers: US-005
• AC-019: update-verification-skill returns exactly one outcome: unchanged, updated, blocked, or product-failure.
  • Covers: US-005
• AC-020: During delivery, updater changes remain on the current branch and pull request and are limited to project-owned content below .agents/skills/verify-behavior/references/.
  • Covers: US-005, US-008
• AC-021: When the verifier is current and the observed product behavior is wrong, the updater returns product-failure and routes the symptom to debugging-phase without changing the documented expected result.
  • Covers: US-005, US-007
• AC-022: Every Drive operation targets a run-owned instance and run-owned scratch state and follows a successful Doctor check.
  • Covers: US-006
• AC-023: After surprising or failed behavior, the verifier repeats Doctor or restores a known run-owned state before another Drive operation.
  • Covers: US-006
• AC-024: Prompts and retained evidence omit credentials, secrets, personal data, and sensitive state; required user authentication or approval remains a user action.
  • Covers: US-006
• AC-025: An inaccessible required state returns blocked with the exact missing credential, entitlement, operating-system capability, or external prerequisite.
  • Covers: US-006
• AC-026: Cleanup removes only resources owned by the verification run and leaves retained proof available at its reported location.
  • Covers: US-006
• AC-027: implement-spec owns behavior-verification evidence, acceptance classification, and affected reruns; review-phase does not execute the Project Verifier or become the freshness authority for its evidence.
  • Covers: US-001, US-007
• AC-028: review-phase remains readonly review of a frozen change with one comprehensive primary reviewer covering specification compliance, standards, security, architecture, simplicity, and skill obligations; autoreview is primary, with one mandatory independent risk-focused challenger for a bounded independent risk area. Additional challengers may be assigned when larger changes require separate independent risk areas. The parent adjudicates findings and owns the report; incomplete coverage is not clean.
  • Covers: US-007
• AC-029: hi scaffold, hi update, their post-command flows, and their generated handoffs do not invoke either verification lifecycle skill and leave existing Project Verifier content unchanged.
  • Covers: US-008
• AC-030: hi-cli scaffold and update guidance states the preservation boundary through one authoritative referenced rule, and generated handoffs render the same current-run action without duplicating a second policy source.
  • Covers: US-008
• AC-031: The adapted creator and updater remain faithful to the upstream repository interview, live proof, cleanup, Feature Map, source reconciliation, and product-regression triage procedures except where the accepted Harness topology, invocation, lifecycle, terminology, or outcome contract requires a change.
  • Covers: US-002, US-005
• AC-032: create-verification-skill and update-verification-skill are model-invoked skills with descriptions that state their distinct Uncovered Surface and Uncovered Behavior triggers.
  • Covers: US-002, US-005
• AC-033: verify-behavior uses explicit context pointers to load the index for a verification run and progressively disclose only the selected app, feature, journey, and helper guidance.
  • Covers: US-001, US-003, US-004
• AC-034: A targeted Uncovered Behavior update preserves unrelated app references, Feature Map entries, journeys, and helpers.
  • Covers: US-005, US-008
• AC-035: An explicit full-audit invocation may cover the complete Project Verifier index but is never a docs-ingest-phase action.
  • Covers: US-005
• AC-036: Each selected verification scenario states an observable result that would falsify the scenario's expected behavior, alongside the positive result that proves it.
  • Covers: US-003, US-006
• AC-037: For important behavior, the selected scenario includes a relevant failure or negative condition and verifies the actual downstream result when that condition is applicable; this does not require exhaustive negative testing for every scenario.
  • Covers: US-003, US-006
• AC-038: implement-spec may reuse creator or updater proof when it already satisfies the original scenario under matching authority, code, runtime, and scenario conditions; otherwise it runs the missing proof before acceptance classification.
  • Covers: US-001, US-002, US-005
• AC-039: A targeted update-verification-skill run may retire an obsolete Feature Map entry only with current source evidence and reconciled references; it preserves unrelated entries and documented product-expected semantics, and never performs broad regeneration.
  • Covers: US-005, US-008

Constraints

• Preserve the canonical terms in the Project Verification Capability Glossary.
• Keep Verification, Code Review, and delivery repair as separate authorities.
• Preserve one shared verify-behavior entrypoint.
• Keep app mechanics project-owned and progressively disclosed by app and feature.
• Use the detailed review topology defined by the Delivery Phase Flow Optimization specification without duplicating that contract here.
• Treat routed wiki specifications and behavior knowledge as initial map inputs, not executable proof by themselves.
• Preserve existing Feature Maps during creation and hi scaffold or hi update follow-through.
• State an observable falsifier for every selected scenario and use relevant negative or failure coverage for important behavior without imposing exhaustive negative testing universally.
• Reuse creator or updater proof only when authority, code, runtime, and scenario conditions match; otherwise obtain the missing proof.
• Retire obsolete Feature Map entries only through a targeted updater with current source evidence and reconciled references; preserve unrelated entries and expected product semantics.
• Keep Uncovered Surface creation and Uncovered Behavior repair inside their respective implement-spec branches.
• Treat Feature Maps as read-only inputs during docs-ingest-phase.
• Use stable driver handles such as accessible labels, route paths, prompt strings, commands, or native harness operations instead of coordinates or fragile navigation sequences.
• Prefer an existing Playwright, browser, PTY, HTTP, or native harness before introducing an app-owned helper.
• Never kill a process by name. Cleanup targets only the instance and state created by the run.
• Keep retained evidence outside cleanup targets.
• Do not expose secrets or sensitive state through prompts, screenshots, recordings, logs, or reports.
• Preserve upstream pstack skill content as closely as possible while applying the accepted Harness-specific deviations.

Dependency Readiness

No Stack Required.

Branch/Base Intent

Not applicable.

Accepted Technical Decisions

Capability topology

.agents/skills/verify-behavior/
├── SKILL.md
└── references/
    ├── README.md
    ├── <app>/
    │   ├── README.md
    │   ├── features/
    │   │   ├── README.md
    │   │   └── <feature>.md
    │   └── scripts/
    └── journeys/
        └── <cross-app-journey>.md

• SKILL.md is the Harness-managed portable entrypoint.
• The project owns references/README.md, app references, Feature Maps, journeys, and helpers.
• references/README.md is the app and journey index.
• Each app README.md co-locates Launch, Doctor, Drive, Evidence, Cleanup, and helper invocation criteria.
• Feature files contain only behavior-specific paths, proof states, variants, and gotchas.
• Journey files compose app references by pointer and do not copy their mechanics.

Dependency direction

• implement-spec selects the accepted scenario and app scope and invokes verify-behavior.
• verify-behavior loads the Project Verifier references selected by that bounded scope.
• An Uncovered Surface returns to implement-spec, which invokes create-verification-skill, waits for the Reference Smoke Proof outcome, and resumes the original scenario when ready.
• An Uncovered Behavior returns to implement-spec, which invokes update-verification-skill, waits for proof of the added drive path, and resumes the original scenario when ready.
• docs-ingest-phase updates ordinary wiki knowledge but never invokes update-verification-skill or writes Feature Maps.
• Shared Harness skills never import app-specific drivers into the CLI or shared protocol.

Agent-document structure

• The index, app, feature, journey, and helper layers use trigger-specific context pointers.
• Definitions, rules, caveats, and completion criteria for one app operation remain co-located.
• Environment-owned facts stay in executable helpers or repository configuration; references point to them instead of caching their implementation.
• The creator and updater are model-invoked because implement-spec must reach each skill for its distinct coverage-gap branch.
• hi-cli remains a small command router. Its scaffold/update post-command reference owns the preservation rule, and generated handoffs render that rule for the current run.
• Verification scenarios carry both a positive proof result and an observable falsifier. Important behavior may require one relevant failure or negative condition and its actual downstream result when applicable; no universal exhaustive negative suite is implied.
• Creator and updater proof is reusable only when the authority, code, runtime, and scenario conditions match the requesting verification scenario. A mismatch requires the missing proof.
• A targeted updater may retire obsolete Feature Map entries when current source evidence supports obsolescence and references are reconciled. It preserves unrelated entries and documented product-expected semantics and does not broadly regenerate the map.

Upstream adaptation

• Use pstack's create-verification-skill and maintain-verification-skill as the content baseline.
• Rename the adapted maintenance skill to update-verification-skill.
• Change generated paths from .cursor/skills/verify-<app>/ to project-owned content below .agents/skills/verify-behavior/references/<app>/.
• Change lifecycle triggers to on-demand Uncovered Surface creation and Uncovered Behavior repair inside implement-spec.
• Change maintenance shipping behavior from a mandatory separate pull request to the current delivery branch and pull request.
• Retain all compatible upstream repository interview, source reconciliation, live driving, evidence, cleanup, and product-gap behavior.

Accepted Testing Decisions

• Test one Reference Smoke Proof for every newly created app reference: Launch, Doctor, one mapped Drive operation, evidence retention, Cleanup, and post-cleanup evidence existence.
• Test that a missing app reference causes on-demand creation and resumes the original implement-spec scenario only after a ready result.
• Test that a blocked creator result keeps the scenario and affected acceptance criteria blocked.
• Test that scaffold/update compilation and application preserve project-owned verifier references byte-for-byte and never invoke creation or maintenance.
• Test that an Uncovered Behavior causes implement-spec to invoke the updater, prove the added drive path, and resume the original scenario only after a ready result.
• Test that a blocked updater result keeps the scenario and affected acceptance criteria blocked.
• Test that docs-ingest-phase never writes Feature Maps, including when its source material describes changed behavior.
• Test that targeted updates leave unrelated app references, Feature Map entries, journeys, and helpers unchanged, while explicit full-audit mode remains separately invocable.
• Test cross-app journey selection without duplicated app mechanics.
• Test run-owned isolation, Doctor-before-Drive, exact inaccessible-state blockers, secret-safe evidence, bounded Cleanup, and evidence survival.
• Test that verifier drift changes only Project Verifier references and product failure does not rewrite expected behavior.
• Test that selected scenarios record an observable falsifier and that important behavior covers a relevant failure or negative condition with the actual downstream result when applicable, without requiring exhaustive negative coverage for every scenario.
• Test that creator or updater proof is reused only under matching authority, code, runtime, and scenario conditions, and that mismatches trigger the missing proof.
• Test that a targeted updater can retire an obsolete Feature Map entry only with current source evidence and reconciled references, while preserving unrelated entries and documented product-expected semantics.
• Test context-pointer routing so a run loads only the selected app, features, journey, and helpers.
• Test that review-phase retains its readonly Code Review contract without invoking verifier creation, maintenance, or execution.

Verification Seams

• implement-spec scenario selection and its call to the single verify-behavior entrypoint prove bounded invocation.
• The Project Verifier index and selected app reference prove progressive disclosure and app-specific mechanics.
• A Reference Smoke Proof result proves that newly created instructions execute against one real mapped behavior.
• The Uncovered Behavior branch and updater result prove targeted Feature Map repair, drive-path proof, resume, or exact blockage.
• Behavior-verification evidence retained by implement-spec proves the selected acceptance path and side effects.
• A docs-ingest-phase before/after comparison proves Feature Maps remain unchanged during documentation ingest.
• A before/after tree comparison across hi scaffold and hi update proves Project Verifier preservation.
• Run-owned process, state, evidence, and cleanup observations prove isolation and evidence survival.
• review-phase routing and report behavior prove that Code Review remains separate from Verification.