Project Verification Capability Grill Log

Sources

In-thread proposal and accepted code-review versus verification boundary.
.agents/skills/verify-behavior/SKILL.md
.agents/skills/implement-spec/SKILL.md
.agents/skills/review-phase/SKILL.md
pstack create-verification-skill
pstack maintain-verification-skill

Branch: Verification And Review Boundary

Q1

Prerequisites:

none

Evidence anchors:

.agents/skills/verify-behavior/SKILL.md:11
.agents/skills/review-phase/SKILL.md:11
.agents/skills/autoreview/SKILL.md:18

Observed constraint:

verify-behavior owns behavioral proof, while review-phase performs readonly review of a frozen target and uses autoreview only as advisory code-review input.

Question: Should project verification replace review-phase or autoreview?

Code consequence:

This decision fixes whether verification becomes a new review implementation or remains a separate delivery gate.

Accepted answer:

No. Verification proves observable product behavior through real user paths.
Code review inspects a frozen change for correctness, standards, security, architecture, simplicity, skill adherence, and Spec compliance.
review-phase remains the code-review gate, and autoreview remains one advisory code-review input.

Branch: Implementation Integration

Q2

Prerequisites:

Evidence anchors:

.agents/skills/implement-spec/SKILL.md:18
.agents/skills/implement-spec/SKILL.md:72
.agents/skills/implement-spec/references/runtime-product-validation.md:7
.agents/skills/implement-spec/assets/IMPLEMENTATION-NOTES-TEMPLATE.md:53

Observed constraint:

implement-spec already calls verify-behavior after task and runtime checks and before acceptance classification, but no project-local launch, doctor, driver, cleanup, or Feature Map contract exists.

Question: Should implement-spec consume a durable project-specific verification capability for end-to-end proof?

Code consequence:

The existing behavior-proof stage gains a reusable project adapter instead of rebuilding project-specific drive instructions during every implementation.

Accepted answer:

Yes. implement-spec consumes a durable project-specific verification capability when acceptance criteria are visibly exercisable.
Existing implement-spec end-to-end checks become consumers of that capability rather than a competing verification system.

Branch: Terminology And Topology

Q3

Prerequisites:

Question: What is the canonical name of the generated project-local executable capability?

Accepted answer:

Canonical term: Project Verifier.
A Project Verifier is the project-owned collection of surface-specific verification references consumed through the single verify-behavior entrypoint.

Q4

Prerequisites:

Evidence anchors:

scripts/validate-packaged-product.mjs:7
apps/api/scripts/validate-runtime-product.mjs:165

Observed constraint:

Independently runnable surfaces have different launch, readiness, driving, isolation, and evidence mechanics. A monorepo can contain several such surfaces.

Question: Is the Project Verifier unit one repository or one independently runnable product surface?

Code consequence:

This decision determines the cardinality and composition of project verification references.

Accepted answer:

Keep one surface-specific verification reference per independently runnable surface.
In a monorepo, the default unit is one reference per app.
The single shared verify-behavior entrypoint can compose more than one surface reference for a cross-surface journey.

Branch: Source And Ownership

Q5

Prerequisites:

Evidence anchors:

.agents/skills/verify-behavior/SKILL.md
apps/cli/src/scaffold/output.ts:2561
packages/scaffold/src/baseline/managed-file.ts:8

Observed constraint:

verify-behavior is already the shared portable entrypoint. Separate verify-<surface> skills would create competing invocation surfaces.

Question: Where is the canonical project-owned verifier stored and how does scaffold reconciliation treat it?

Code consequence:

The shared entrypoint and project-authored surface knowledge need a split ownership boundary within one skill directory.

Accepted answer:

Keep one clean entrypoint at .agents/skills/verify-behavior/SKILL.md.
Store each surface under .agents/skills/verify-behavior/references/<surface>/.
Surface references are project-owned content and must survive ordinary scaffold reconciliation even though the shared verify-behavior skill is Harness-managed.

Branch: Invocation And Lifecycle

Q6

Prerequisites:

Evidence anchors:

.agents/skills/implement-spec/SKILL.md:72
.agents/skills/verify-behavior/SKILL.md:23

Observed constraint:

implement-spec knows the accepted scope and runnable surfaces. verify-behavior owns scenario mapping and interactive proof.

Question: Which layer resolves and invokes the Project Verifier?

Code consequence:

This decision fixes dependency direction and prevents the portable protocol from searching arbitrary project code without bounded scope.

Accepted answer:

implement-spec selects the applicable surface or surfaces and invokes verify-behavior with that scope.
verify-behavior remains the single entrypoint and loads the matching surface references.
Shared verification does not import product-specific drivers into Harness or the CLI.

Q7

Prerequisites:

Evidence anchors:

apps/cli/src/content/scaffold-copy.ts
/Users/stefan/.codex/skills/hi-cli/references/post-command-flow.md
.agents/skills/docs-ingest-phase/SKILL.md

Observed constraint:

hi scaffold emits a post-scaffold handoff and does not finish repository-specific authoring. docs-ingest-phase already owns durable knowledge refresh after proven changes.

Question: When is a missing Project Verifier created, and where does later upkeep run?

Code consequence:

Creation remains a proved repository-interview task instead of static scaffold generation, while maintenance joins the existing post-change knowledge lifecycle.

Accepted answer:

hi scaffold does not generate a Project Verifier directly.
The post-scaffold handoff prompt and hi-cli skill guidance instruct the continuing agent to invoke create-verification-skill.
Creation is complete only after the generated guidance successfully drives and proves one real mapped feature.
Canonical maintenance skill: update-verification-skill.
docs-ingest-phase invokes update-verification-skill after applicable proven changes rather than leaving upkeep as an unrelated periodic step.

Branch: Feature Map

Q8

Prerequisites:

Question: What knowledge is the Feature Map authoritative for?

Accepted answer:

The Feature Map is authoritative for executable knowledge: user entry points, driver recipes, expected observable states, variants, side effects, prerequisites, and gotchas.
Product specifications and domain documentation remain authoritative for product meaning and requirements.
Feature Map entries link to the applicable product or domain authority instead of duplicating it.

Branch: Feature Map Update Lifecycle

Q9

Prerequisites:

Question: Does normal docs ingest update only affected surfaces and features, or run a full Project Verifier audit?

Accepted answer:

Normal docs-ingest-phase updates and re-verifies only the affected Surface Verification References and Feature Map entries.
A full all-surface audit remains an explicit mode for periodic checks or suspected broad drift.
A normal docs-only change with no verification impact does not launch or drive the product.

Q10

Prerequisites:

Evidence anchors:

.agents/skills/verify-behavior/SKILL.md:23
.agents/skills/implement-spec/references/runtime-product-validation.md:7
upstream create-verification-skill sections 2-4

Observed constraint:

The shared entrypoint needs one predictable contract, while each surface can require a different driver.

Question: What contract must each Surface Verification Reference provide?

Code consequence:

Surface references remain composable without forcing browser, Playwright, PTY, HTTP, or native surfaces into one implementation.

Accepted answer:

Each Surface Verification Reference provides Launch, Doctor, Drive, Evidence, Cleanup, Feature Map, and any surface-owned helpers.
The contract is uniform; its driver remains surface-specific.

Q11

Prerequisites:

Evidence anchors:

.agents/skills/docs-ingest-phase/SKILL.md
.agents/skills/docs-ingest-phase/phases/router.md

Observed constraint:

docs-ingest-phase already inspects proven changes and routes only the affected knowledge work.

Question: Which changes require update-verification-skill?

Code consequence:

Verification maintenance becomes an owned docs-ingest branch with an explicit no-op path.

Accepted answer:

The trigger is implemented inside docs-ingest-phase.
docs-ingest-phase invokes update-verification-skill when a proven change affects user behavior, entry points, launch or readiness, authentication or setup, driving mechanics, observable states or side effects, variants, prerequisites, or gotchas.
Internal refactors and prose-only documentation changes are verification-maintenance no-ops unless they expose actual drift.

Branch: Cross-Surface Composition

Q12

Prerequisites:

Question: How are cross-surface journeys indexed and driven?

Accepted answer:

Maintain one reference index that names every Surface Verification Reference and every cross-surface journey.
A cross-surface journey composes the applicable surface references through verify-behavior without duplicating their launch or driver instructions.

Branch: Evidence Ownership

Q13

Prerequisites:

Question: Should review-phase accept verification evidence only when it is bound to the exact frozen code snapshot being reviewed?

Accepted answer:

No new verification-freshness gate belongs in review-phase.
implement-spec owns verification, its evidence, and any affected rerun before acceptance classification.
review-phase remains separate code review. It may read existing verification evidence as context, but it does not validate, rerun, or become the authority for that evidence.
Do not add a separate verifier revision or cross-phase snapshot-binding contract for a low-likelihood race between verification and review.

Branch: Coverage Gaps And Failure Routing

Q14

Prerequisites:

Question: What happens when a required Surface Verification Reference is missing or unusable?

Accepted answer:

An Uncovered Surface is an independently runnable app in scope with no .agents/skills/verify-behavior/references/<app>/ reference.
Uncovered Behavior means the app reference exists but the required behavior has no Feature Map entry or drive path.
A new app or wholly missing app reference invokes create-verification-skill before acceptance.
Uncovered behavior or verifier drift enters the verification-maintenance branch inside docs-ingest-phase; that branch invokes update-verification-skill, then returns control to verification.
A working verifier that exposes incorrect product behavior routes to debugging-phase.
Verification does not silently substitute improvised automation and report completion.

Adapted upstream skill structure

The pstack creator and maintainer are source material. Harness changes their generated topology and lifecycle while retaining their repository interview, live proof, cleanup, Feature Map, and regression-triage rigor.

.agents/skills/verify-behavior/
├── SKILL.md                         # single Harness-managed entrypoint
└── references/
    ├── README.md                    # project-owned app and journey index
    ├── <app>/
    │   ├── README.md                # Launch, Doctor, Drive, Evidence, Cleanup
    │   ├── features/
    │   │   ├── README.md
    │   │   └── <feature>.md
    │   └── scripts/                 # optional app-owned helpers
    └── journeys/
        └── <cross-app-journey>.md

create-verification-skill writes or completes one or more <app>/ references and the shared reference index. It never creates .cursor/skills/verify-<app>/ or another app-specific SKILL.md.
Harness renames the adapted maintenance capability to update-verification-skill.
update-verification-skill locates app references below verify-behavior/references/. In normal docs-ingest-phase use it updates only affected apps, features, journeys, and helpers; its explicit full-audit mode covers the complete index.
Both skills preserve project-owned reference content and use the single verify-behavior entrypoint for live proof.

Writing-for-agents validation

Evidence anchors:

.agents/skills/writing-for-agents/SKILL.md
.agents/skills/writing-for-agents/SKILL-MECHANICS.md

The directory structure is valid progressive disclosure when its context pointers encode the branches that load each level:

verify-behavior/SKILL.md loads references/README.md for every verification run, then loads only the selected <app>/README.md, mapped feature files, and cross-app journey file.
references/README.md is an index. It does not repeat app mechanics.
<app>/README.md co-locates Launch, Doctor, Drive, Evidence, Cleanup, and helper invocations.
Feature files contain feature-specific user paths, proof states, and gotchas. They do not repeat app launch mechanics.
Journey files compose app references by pointer. They do not copy their content.
Helper behavior lives in executable scripts; the app reference states when to invoke them and their completion criteria instead of duplicating their implementation.

create-verification-skill and update-verification-skill must be model-invoked skills because implement-spec and docs-ingest-phase must reach them. Their descriptions carry the exact creation and update trigger branches. The adapted skills therefore omit upstream disable-model-invocation: true. This invocation change, path change, lifecycle integration, and outcome adaptation are the necessary deviations from upstream; preserve the remaining upstream procedure and wording as closely as the Harness terms allow.

The top-level hi-cli skill remains a small command router. The detailed preservation rule lives once in its scaffold post-command reference, and the generated post-scaffold handoff renders the same rule as the current run's action:

inspect the Project Verifier index;
preserve every indexed app reference;
invoke create-verification-skill only for an independently runnable app missing from the index;
never regenerate an existing Feature Map during scaffold or update follow-through.

This proposed post-scaffold creation behavior is superseded by Q19. The current contract is narrower: hi-cli and generated handoffs preserve existing Project Verifier content and perform no creation or regeneration. implement-spec owns the Uncovered Surface trigger.

Branch: Initial Feature Map Coverage

Q16

Prerequisites:

Question: How much initial Feature Map coverage must each app reference contain?

Accepted answer:

Use the repository's routed wiki specifications, behaviors, flows, and applicable domain knowledge to seed initial Feature Map depth for the selected app.
Reconcile that documentation with current routes, commands, menus, tests, and runtime code before recording executable paths.
Preserve existing app references and maps. Post-scaffold and post-update handoffs add missing app references only; they never regenerate existing maps from scratch.
Later proven behavior changes enrich only affected entries through the docs-ingest-phase update branch.
Record evidence gaps explicitly instead of inventing undocumented or unproved coverage.

Branch: Runtime Safety

Q17

Prerequisites:

Question: Which isolation, credential, and inaccessible-state rules must every app reference enforce?

Accepted answer:

Drive only a run-owned instance and run-owned scratch state.
Run Doctor before Drive and again after surprising behavior before continuing.
Keep credentials, secrets, personal data, and sensitive state out of prompts and retained evidence.
Leave required user authentication and approval with the user.
Record an inaccessible required state as blocked with the exact prerequisite.
Cleanup removes only run-owned resources. Retained evidence survives cleanup.

Branch: Update Outcomes And Edit Boundary

Q18

Prerequisites:

Question: What outcomes and edit boundary must update-verification-skill enforce?

Accepted answer:

Outcomes are unchanged, updated, blocked, or product-failure.
During delivery, updates join the current branch and pull request rather than creating a separate maintenance pull request.
The skill edits only project-owned content below .agents/skills/verify-behavior/references/.
A product failure routes to debugging-phase; it is not hidden by changing the Feature Map or driver expectations.

Branch: On-Demand Creation

Q15

Prerequisites:

Question: Must a newly created app reference prove one real feature before it is ready?

Accepted answer:

Yes. Each newly created app reference must complete one Reference Smoke Proof: Launch, Doctor, one mapped user action, retained evidence, and Cleanup.
This proof runs only when that app reference is first created. It is not a full Feature Map audit or regeneration.
A failed or inaccessible smoke proof leaves the new reference and the requesting verification scenario blocked with the exact reason.

Q19 supersedes Q7 post-scaffold activation

Prerequisites:

Question: When does Harness invoke create-verification-skill?

Accepted answer:

hi scaffold, hi update, their post-command flow, and their generated handoffs do not invoke create-verification-skill.
Those flows preserve existing Project Verifier references and Feature Maps without creating or regenerating them.
During implement-spec, after a verification scenario and app are selected, an Uncovered Surface eagerly invokes create-verification-skill.
The creator writes the missing .agents/skills/verify-behavior/references/<app>/ content, completes its Reference Smoke Proof, and returns control to the original implement-spec verification scenario.
If creation or its smoke proof is blocked, the scenario and affected acceptance criteria remain blocked.
Q19 supersedes Q7 only for creation timing. Q7's docs-ingest-phase ownership of update-verification-skill remains accepted.

Branch: Uncovered Behavior Ownership

Q20 supersedes docs-ingest ownership and routing in Q7, Q9, Q11, Q14, Q16, and Q19

Prerequisites:

Question: Which workflow fills an Uncovered Behavior, and may docs-ingest-phase write Feature Maps?

Accepted answer:

implement-spec owns the Uncovered Behavior branch for its selected verification scenario.
When the app reference exists but the required behavior has no Feature Map entry or executable drive path, implement-spec invokes update-verification-skill before acceptance classification.
The updater reads applicable wiki specifications, behavior pages, flows, domain knowledge, current code, and runtime entry points. It adds or repairs only the executable knowledge required by the selected scenario, proves the resulting drive path, and returns control to that scenario when ready.
A blocked updater result keeps the scenario and affected acceptance criteria blocked with the exact reason.
docs-ingest-phase never invokes update-verification-skill and never creates, edits, regenerates, or maintains Feature Maps. Wiki knowledge remains input that the creator and updater may read.
An explicit full-audit invocation remains available outside docs-ingest-phase; targeted updates preserve unrelated app references, Feature Map entries, journeys, and helpers.
Q19 remains authoritative for Uncovered Surface creation. Its final sentence retaining Q7 updater ownership is superseded.

Closure

The user confirmed shared understanding on 2026-09-04, then corrected Feature Map update ownership through Q20.
implement-spec invokes update-verification-skill for Uncovered Behavior and resumes the selected verification scenario only after the added drive path is proved.
docs-ingest-phase never invokes the updater or writes Feature Maps.
Scaffold and update post-command flows leave Project Verifier content unchanged. They do not create, update, or regenerate Feature Maps.
Q19 remains the sole creation trigger: implement-spec invokes create-verification-skill for an Uncovered Surface discovered while preparing an in-scope verification scenario.

Follow-Up: Proof Quality And Targeted Map Maintenance (2026-09-05)

The Astra re-analysis challenged proof quality and maintenance precision while preserving the accepted topology, lifecycle, ownership, and review boundary. The user accepted these clarifications and declined additional experiments.

Accepted clarifications

Each selected verification scenario states an observable result that would falsify its expected behavior, alongside the positive result that proves it.
Important behavior includes a relevant failure or negative condition and verification of the actual downstream result when applicable. The condition is selected by relevance; this does not create a universal exhaustive negative-testing requirement.
Creator or updater proof may be reused when authority, code, runtime, and scenario conditions match the requesting scenario. Otherwise implement-spec obtains the missing proof. No additional review freshness gate is introduced.
A targeted update-verification-skill run may retire an obsolete Feature Map entry only when current source evidence supports obsolescence and references are reconciled. It preserves unrelated entries and documented product-expected semantics and never broadly regenerates the map.
These clarifications add no experiment or benchmark requirement. The existing admission policy remains unchanged.

Resulting specification obligations

Add criteria for Scenario Falsifiers, relevance-bounded negative or failure proof, conditional proof reuse, and evidence-backed targeted retirement.
Keep implement-spec as the verification freshness authority and docs-ingest-phase read-only for Feature Maps.
Preserve the parent-orchestrator-only delegation rule and run-owned isolation.

Accepted Follow-Up: Review Topology Alignment (2026-09-05)

review-phase uses one comprehensive primary reviewer with explicit coverage of all five mandatory Code Review axes. autoreview is the primary reviewer; no additional advisory scan is required.
One independent risk-focused challenger is mandatory and checks a bounded independent risk area. Additional challengers may be assigned when larger changes require separate independent risk areas. Challengers and the primary reviewer read the same frozen factual packet; each challenger completes its independent check before consuming primary conclusions.
The parent adjudicates findings and owns the final report. Missing per-axis coverage or incomplete challenger work is incomplete and cannot be clean.
Detailed review topology remains authoritative in the Delivery Phase Flow Optimization specification; this leaf records only the cross-authority boundary.

Project Verification Capability Grill Log

On this page