Grilling
Project Verification Capability Grill Status
Project Verification Capability Grill Status
Phase State
- Topic: Project Verification Capability.
- Scope boundary: requirements only; no implementation, backlog, or provider mutation.
- Shared understanding: confirmed.
- Starting direction: add a maintained project-specific executable layer beneath
verify-behavior; preservereview-phaseas code review.
Branch Dashboard
| Branch | Completion | Locked direction | Still-open items |
|---|---|---|---|
| Verification and review boundary | 100% | Verification and its evidence remain inside implement-spec; review-phase remains separate code review with one comprehensive primary reviewer (autoreview), explicit per-axis coverage, and bounded independent risk-focused challenge when useful. The parent adjudicates findings and owns the report; no verification-freshness gate is added. | Detailed topology is authoritative in the Delivery Phase Flow Optimization specification. |
| Project verifier topology | 100% | The Project Verifier uses a progressive-disclosure index, one app reference per runnable surface, feature references, optional helpers, and composed cross-app journeys. | None. |
| Source and ownership | 100% | One Harness-managed entrypoint loads preserved project-owned references. Context pointers select only the index, app, feature, and journey material needed for a run. | Exact scaffold representation belongs to specification. |
| Generation and invocation | 100% | implement-spec eagerly invokes creation only when its selected scenario has an Uncovered Surface; scaffold and update flows leave Project Verifier content unchanged. | None. |
| Feature Map | 100% | Initial maps use routed wiki specs and behaviors reconciled with code; selected scenarios state an observable falsifier; important behavior covers a relevant negative or failure condition when applicable; implement-spec invokes targeted repair when its selected scenario has Uncovered Behavior. docs-ingest-phase never writes Feature Maps. | None. |
| Runtime and evidence | 100% | Every newly created app reference passes one Reference Smoke Proof; app references drive run-owned state, protect secrets, block on inaccessible prerequisites, and preserve evidence through cleanup. | None. |
| Maintenance | 100% | The updater runs from implement-spec for Uncovered Behavior, returns unchanged, updated, blocked, or product-failure, edits only Project Verifier references, may retire obsolete entries only with current source evidence and reconciled references, preserves unrelated entries and expected semantics, joins current delivery, and routes product failures to debugging. | None. |
Technical Grounding
| Branch | Evidence anchors | Applicable dimensions | Open technical decisions | Grounding |
|---|---|---|---|---|
| Project verifier topology | .agents/skills/verify-behavior/SKILL.md:11; scripts/validate-packaged-product.mjs:7 | topology, index, and cross-surface composition fixed | none | grounded |
| Source and ownership | apps/cli/src/scaffold/output.ts:2561; packages/scaffold/src/baseline/managed-file.ts:8; .agents/skills/writing-for-agents/SKILL.md | canonical location, content contract, progressive disclosure, and preservation fixed; ownership representation deferred to spec | none | grounded |
| Generation and invocation | apps/cli/src/scaffold/run.ts:193; apps/cli/src/content/scaffold-copy.ts; .agents/skills/implement-spec/SKILL.md:72 | dependency direction, on-demand creation lifecycle, and missing-capability route fixed | none | grounded |
| Feature Map | routed wiki specs and behavior pages; .agents/skills/verify-behavior/SKILL.md:23; .agents/skills/implement-spec/SKILL.md:72 | authority, initial sources, conservative preservation, on-demand Uncovered Behavior repair, explicit full audit, and docs-ingest read-only boundary fixed | none | grounded |
| Runtime and evidence | .agents/skills/implement-spec/references/runtime-product-validation.md:7; apps/api/scripts/validate-runtime-product.mjs:165 | surface contract, smoke proof, evidence ownership, failure routing, isolation, credentials, inaccessible state, and cleanup fixed | none | grounded |
| Review integration | .agents/skills/review-phase/phases/run-review.md:20; .agents/skills/review-phase/references/durable-report.md:53 | code-review boundary fixed; no verification-freshness coupling | none | grounded |
Current Round
- Round: 2026-09-05 review-topology follow-up (umbrella R6)
- Prior round: R5 (closed)
- Current frontier: empty
- Shared-understanding confirmation: confirmed
| Question id | Prerequisites | Question | State |
|---|---|---|---|
| Q3 | Q1, Q2 | What is the canonical name of the generated project-local executable capability? | answered |
| Q4 | Q2 | Is the verifier unit one repository or one independently runnable product surface? | answered |
| Q5 | Q2 | Where is the canonical project-owned verifier stored and how does scaffold reconciliation treat it? | answered |
| Q6 | Q2 | Which layer resolves and invokes the project verifier? | answered |
| Q7 | Q2 | When is a missing project verifier created? | answered |
| Q8 | Q2 | What knowledge is the Feature Map authoritative for? | answered |
| Q9 | Q7, Q8 | Does normal docs ingest update only affected surfaces/features, or run a full verification audit? | answered |
| Q10 | Q4, Q5, Q6 | What exact contract must each surface reference provide? | answered |
| Q11 | Q7, Q8 | Which changes require update-verification-skill during docs ingest? | answered |
| Q12 | Q4, Q8 | How are cross-surface journeys indexed and driven? | answered |
| Q13 | Q1, Q6 | What identity and freshness data binds verification evidence to code review? | answered |
| Q14 | Q6, Q7 | What happens when a required surface reference is missing or unusable? | answered |
| Q15 | Q4, Q7, Q14 | Must a newly created app reference prove one real feature before it is ready? | answered |
| Q16 | Q8, Q9, Q14 | How much initial Feature Map coverage must each app reference contain? | answered |
| Q17 | Q10, Q14 | Which isolation, credential, and inaccessible-state rules must every app reference enforce? | answered |
| Q18 | Q7, Q9, Q14 | What outcomes and edit boundary must update-verification-skill enforce? | answered |
| Q19 | Q6, Q14, Q15 | When does Harness invoke create-verification-skill? | answered |
| Q20 | Q6, Q8, Q14, Q18, Q19 | Which workflow fills an Uncovered Behavior, and may docs ingest write Feature Maps? | answered |
Parked Branches
- None.
Glossary
Terms
- Verification: Exercising observable product behavior through a real user path and retaining evidence of the action, resulting state, and applicable side effects. Avoid: code review, static validation.
- Code review: Readonly inspection of a frozen change against Spec, standards, security, architecture, simplicity, and skill obligations. Avoid: verification.
- Project Verifier: The project-owned collection of surface-specific verification references consumed through the single
verify-behaviorentrypoint. Avoid: verification skill. - Surface Verification Reference: The Project Verifier content for one independently runnable surface, stored below
.agents/skills/verify-behavior/references/<surface>/. Avoid: separate verifier skill. - Update Verification Skill: The shared
update-verification-skillcapability invoked byimplement-specto add or repair executable coverage for an Uncovered Behavior in its selected scenario. Avoid: periodic maintenance as the only update path. - Uncovered Surface: An independently runnable app in scope with no Surface Verification Reference. Avoid: failed verification.
- Uncovered Behavior: In-scope behavior with no Feature Map entry or executable drive path in an existing app reference. Avoid: product regression.
- Reference Smoke Proof: A one-time end-to-end check that a newly created app reference can launch, pass Doctor, drive one mapped feature, retain evidence, and clean its run-owned state. Avoid: full Feature Map audit.
- Feature Map: A maintained index of executable user entry points, driver recipes, expected observable states, variants, and gotchas. Avoid: product specification, domain authority.
Relationships
implement-specuses Verification before acceptance classification when acceptance criteria are visibly exercisable.- A Project Verifier supplies project-specific mechanics to the portable
verify-behaviorprotocol. - A Project Verifier contains one Surface Verification Reference per independently runnable surface.
implement-specinvokes the Update Verification Skill when its selected scenario has Uncovered Behavior.- A Cross-Surface Journey composes two or more Surface Verification References through the single
verify-behaviorentrypoint. review-phaseperforms Code review and may audit retained verification evidence without creating that evidence.
Axioms
- Verification and code review are separate gates.
- Product-specific launch and driver mechanics do not belong in the shared portable
verify-behaviorprotocol. - Evidence must survive cleanup, while cleanup removes only resources owned by the verification run.
implement-specowns both verification coverage-gap branches: creation for an Uncovered Surface and targeted update for Uncovered Behavior.review-phasedoes not own verification evidence freshness or reruns.- The adapted creator and updater write project-owned content only below
.agents/skills/verify-behavior/references/; they do not create app-specific skills. docs-ingest-phasenever invokes the Update Verification Skill or writes Feature Maps.- Post-scaffold and post-update follow-through leaves Project Verifier content unchanged.
create-verification-skillandupdate-verification-skillare model-invoked becauseimplement-specmust reach them for the two distinct coverage-gap branches.implement-specinvokescreate-verification-skillonly when its selected scenario has an Uncovered Surface.
Flagged Ambiguities
- "Verification skill" previously meant the shared generator, generated capability, or portable protocol. Resolution: use Project Verifier for the generated capability and keep
verify-behavior,create-verification-skill, andupdate-verification-skillas exact skill IDs. - "Missing reference" means an Uncovered Surface, not any failed product check.
- "Prove one real feature" means a one-time Reference Smoke Proof for a newly created app reference, not a full app verification or map regeneration.
- Q19 supersedes Q7's post-scaffold creation timing. Scaffold and update follow-through only preserve Project Verifier content.
- Q20 supersedes Q7's updater ownership, Q9's docs-ingest maintenance route, Q11's docs-ingest trigger, Q14's Uncovered Behavior route, Q16's later-update route, and Q19's statement that Q7 updater ownership remained accepted. Q19 remains authoritative for Uncovered Surface creation.
Accepted Follow-Up — 2026-09-05
- Selected verification scenarios must state an observable Scenario Falsifier alongside the positive proof result.
- Important behavior includes a relevant failure or negative condition and the actual downstream result when applicable. This is relevance-bounded and does not require exhaustive negative testing for every scenario.
- Creator or updater proof may be reused when authority, code, runtime, and scenario conditions match the requesting scenario. A mismatch requires the missing proof;
implement-specremains the freshness owner. - A targeted
update-verification-skillrun may retire an obsolete Feature Map entry only with current source evidence and reconciled references. It preserves unrelated entries and documented product-expected semantics and never broadly regenerates the map. - These are specification clarifications and add no experiment or benchmark requirement; the current admission policy remains unchanged.
Final Handoff
- Shared understanding confirmed on 2026-09-04.
- Requirements are ready for
create-spec. - No branches are parked.
- Remaining non-design work: implement the adapted skills and lifecycle pointers, then prove Uncovered Surface creation, Uncovered Behavior repair, on-demand resume, docs-ingest immutability, preservation, isolation, cleanup, and failure routing with live fixtures.