Issue 197 Scaffold Lifecycle Architecture Research and Brainstorm

Scope and decision state

Assess the complete scaffold/update/check lifecycle: skill delivery, lint, Lefthook, scoped coding standards, native subagents, and Post-Command Handoff. Research base: b9bb213d, current upstream main when this branch was created. Branch: team/stefan/issue-197-scaffold-architecture.

The user accepted three foundations during R1:

1. CLI completes routine setup and verification; agent handles interpretation.
2. Selected baseline contributions provide supported quality-command defaults, with explicit repository overrides.
3. Check reports independent areas and explicit unknowns, returning nonzero while required state remains unresolved.

The user then added a fourth requirement: scaffolded context intended for editing must not produce false drift merely because it was edited. The recurrence is user-reported; the ownership paths below are verified statically, but a concrete false-positive instance has not been reproduced in this assessment.

Everything below that extends those foundations is a brainstorm candidate, not an accepted specification. See the decision log and current status.

Evidence worth retaining

Check cannot describe this repository's state

A fresh network-enabled installed CLI 4.0.4 hi check --json exited 1: Commit Gate Consumer apps/api has an incomplete Quality Command Contract. No managed-file or baseline drift was established by that run. An earlier sandboxed attempt reported unavailable baseline authority; it is separate from the reproduced local validation error.

apps/api/package.json uses format: "oxfmt --check ." and delegates to it from check. qualityCommandContractFor recognizes selected format-check script names or a direct oxfmt --check fragment in check, and returns no contract if either field is missing (apps/cli/src/integrations/repository-detector.ts:124-142). Repository detection inherits the root install topology across workspace manifests (:98-122, :526-551). Context compilation treats package-manager plus lockfile as sufficient to require the complete contract and throws (apps/cli/src/features/context-planning/compiler.ts:264-308).

runRepositoryCheckApplication calls update assessment before the remaining checks (apps/cli/src/features/repository-check/application.ts:476-513). The session hook treats a nonzero response without its recognized drift-detected issue as unavailable (apps/cli/src/data/hooks/scaffold-update-check.mjs:52-119). Thus a local setup error becomes generic source-recovery advice.

Commit Gate documentation exceeds its production wiring

The runbook and accepted Lefthook grill describe scaffold materialization plus agent installation. Current source has a planner and merge helper but no production call to planCommitGate or mergeLefthookCommitGate. The handoff renderer accepts commitGateHandoff, while its scaffold caller omits it (apps/cli/src/content/scaffold-copy.ts:259-309; apps/cli/src/scaffold/output.ts:241, :4069-4094).

Harness projection explicitly skips Commit Gate contributions (apps/cli/src/data/scripts/harness-projection/core.mjs:125-130). No production scaffold producer for .devpunks/commit-gate-contracts.json was found. Check nevertheless assesses gate health (repository-check/application.ts:439-468). This is an integration gap; existing planner/unit tests are not end-to-end proof. Routine installation moving into the CLI is a further accepted change under Q1.

The candidate failure is reported twice, but its root cause remains open

Issue #197 reports missing candidate workspace-local effect-tsgo in Harness. Its Emera reproduction reports the same failure despite successful ordinary installs and exact tool pins. The latter reports recovery through scaffold followed by manual Commit Gate setup. These are provider-read reports, not fresh local reproductions here.

Update prepares dependency and structured-entry changes before lint preview (apps/cli/src/update/run.ts:3302-3309, :4783-4829). Preview's outer exception handler labels broad failures config-load (apps/cli/src/runtime/scripts.ts:712-760); that category alone does not prove the failing boundary. Real workspace-bin topology, reconciled candidate manifests/lockfile and invocation paths require a disposable real-install probe.

Preserve issue #181 AC-011/012: isolated candidate, contained symlinks, disabled lifecycle scripts, protected live state, proven cleanup; dependency/process/config failures block publication, ordinary lint findings warn. The issue's expectation that lint must be clean does not supersede that accepted warning contract.

Agent regeneration is broad, while the operator instructions are selective

Update compiles with preserveExistingSubagentManifest: false (apps/cli/src/update/run.ts:3522-3529, :5301-5308). Scaffold planning includes subagent prompt/spec/module generation (apps/cli/src/scaffold/output.ts:3613-3711) and invokes synchronization during materialization (:3910 onward). Existing merge logic can preserve custom fields (:2148-2345), but update bypasses it. This proves broad planning/synchronization, not that every invocation rewrites every authored file.

The current Context Plan contains semantic contribution information but not per-output invalidation lineage (packages/scaffold/src/context-plan.ts). Keep filesystem input hashes outside that semantic model. The independently installed hi-cli post-command contract already says to follow current changed categories, not infer work from the mere existence of .devpunks/ artifacts.

Scoped AGENTS.md is repository-authored; templates/specs and native agent projections are different outputs. Current context compilation primarily derives skill selection from packs and prompt specs (scaffold/output.ts:2866 onward), while the runbook describes authored scoped skill rows as activation authority. That discrepancy needs an explicit consistency contract, not another silent source of agent skill selection.

Proposed architecture

Additional ownership evidence from Q4 research

Current check already exempts categories from byte drift; this is not a complete absence of editable-context handling. isProjectAuthoredManagedFile (apps/cli/src/update/run.ts:1051-1084) covers existing agent-prompt files, handoff, manifest, prompt-spec, subagent, and selected mirror/script/wiki paths. resolveFileOwnership converts these to project-generated-current (:1788-1840), which compareObservedState skips (:3163-3173). Other managed files still receive byte or semantic JSON comparison (:3190-3205).

Some exemptions depend solely on kind, so a missing prompt spec, subagent, manifest or handoff can also be skipped. Conversely, .agents/AGENTS.md is an agent-prompt (apps/cli/src/scaffold/output.ts:2587-2630), exempting edits even though the runbook describes shared guidance as verbatim baseline output. These are false-negative risks and contradictory ownership, not proof of the user's specific false-positive reports.

Managed file receipts contain path/kind/hash/type/mode and optional ProjectGenerated ownership (packages/scaffold/src/baseline/managed-file.ts:8-91; apps/cli/src/update/run.ts:3036-3105), but do not explicitly separate content editability from required presence. Provider mirror handling already uses more specific prior-hash/source-parity evidence (update/run.ts:911-941, :999-1032). Retain that precision instead of broadening path exemptions.

Proposed lifecycle

Use the existing Context Plan, Scaffold Plan and receipt boundaries. Make their connections complete before adding another general workflow engine.

The loop terminates when identified work is verified; an unchanged pending action must be resumed, not rediscovered and recreated. A new command without relevant changes does not trigger authoring. Installer execution is an environment effect: the plan is reproducible, while operational success must still be observed.

1. Give commands one lifecycle with distinct intent

Candidate: scaffold establishes the first managed setup; update reconciles an existing setup; check observes the same desired state. Existing CLI names and aliases need not change. Init remains onboarding, ensure remains settings, and operator/global-tool commands retain their separate scope.

Consequence: a scaffold fallback cannot bypass a required update validation contract. Use the same dependency/config model and affected validation rules in both write paths. This addresses the reported path divergence without asserting that the candidate ENOENT is solved by architecture alone.

Tradeoff: first-adoption and refresh still need different ownership defaults; sharing a plan does not mean overwriting repository-owned setup.

2. Make ownership explicit at the actual unit of change

Ownership may be an entry within a shared file, such as Lefthook's HI lint command or a package dependency. A whole-file hash cannot authorize overwriting unrelated repository entries. Existing managed reconciliation already has this concept; apply it consistently instead of introducing a second ownership system.

3. Regenerate from declared dependencies, not a new baseline label

Candidate: attach producer version and relevant input fingerprints to existing Scaffold Plan/receipt outputs. Keep these outside Context Plan. A materializer declares which inputs it consumes; check compares those inputs and observed outputs before scheduling work. This records why an output is stale.

Tradeoff: skill semantics cannot safely be inferred from arbitrary Markdown diffs. Prefer an explicit authoring/activation revision declared by the baseline producer for semantic changes, and use content hashes for byte delivery. Where that declaration is absent, report uncertain impact or perform bounded inspection; do not pretend a content hash proves semantic compatibility.

Local topology changes and explicit repair are proposed additional triggers. The user's requirement rules out unrelated baseline churn; whether local changes automatically schedule authoring still requires closure. Do not hash all source files and invalidate scoped guidance after every code edit.

4. Make handoff residual, structured and resumable

Candidate: emit each remaining action with scope, reason, exact changed inputs, prerequisites, permitted outputs, ownership constraints and completion checks. Use the structured command result as the source for both JSON and human/agent instructions. Do not ask an agent to reread every generated artifact.

Example: changing a backend authoring criterion identifies the API scope, the old/new criterion, its current AGENTS.md, and the validation needed. It does not request reauthoring unrelated UI guidance or reinstalling unchanged skills.

Reuse existing receipt infrastructure with sufficient per-action evidence to resume after interruption. Distinguish files applied, runtime verified, and authoring pending. An unchanged retry must retain the same pending work and successful evidence. Avoid creating a second database or an autonomous background agent service for this bounded workflow.

5. Make partial failures truthful and useful

Accepted Q3 requires check to keep independent results visible. Candidate dimensions: baseline availability, managed files, dependency/runtime health, Commit Gate, skill binding, scoped guidance freshness, native projection and pending handoff. Each has current/drifted/blocked/unknown/non-applicable evidence as needed; exact schema remains open. No unavailable component becomes clean.

Writes should apply independent safe work under existing mixed reconciliation, but treat dependent changes as a unit: config, required dependency and its runtime verification cannot be claimed complete separately. A successful skill copy may remain applied when independent lint installation fails. Receipts must preserve that distinction without advancing failed action evidence or implying complete baseline adoption from a pin alone.

Preserve strict prior receipt validation from issue #194. A missing completed receipt is a failure, not permission to reconstruct success from generated files.

6. Make the isolated candidate match the intended dependency topology

Candidate: build the preview from the same resolved action set that publication will apply. Include root/workspace manifests, catalog configuration, supported lockfile, owned config changes, patches and install topology needed by affected commands. Verify the executable from each actual invocation directory before lint runs. Keep containment and disabled lifecycle scripts intact.

Classify failures at their real boundary: candidate construction, manifest/lock resolution, install, executable resolution, patch, config load, diagnostics or cleanup. Capture confined relative owner/path and command evidence, not merely applied: false or a catch-all config-load label.

Only preview dependency/config execution when its inputs changed or evidence is missing/stale. A skill-only update should not reinstall a complete lint candidate. Skipping a required preview needs valid dependency evidence; it cannot be based only on a baseline version or selected pack name.

7. Check ownership and obligations, not every original byte

User requirement Q4 makes this a central correctness property. “Created by scaffold” does not mean “forever owned by the baseline.” Candidate classification:

Separate content edit policy from presence/validity policy. Editable content may differ from the seed while a required artifact must still exist, parse where applicable, and maintain mechanically verifiable references. Optional output may be absent without an error. Baseline-owned shared .agents/AGENTS.md and repository-authored scoped AGENTS.md should not inherit one policy merely because both names end in AGENTS.md.

Candidate examples: editing an API coding standard's prose is normal authoring; breaking a required reference is a specific contract failure; a new baseline authoring requirement is pending reconciliation; editing a generated native mirror away from its current authoritative manifest is real generated-output drift. These conditions must have separate reasons and repair actions.

Do not update recorded hashes to make check green, ignore all AGENTS.md files, or relabel every local edit repository-owned. Ownership must come from the artifact producer's explicit contract, including field-level ownership where needed. Legacy receipts require an evidence-based migration: recognize proven authored outputs, retain content, and mark uncertain ownership unresolved.

Existing Scaffold Output Normalization decisions kept particular baseline-owned files byte-authoritative. Q4 does not automatically reclassify those files. Each artifact needs an honest ownership decision, and any changed prior decision must be recorded explicitly.

Behavioral proof must include scaffold → intended authoring → check, with no false drift; unrelated baseline update → check, with no forced reauthoring; relevant authoring-contract update → scoped pending work; generated mirror tampering → real drift; mixed-file user entry edits → no HI drift; removal of an HI-owned required entry → a specific finding. Semantic requirements that the CLI cannot verify mechanically must be reported as needing interpretation, not guessed from text hashes.

Operating from the agent's seat

All six surfaces apply because the operator alternates CLI execution and agent authoring. No remote agent scheduler or new hosted service is required by the current brief.

Focused brainstorm: .devpunks/ artifact lifecycle

The user requested this final focused pass because .devpunks/ artifacts often appear in check/update failures. The recurrence remains user evidence; this pass identifies concrete responsibilities and failure boundaries without claiming to have reproduced each historical failure.

Current evidence

• The local .devpunks/scaffold-manifest.json mixes installation receipts with discovery facts, output inventories, prompt specs, baseline details and projection status. Its saved absolute cwd and outputDirectory differ from this worktree. That is relocation evidence, not proof of a drift bug.
• The directory also contains delivery/, finder-phase/handoffs/, pre-existing-skills/, replaced-skills/ and replaced-scaffold/. Their presence does not establish current scaffold obligations. Repository detection already excludes .devpunks (apps/cli/src/integrations/repository-detector.ts:24).
• Settings already distinguish user choices, calculated tools, managed versions and Commit Gate policy (apps/cli/src/features/project-settings/model.ts:15-104; service.ts:304-390). Preserve this field-level authority.
• The current runbook documents semantic context-plan/receipt comparisons, broad authored-artifact exemptions, and migration from required-tools.json into settings (docs/runbooks/hi-cli-scaffolding.md, drift and consumer-gate sections). These are existing mechanisms to correct and complete, not new ideas that the code wholly lacks.
• Handoff prose embeds the output root (apps/cli/src/content/scaffold-copy.ts:394). Portable identities and machine-local diagnostics should remain distinct. Ordinary path relocation must not be equated with changed baseline intent, while a real broken path reference must remain actionable.
• Update/check decode the manifest before continuing (apps/cli/src/update/run.ts:366-373); that boundary has no missing/corrupt manifest recovery path. Sync separately validates manifest ownership and projection entries (apps/cli/src/data/scripts/sync-subagents.mjs:1362-1400).
• context-plan.json is an active input to synchronization, not merely a report. Absence can select legacy behavior, while malformed content produces typed projection failure (sync-subagents.mjs:2632-2651, :2968-3014). A repository that has adopted canonical context should not silently fall back to legacy behavior merely because that required file disappeared.
• Sync writes projection proof and then updates manifest evidence through separate guarded writes (sync-subagents.mjs:1546-1651). The pair is not a single atomic filesystem transaction. Interrupted publication is a concrete recovery boundary. Prior copy-fallback loading catches malformed receipt parsing (:3024-3039); that tolerant path must not be mistaken for evidence that a corrupt adopted receipt is healthy.
• Update already treats context/receipt JSON through a separate evidence lane (update/run.ts:4190-4209, :4237-4246). Preserve semantic-format tolerance while fixing lifecycle coherence; do not replace it with raw byte equality.
• Sync imports the authored neutral subagent manifest and checks its IDs/skill closure against the context plan (sync-subagents.mjs:2984-3008). By contrast, the bounded research found prompt/lint specs and subagent authoring specs used as handoff/trace context, not native runtime agent manifests. Keep their presence obligations tied to the current authoring action.

Proposed artifact contract at the existing paths

Keep the paths initially. Directory reorganization adds migration work without solving authority. Centralize the policy for each existing artifact: owner, authoritative inputs, content comparison, required presence, consumers and recovery. Reuse established kinds/ownership where sufficient; avoid a second manifest that duplicates the same state.

Important migration constraint: calling an artifact “derived” does not make its current reader disappear. A file still consumed by synchronization or a runtime command remains required until that reader can receive or recreate validated current input. Similarly, receipts are operational evidence, not disposable caches. No proposal permits deleting .devpunks/ as a repair strategy.

Candidate improvements and tradeoffs

A. Let generated instructions be views, with one structured authority. Evidence: generated handoff text, scaffold inventories, context snapshots and receipts currently coexist, while handoff text is treated as project-authored. Consequence: the operator reads a current action, its reason, allowed outputs and proof requirements instead of reconciling several old descriptions manually. Human notes and authored results remain protected. Tradeoff: migration must extract or retain existing authored handoff content; treating all historical handoffs as freely replaceable would discard user work.

B. Distinguish reconstructible inputs from historical proof. Evidence: context snapshots describe intended contributions; receipts attest to completed outputs and are consumed under the strict issue #194 contract. Consequence: missing generated context can be recomputed from valid authority, but missing application evidence cannot be invented. A proof-recovery path must verify actual state and establish new evidence without claiming past success. Tradeoff: uncertain ownership may prevent safe writes even when some output bytes can be regenerated. Block that dependent action and report independent results rather than resetting ownership globally.

C. Prevent circular validity dependencies. Candidate invariant: accepted inputs produce a plan; the plan produces outputs; verification produces receipts; the final adopted manifest references completed evidence. A consumer must not require the new receipt before its producer has finished. Prior valid evidence may seed an update only when its recorded input identity matches. A receipt does not recursively hash itself or volatile views. This extends the issue #194 producer-ordering lesson; it is not a claim that a new self-hash cycle was reproduced here. Tradeoff: coupled output/evidence publication needs a recoverable commit boundary and interruption tests.

Concrete minimum: publish outputs, projection proof and manifest advancement as one recoverable operation, with enough durable pending-state evidence to detect and finish or safely reject interrupted publication. Do not claim a multi-file atomic rename. Do not require the next receipt as a prerequisite to repairing the operation that was supposed to produce it. Check reports partial evidence; an authorized recovery verifies current output before publishing fresh proof.

D. Make repeat runs stable and worktree relocation ordinary. Evidence: saved roots differ from the current checkout; handoff text contains an absolute output root, and semantic JSON comparison already exists. Consequence: compare repository-relative identities and canonical semantic content. Keep timestamps, temporary roots and diagnostic locations out of semantic fingerprints. Preserve ordering where it changes contribution behavior; sort only collections whose order is irrelevant. Tradeoff: genuinely machine-local evidence, such as a live Git hook installation, must be reverified in the current worktree rather than blindly reused.

E. Report the broken obligation and its affected consumer. Candidate output: projection evidence missing; native agent verification unknown, authoring contract changed; API guidance review pending, or generated lint selection invalid; lint setup blocked. A historical run note or backup difference produces no scaffold finding. An optional generated view that no active consumer needs can be regenerated without claiming installed-product drift. Check remains read-only; update performs the authorized repair. Tradeoff: exact exit policy depends on whether the affected consumer is required; that requirement must be explicit, not inferred from the directory name.

F. Make legacy artifact migration finite and ownership-preserving. Evidence: old required-tools.json references already migrate into settings, and authored context currently receives broad exemptions. Consequence: versioned migration should normalize known old contracts once, preserve authored material, and distinguish unsupported schema from local content edits. Unknown files stay user-owned. Tradeoff: migration cannot silently adopt missing/ambiguous historical ownership or erase evidence to force a clean result.

Agent operating sequence

Intake: read the current command result and only artifacts explicitly referenced by pending work. State: distinguish policy, desired context, applied facts and run evidence. Control: resume the bounded action against its current input identity. Feedback: report the artifact, violated obligation and affected scope. Recovery: recreate derived state where authority is intact; reverify missing proof; preserve uncertain ownership. Handoff: author identified context, then return it to deterministic validation/projection. All six surfaces apply; no background scheduler or wholesale directory rewrite is required.

Behavioral cases needed for this directory

1. Move a clean checkout to another worktree: no semantic scaffold drift from saved absolute roots; reverify genuinely local hook evidence as needed.
2. Reformat JSON or change an authored handoff note: no false baseline drift.
3. Delete an optional unused view: check remains truthful; update can regenerate it. Delete a required consumed input: report the specific affected consumer.
4. Delete/corrupt a completed receipt: report missing/invalid proof, never clean state inferred from file existence; allow only evidence-backed recovery.
5. Interrupt output/receipt publication: retain truthful prior and partial state; retry converges without a missing-producer deadlock or unrelated reauthoring.
6. Edit/archive workflow records or recovery copies: no baseline drift or automatic deletion. Unresolved rollback still retains its required evidence.
7. Apply a known legacy-schema migration and repeat update: no recurring migration work, unnecessary file rewrites, or lost authored content.
8. Change a relevant authoring or lint contract: only affected inputs, outputs and handoff entries change. Repeat without changes: no new work.

These are candidates, not executed tests. Remaining decisions concern which views should remain persisted, how legacy authored text is retained, explicit required-versus-optional consumers, and the exact proof-recovery boundary.

Evidence needed before implementation can claim success

• Reproduce issue #197 with a disposable real Bun monorepo install and compare candidate versus ordinary workspace binary resolution. Root cause remains open.
• Exercise public scaffold/update/check seams: fresh supported project without quality scripts; custom override; conflicting hook manager; disabled policy; missing receipt; partial failure and retry; no-op repeat.
• Verify lint-only, skill-body-only, authoring-contract-only, adapter-only and topology changes touch only their expected dependency closures.
• Prove first scaffold and update produce matching required gate state, including contracts, runner, dependency, shared config ownership, installed hook and proof.
• Verify check reports independent facts with explicit unknowns and never invokes mutating producers. Confirm the session hook preserves these distinctions.
• Test authored guidance/custom roles survive unrelated updates. Native projections consume one explicit authority for scope/skill membership and remain coherent.

These are proposed behavioral proofs, not tests run in this assessment. No product source, reusable skills, releases, external issues or consumer setups were changed. Documentation route/projection validation is recorded at handback.

Assessment validation on 2026-09-05: node apps/wiki/scripts/sync-content.mjs --check passes after refreshing the one changed runbook projection; git diff --check passes. Fresh installed hi check --json fails with the recorded incomplete Quality Command Contract. The candidate update failure was not rerun here.

Research lanes and unresolved choices

Three readonly lanes covered update/check/candidate execution; scaffold/quality/ Commit Gate wiring; and skills/scoped guidance/native projection. A bounded follow-up traced editable-context ownership and drift suppression. Two additional bounded follow-ups mapped .devpunks/ consumers and receipt publication/recovery. The coordinator read the GitHub issue and discussion, reproduced check, resolved the runbook/code contradiction and wrote this consolidated report.

Remaining decisions: semantic change declaration ownership; local-change authoring triggers; first-adoption versus refresh ownership; minimal durable handoff shape; exact coupled-action failure boundaries; and compatibility for existing receipts/operator skill versions. R1 foundations are accepted, but shared understanding is not yet closed and no delivery backlog is projected.

Requirements closure after this assessment

Q1-Q7 are now accepted. The closed grill status and compiled specification supersede earlier open-decision language above. The report retains the chronology of discovery; candidate cases remain unexecuted delivery verification targets.