Harness Intelligence Requirements

This document summarizes the locked Harness Intelligence product contract. The detailed decision record remains in:

• Grill log
• Grill status

Product Boundary

Harness Intelligence is the internal AI engineering operating system.

It combines:

• scaffold distribution through the CLI
• private methodology/wiki documentation
• scoped prompts and AGENTS.md policy surfaces
• skill packs and lazy skill loading
• hooks, external tools, and validation gates
• subagent templates and execution modes
• adoption telemetry and feedback loops
• internal backoffice visibility

The public landing/sales narrative is parked. It should eventually expose a sanitized promise and proof story without exposing internal prompts, raw wiki content, or client-specific material.

Target Monorepo

The target Turborepo shape is:

apps/
  cli/          punks/dp CLI executable
  api/          Effect backend/control plane
  wiki/         private Fumadocs wiki and source owner
  backoffice/   internal adoption/reporting UI
  web/          parked public landing

packages/
  contract/     shared Effect HttpApi contract
  scaffold/     only if scaffold schemas/models are shared by apps
  config/       shared build/config support as needed
  ui/           only if apps share real UI primitives

apps/cli is an application, not a package. Internal packages must not import sibling apps.

Use packages/contract for the shared typed API contract imported by apps/api, apps/cli, and apps/backoffice where applicable. Prefer Effect v4 HttpApi and typed clients for v1 backend operations. Use Effect RPC only when later requirements need RPC-specific or streaming/procedure semantics.

packages/scaffold is valid only for stable scaffold/baseline schemas or models that are genuinely shared by apps.

Backend And Distribution

apps/api is the authenticated control plane. The CLI remains the local executor.

The backend owns:

• baseline registry
• channels and versions
• compatibility metadata
• provenance and validation status
• authenticated artifact access
• product telemetry ingestion
• harness report ingestion
• backoffice read APIs

The CLI owns:

• repo scanning
• interactive setup
• filesystem writes
• local cache
• bundled/offline fallback
• update and diff ergonomics
• scoped prompt/skill/hook/tool placement

Baseline artifacts should move toward backend-mediated storage, likely Vercel Blob. GitHub Releases can remain during migration, but they are not the long-term runtime distribution contract.

Wiki

apps/wiki is a private internal Fumadocs app and owns the canonical routed wiki content under apps/wiki/content/docs.

The wiki has a separate harness domain for methodology, theory, criteria, trust model, default skill packs, and adoption flows. The existing cli domain stays focused on hi/hint mechanics.

docs-ingest-phase should ingest real future specs and implementation notes into routed wiki/project pages. It should not synthesize fake specs from grill decisions.

Phases And Goals

Harness execution is organized around public phase wrapper skills.

A phase is a reusable workflow unit with entry conditions, stop conditions, delegated inner skills, artifacts, and validation gates. Phases can be invoked directly in a normal turn, composed by another phase, or named inside a /goal prompt.

The current phase inventory is:

• $requirements-phase: human-centric discovery that wraps $requirements-grill -> $write-backlog
• $delivery-phase: goal-compatible delivery that runs $create-spec -> $create-plan -> $implement-spec -> $review-phase -> optional $debugging-phase -> $docs-ingest-phase
• $review-phase: readonly-by-default findings-first review using autoreview as runtime, then applying architecture, simplification, scoped guidance, and validation lenses
• $debugging-phase: runtime-evidence debugging through logs, repros, optional parallel research, bounded fixes, regression coverage, and reruns
• $bug-discovery-phase: AFK ClawPatch discovery, report generation, and triage without source mutation by default
• $bug-resolution-phase: selected ClawPatch finding classification and bounded resolution
• $docs-ingest-phase: routed wiki/project docs and repo-docs alignment after proven changes
• $resolve-debt-phase: resolution of one concrete tech-debt artifact through the delivery workflow

$goalify is a goal activator, not a phase. It turns messy intent, handoffs, specs, plans, logs, tracker context, and examples into one lean goal, activates it immediately, and continues work under that active goal.

A good $goalify contract includes only one objective, exact context, terse workflow, proof, and stop conditions. It must stay concise enough for /goal; the current hard ceiling is 4,000 characters.

Requirements work remains HITL by default. Delivery goals are the workhorse for scoped implementation because every coding run needs review, optional debugging when evidence requires it, and $docs-ingest-phase when behavior, architecture, setup, operator workflow, docs contracts, or source knowledge changed.

Backlog remains product-facing. write-backlog formats accepted requirements as EPICS and subchildren stories. The EPIC is the parent capability boundary and the subchildren stories are the product slices beneath it.

Execution details belong in SPEC.md, PLAN.md, IMPLEMENTATION-NOTES.md, review notes, validation logs, and debt artifacts. create-spec targets one EPIC and must include every requirement from that EPIC's subchildren stories. create-plan targets the reviewed spec and maps tasks back to the owning EPIC and relevant subchildren stories. implement-spec executes that approved plan without dropping or weakening subchild-story requirements. $delivery-phase preserves this EPIC -> subchildren stories contract while composing $create-spec -> $create-plan -> $implement-spec -> $review-phase -> optional $debugging-phase -> $docs-ingest-phase.

Phase skills are global orchestration entrypoints. Scoped AGENTS.md files should list local execution/review skills only; they should not advertise lifecycle phase wrappers as scoped primary skills.

ClawPatch-backed bug work uses a staged manual lifecycle: discover, inspect, choose, resolve. $bug-discovery-phase is the AFK discovery/report/triage wrapper. It records provider/model as automatic default or explicit override, scope, --limit, --jobs, state directory, report path, and finding IDs. It does not mutate source files by default and does not treat findings as proven runtime root causes. Discovery should not create repo tech-debt docs.

$bug-resolution-phase starts after a finding is selected. It classifies selected findings as resolve-now, needs-runtime-debugging, tech-debt, false-positive, or blocked. Normal Harness/Codex edits are the default patch path; ClawPatch supplies context, report linkage, triage/history, optional dry-run inspection, and revalidation. Runtime-evidence needs route to $debugging-phase. Tech-debt capture happens only after selected-finding classification and routes through $docs-ingest-phase.

Keep all-in-one discover-and-resolve out of the default story. If an operator explicitly asks for a single combined goal, automatic resolution should select only high-confidence findings and keep provider/model default or explicit override, scope, validation, and ClawPatch status gates explicit.

ClawPatch inherits the current agent/launcher provider by default. Goals, commands, and reports should record that automatic default unless the operator explicitly chooses a ClawPatch provider/model override; default command paths omit --provider and --model.

Required Toolchain

Harness setup includes markdown/context assets and external CLI tools.

Required tools include:

• skills
• opensrc
• agent-browser
• portless
• clawpatch

portless is required, not optional. The internal hi-cli skill is part of the distributed harness so agents know how to operate and report through the CLI.

Backoffice

apps/backoffice is a separate authenticated internal product surface.

It exists to answer product/adoption questions:

• which projects adopted the harness
• which repositories belong to those projects
• which baselines are in use
• which repos are stale
• where harness friction is being reported
• who can be contacted for context

OpenTelemetry remains for traces, logs, backend health, debugging, and internal system observability. It is not the primary product/adoption UI.

Product telemetry is persisted in Postgres through apps/api.

The backoffice now consumes authenticated API endpoints for project usage summaries, project details, repository mapping, and harness report triage. It is intentionally separate from apps/web, which remains the parked public surface.

Project Usage Model

V1 has a ProjectUsage aggregate. It maps one-to-many to observed repositories.

Multiple repositories can belong to the same project.

project_usage
  1 -> n observed_repositories
  1 -> n cli_usage_events
  1 -> n harness_reports

observed_repository
  n -> 1 project_usage
  1 -> n cli_usage_events
  1 -> n harness_reports

Observed repositories are facts collected automatically from CLI execution on a developer machine. Automatic mapping can only use what the CLI can safely observe:

• git remote
• branch
• commit SHA
• dirty state
• repository metadata visible from files
• authenticated Better Auth user/session
• CLI version
• baseline/channel/version
• safe environment/source flags

The backend/backoffice can create a project usage aggregate from the first observed repository. Maintainers can later map more observed repositories into the same aggregate.

Backoffice dashboard counts use normal-source counted events for adoption. Local development, test, debug, and CI sources can remain stored for context but should not inflate product adoption statistics.

Identity

Use Better Auth as the application/session framework.

Use Better Auth email OTP as the canonical backoffice login. After successful OTP verification, the backoffice client explicitly refreshes the session-bound router state and returns the operator to the requested protected destination. The backoffice forwards the browser session cookie to the API; the API validates the Better Auth session itself before checking durable access records. CLI baseline, telemetry, and report calls are intentionally unauthenticated.

Backoffice access is invite/pre-authorize based and stored in the database. The API bootstraps default admin access records for:

• stefan.garofalo@devpunks.com
• simone.sabba@devpunks.com

Admins can invite or pre-authorize members/admins and revoke access records from the backoffice. Invited users become active on first verified email OTP session. Revoked users are denied by API backoffice authorization even if a Better Auth session exists.

Backoffice protected server pages call requireBackofficeOperator(nextPath) before loading request-bound access and project data. That gate must wait for the incoming request before checking cookies so Cache Components / partial prerendering cannot bake an anonymous /login?next=... redirect into protected pages. Unauthorized requests redirect to /login?next=...; signed-in operators who open /login are sent back to the requested destination. The shell shows the signed-in account, role, and logout action, and only shows the Access navigation item to admins. The backoffice uses the shared @punks/contract Effect HttpApi client and forwards the signed-in browser session cookie to the API for reads and server-action mutations. API backoffice handlers validate the Better Auth session and durable access records before reading or mutating project usage, report data, or access records. The backoffice app hosts /api/auth/* so email OTP can create a backoffice-origin Better Auth session.

Deployments need the shared database, BETTER_AUTH_SECRET, CORS_ORIGIN, and Resend settings:

• RESEND_API_KEY
• RESEND_FROM_EMAIL

For the backoffice deployment, set BETTER_AUTH_URL to the backoffice auth route, for example https://harness-intelligence-backoffice.vercel.app/api/auth. For the API deployment, set BETTER_AUTH_URL to the API auth route. Vercel deployments may omit BETTER_AUTH_URL because VERCEL_URL is used as the allowed Better Auth host. Keep BETTER_AUTH_SECRET and RESEND_API_KEY in deployment env only.

BACKOFFICE_OPERATOR_IDS and BACKOFFICE_OPERATOR_EMAILS are not authority. Remove them from production deploys.

Git user.name and user.email are observed metadata and fallback context. They are not the canonical developer identity.

Developer usage is audit/contact context, not a KPI. Backoffice must not rank, score, compare, or present "AI usage by developer" as a core metric.

Product Events

V1 stats track only happy-path value events:

• cli.artifact.pull.completed
• cli.scaffold.init.completed
• cli.scaffold.setup.completed
• cli.scaffold.update.applied
• cli.harness.report.submitted

Do not track these as product stats:

• setup started
• baseline resolved
• update checked
• automatic background tool checks

Detailed failures, background checks, and noisy lifecycle timing belong in logs, OTel, or debug surfaces.

Reports

Harness reports are GitHub issues opened by the CLI and triaged in backoffice.

GitHub is the follow-up system of record because harness reports are closer to implementation/tooling issues than product-planning backlog items.

Backoffice report statuses should include at least:

• new
• triaged
• accepted
• wontfix
• resolved

The CLI report path is punks report / hi report. It submits raw report context to /api/reports, carries safe observed repository facts, and emits cli.harness.report.submitted only after the API returns a GitHub issue URL. Public submissions use deterministic metadata; authenticated backoffice operator submissions can additionally generate AI metadata. Operator identity is optional enrichment for this public endpoint: unavailable auth configuration, access, or persistence continues anonymously, while protected backoffice endpoints retain typed auth persistence failures. Accepted reports create or dedupe descriptive GitHub issues in wearedevpunks/harness-intelligence with the hidden harness-report.v1 marker and labels by type, severity, area, skill pack, wiki surface, or CLI command.

The backoffice report table fetches harness-report issues from GitHub and treats GitHub open/closed state as report status for GitHub-backed reports. Legacy API-persisted reports can still be displayed for migration/local smoke tests.