Harness Intelligence Wiki
Harness

Trust

How Harness makes AI-assisted work inspectable, adoption-ready, and supportable

Harness trust starts with inspectable evidence. The team should be able to see what the harness changed, which guidance shaped the work, which checks ran, which risks remain, and how friction returns to maintainers.

This page merges the trust and adoption path into one operating surface. It explains what Harness can prove, where judgment stays human, how adoption readiness is checked, and how support feedback moves from local friction to durable fixes.

Trust Boundaries

Trust the harness when its artifacts and checks are inspectable: generated files, selected packs, scoped prompts, validation output, review findings, implementation notes, and docs ingest evidence.

The team should be able to answer concrete questions after a run:

  • Which repo guidance, scoped AGENTS.md files, memory notes, and skills shaped the agent?
  • Which generated prompts, hooks, subagents, required tools, and metadata changed?
  • Which tests, type checks, builds, route checks, browser checks, or manual review steps ran?
  • Which files, specs, logs, screenshots, reports, or docs artifacts support the result?
  • Which parts still need operator judgment because credentials, external systems, policy, or live-user context were unavailable?

AI output still needs review. Missing tests, flattened child stories, broad parallel workers, stale docs, missing tools, unchecked scaffold drift, and unsupported confidence all require blocking, debugging, debt capture, or support-loop reporting.

Validation Story

Automatic checks prove repeatable facts. Tests, types, builds, route rendering, schema validity, and scaffold update drift give the team evidence that a narrow surface behaves as expected.

Human review owns scope fit, risk, wording, supportability, and product boundary. Review decides whether the chosen checks match the change and whether the remaining judgment is acceptable.

A strong validation story names both sides:

  • Automatic proof: commands run, surfaces checked, generated files inspected, and drift explained.
  • Human proof: acceptance criteria preserved, risks reviewed, docs updated or explicitly skipped, and unresolved context named.
  • No-op proof: skipped gates have specific reasons such as docs-only change, unavailable browser target, or existing coverage for the touched path.

When validation cannot run, the final report should say so plainly and keep the blocker visible.

Failure Model

The harness is allowed to fail loudly. Silent confidence is the failure to avoid.

Known failure modes include:

  • Stale prompts that no longer match the repo.
  • Global phase skills copied into scoped primary skill lists.
  • Child-story acceptance detail lost during planning or backlog sync.
  • Overlapping parallel workers with unclear write ownership.
  • Missing browser proof for user-facing behavior.
  • Docs that describe intent while the implementation says something else.
  • Reports that never enter the support loop.

Handle these failures by stopping at the failing gate, gathering evidence, and routing the work to the right recovery path. Runtime symptoms go through debugging. Stale scaffold or prompt output goes through update drift checks. Missing docs evidence goes through docs ingest. Reusable friction goes through dp report or punks report.

Criteria Checklist

A repository is adoption-ready only when the harness evidence is inspectable.

Before adoption, verify:

  • Setup wrote the expected managed files.
  • Required tools are present or have actionable remediation.
  • Scoped prompts match the real repo surfaces and ownership boundaries.
  • Generated skills, hooks, subagents, and metadata are coherent with the selected stack.
  • dp update --check or punks update --check is clean, or every drift item is explained.
  • Relevant validation passes for the touched surfaces.
  • Manual review owns the parts automation cannot prove.
  • The support loop is known, including when to use reports instead of product backlog items.

This checklist protects adoption from becoming belief in generated text. The repo is ready when the team can inspect the operating trail.

Golden Repo Walkthrough

Use a representative golden repository to prove the adoption path before treating the harness as ready for wider use.

The walkthrough starts from a realistic repo, then:

  1. Run punks scaffold setup.
  2. Inspect generated prompts, skills, hooks, subagents, metadata, and required tools.
  3. Run punks update --check.
  4. Validate the detected stack with the narrowest useful checks.
  5. Record what automatic checks proved.
  6. Record what human review still owns.
  7. Capture any failure, remediation, or support-loop report created during the run.

Use a generated representative fixture until a real internal golden repo is approved. The walkthrough should show both the happy path and the failure handling path, because adoption trust depends on both.

Support Feedback Loop

Harness friction should become maintainable evidence. Local quirks can be fixed in scope. Reusable friction should enter the report path, get followed up in GitHub, and move to Linear only when it becomes product planning scope.

punks report and dp report create rich GitHub issues through the report command flow. Use title, body, type, severity, area, affected skill pack, affected command, expected behavior, actual behavior, evidence, and labels to make the issue actionable.

Accepted fixes flow back into skills, hooks, prompts, wiki pages, runbooks, CLI behavior, npm releases, or baseline releases. The loop is useful because the same friction that slows one repo can improve the harness for the next repo.

Project Usage

The Harness backoffice is the internal product surface for project usage visibility. It shows ProjectUsage aggregates, mapped observed repositories, counted happy-path events, current baseline or CLI state, and open reports.

Better Auth email OTP is the canonical backoffice operator identity flow. Protected server pages authorize with requireBackofficeOperator() before loading request-bound project usage, access, or report data. The backoffice forwards the browser session cookie through the shared @punks/contract typed client, and the API validates that session before checking backoffice_access. Git identity is displayed only as observed fallback metadata.

The backoffice must not rank, score, or compare developers. Operator fields exist for audit and contact context.

Backoffice Report Triage

Harness reports are submitted through punks report or dp report as GitHub issues in wearedevpunks/harness-intelligence.

The CLI writes a descriptive issue body, applies harness-report classification labels, and embeds a hidden harness-report.v1 metadata block. Requested issue creation fails before persistence when the GitHub integration is disabled, and provider work is cancelled with the submission. The backoffice reports table fetches those GitHub issues and treats GitHub open or closed state as the status source for GitHub-backed reports. Project usage list/detail surface GitHub availability and database persistence as distinct typed failures.

Maintainers use labels such as area:wiki, surface:cli, pack:dp-cli, or command:dp-report to route follow-up. Linear remains product planning state unless the report becomes product-scope work.

Reports sit near project adoption, but they are separate from happy-path product usage events. Successful report submission emits cli.harness.report.submitted after the GitHub issue exists.

Limits

Remaining judgment stays with the operator when credentials, external systems, policy, live-user context, or incomplete evidence prevent full validation.

Harness gives the team a standard way to shape, constrain, validate, review, document, and improve AI-assisted work. Trust comes from the visible operating trail.

On this page