Harness Intelligence Wiki
SpecsCLIIP-106-cli-harness-reporting

Spec: CLI-Backed Harness Reporting

Spec: CLI-Backed Harness Reporting

User Input

IP-106 outcome: CLI-backed harness friction reports, attached to ProjectUsage, observed repository, and authenticated operator when available; maintainers triage in backoffice with GitHub-linked follow-up. Coordinate with IP-143.

Harness reports link to GitHub in v1. Linear remains product/backlog planning. Backoffice report statuses include at least new, triaged, accepted, wontfix, resolved.

Initial Situation

The CLI has auth, scaffold, and update commands only. The API and shared contract expose health and baseline endpoints only. Docs define reports as a v1 feedback loop, but no CLI report command, API report endpoint, report schemas, persistence model, or distributed dp-cli guidance exists.

Issue

Agents and developers currently have no structured, central path to report harness friction from the repository where the friction occurs. Without that path, issues stay in chat, local notes, or product backlog noise, and maintainers lose the repository context needed to improve skills, hooks, wiki guidance, CLI behavior, or baselines.

Solution

Add a CLI-backed reporting path that submits structured reports to the control plane. Reports carry safe observed repository facts, project usage linkage when available, authenticated operator identity when available, git identity as fallback/context, type/severity/title/body/timestamp, and later GitHub follow-up metadata. The CLI command is ergonomic for agents and humans. Backoffice triage UI remains M4.5, but the report contract and persistence shape must support IP-143.

Child Coverage

ChildRequired outcomeCovered by
IP-133Agent/developer can submit structured harness friction through dp; success emits cli.harness.report.submitted; reports persist for backoffice; GitHub is follow-up target.CLI report command, API report endpoint, telemetry event hook, docs/skill guidance.
IP-134Systemic reports can become central backlog items without turning every project-specific report into Linear work.Report status/type model, GitHub link field, docs/runbook triage workflow.

Non-Goals

  • Full backoffice triage UI.
  • Making Linear the default destination for every report.
  • Public customer support portal.
  • Automatic GitHub issue creation if required GitHub credentials are absent.
  • Reporting consumer-repo business logic into the Harness CLI.

Acceptance Criteria

  • CLI exposes a report command usable as punks report / dp report with non-interactive flags and JSON output.
  • Report payload includes title, body, type/category, severity, timestamp, CLI version, safe observed repository facts, project usage reference when available, authenticated operator when available, and git identity metadata as fallback/context.
  • CLI rejects empty title/body and does not leak raw local paths or credentials in repository metadata.
  • API exposes an authenticated report submission endpoint with typed request/response schemas and typed errors.
  • Reports persist in a model compatible with ProjectUsage, observed repositories, report status, and optional GitHub issue/PR URL.
  • cli.harness.report.submitted emits only after successful report submission.
  • Docs and distributed dp-cli guidance explain when agents should submit reports and when they should create product backlog issues instead.
  • IP-143 backoffice/GitHub linkage remains coordinated and not prematurely implemented as a full app.

Constraints

  • IP-106 follows IP-105 telemetry foundation.
  • GitHub is v1 follow-up system of record for reports.
  • Linear remains planning/backlog state.
  • Reports must distinguish local quirks from systemic harness issues.
  • Report ingestion belongs to apps/api; CLI only collects safe local context and submits.

Technical Notes

  • Current CLI client has only resolveStableBaselineMetadata.
  • Report schemas likely belong beside telemetry schemas in packages/contract.
  • Report persistence likely shares ProjectUsage and ObservedRepository tables with IP-105 and adds harness_report.
  • The command can start with explicit flags and JSON mode; interactive prompting can remain minimal if non-interactive agent usage is first-class.

Validation Plan

  • Contract tests for report schemas, status literals, and OpenAPI endpoint.
  • API tests for unauthenticated rejection, valid report submission, invalid payload rejection, and persisted/new status.
  • CLI tests for flags, JSON output, empty field rejection, auth token reuse, repository context sanitization, and telemetry event after successful submit.
  • Built CLI smoke: punks report --help and a local fake/control-plane report scenario where possible.

Decision Log

DecisionRationale
Make report command top-levelIt is a product feedback path, not a scaffold subcommand.
Persist reports before full backoffice UIIP-106 needs submission and triage support; M4.5 owns the primary UI.
Keep GitHub link optional at submissionEnvironments may lack GitHub credentials; report should still be accepted for later maintainer triage.

Open Questions

#QuestionAffectsOwnerStatus
1Should automatic GitHub issue creation be enabled in M4 or represented as a linkable field until backoffice IP-143?GitHub smoke evidenceDeliveryNon-blocking; default to linkable field unless credentials/config exist.

On this page