Why: No task-level measurement of real agent behavior existed, so the
failure chain documented in the diagnosis was invisible to CI. The harness
drives a configured backend over its public HTTP surface (upload, message,
poll, HITL auto-resolve), computes per-turn metrics (tool failures, full
regenerations, API-discovery calls, original-material reads, plan usage,
gate checks) and outcome checks (unverified facts vs material text, final
claims vs artifact headings), and writes a JSON + Markdown report.
Evidence: SELF - built behind the live build tag (CI skips); vet passes with
-tags live; drove the qa instance (remote main 1ebd46e) through the four
golden tasks end to end; report referenced on the phase01 task card.
Impact: New eval/live package (tasks.json + runner). Materials stay outside
the repo per eval/README; runs are opt-in via AGENT_EVAL_BASE.
Refs: Dev task feature/agent-quality-phase01; documents proposal
2026-10-09-agent-quality-improvement-plan.md §4 P0.
