Why: Real transcripts (dev instance, 62 assistant turns) showed the artifact
gate silently off and self-deadlocking. lintFile ran python3 -m housestyle.lint
without the skills PYTHONPATH, so the import failed and nil was treated as "no
issues" (same docx: 9 lint errors with PYTHONPATH, 0 without). stop() re-scored
every artifact registered in the run, including superseded versions, so a fixed
document could never pass and the composed reply reported the old verdict.
ctxasm rendered history artifacts as "[产出工件 …]" - the exact pattern
ClaimGate treats as a forged tool marker, so models copying the format were
rejected. scoreReadable flagged table cells like "项目概述撰写" as column
headings. Prompts and recipes said "python -m" (no python on PATH; exit 127)
and housestyle.summarize was invoked as a module that does not exist.
Evidence: SELF - new tests TestLintUnavailableFailsVisibleWhenSkillsRootSet,
TestLintCommandEnvCarriesSkillsRoot, TestDocxHeadingMarkedAndTableCellNotFlagged,
TestLatestDeliverablesKeepsLastPerName, TestRecipesUseRunnablePythonCommands,
TestLibraryUsesRunnablePythonCommands; go test ./internal/... passes; manual:
python3 -m housestyle.outline_cli reads a real artifact docx cleanly, lint
without PYTHONPATH exits 1 with ModuleNotFoundError and is now surfaced as
自检未运行 when the skills root is configured.
Impact: compose root injects quality.SetLintEnvRoot(<data>/skills). Final-turn
checks score only the newest artifact per name; deliverable lists and reply
cards follow the same dedupe. History artifact placeholders no longer collide
with the forged-marker regex. docx text extraction marks Title/HeadingN styles
so readability only judges real headings, with line numbers in the reason.
Check reminders no longer invite the model to ignore failed checks. Oversized
tool output is truncated by runes, not bytes.
Refs: Dev task feature/agent-quality-phase01 (quality gap diagnosis 2026-10-09).
Why: No task-level measurement of real agent behavior existed, so the
failure chain documented in the diagnosis was invisible to CI. The harness
drives a configured backend over its public HTTP surface (upload, message,
poll, HITL auto-resolve), computes per-turn metrics (tool failures, full
regenerations, API-discovery calls, original-material reads, plan usage,
gate checks) and outcome checks (unverified facts vs material text, final
claims vs artifact headings), and writes a JSON + Markdown report.
Evidence: SELF - built behind the live build tag (CI skips); vet passes with
-tags live; drove the qa instance (remote main 1ebd46e) through the four
golden tasks end to end; report referenced on the phase01 task card.
Impact: New eval/live package (tasks.json + runner). Materials stay outside
the repo per eval/README; runs are opt-in via AGENT_EVAL_BASE.
Refs: Dev task feature/agent-quality-phase01; documents proposal
2026-10-09-agent-quality-improvement-plan.md §4 P0.
Why: lint-boundaries rejects new app->quality edges (architecture ratchet);
runtime already depends on quality for the scorer and holds SkillsDir, so the
injection belongs there. Same behavior, allowed dependency direction.
Evidence: SELF - go vet + go test ./internal/runtime ./internal/app pass;
lint-boundaries edge list unchanged.
Impact: quality.SetLintEnvRoot now called from LLMRunner.Execute when
SkillsDir is configured (idempotent, process-wide).
Why: The live report's model field came back empty because the settings
endpoint nests models under the user scope ({defaults, project, schema, user}),
not at the top level; the before-baseline report had to be annotated by hand.
Evidence: SELF - vet with -tags live passes; the scope shape verified against
GET /api/settings on the qa instance.
Impact: Future live reports carry the served model without manual annotation.
weichen
merged commit c0b52ac8bd into main2026-10-09 14:12:47 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
摘要
Phase 0+P1 首个增量（主记录：Documents
2026-10-09-agent-quality-improvement-plan.md）：[产出工件 …]同形python3 -m；新增housestyle.outline_clieval/live/（build taglive，CI 跳过）：golden 任务集 + 真实实例HTTP 运行器 + 指标报告（失败率/整篇重跑/读原件/计划使用/检查记录/未核实事实/终轮一致性）。
before 基线（旧 main 1ebd46e，真实模型 17 分钟，Dev 任务卡已入库）
诊断的失败链全部复现。after 对照在本 PR 合入后按同一任务集补跑（裁定 6 口径）。
Evidence
go test ./internal/...全绿（新增 6 个守护测试：lint fail-visible/env、docx 标题感知、同名去重、recipes/prompts 命令守护）
make test在 Dev worktree 通过（任务卡 evidence 行，SELF）.task/phase01-eval/before-main-1ebd46e.jsonpython3 -m housestyle.outline_cli干净回读真实工件；无 PYTHONPATH 时lint exit=1 且评分器报「自检未运行」
Impact
配置时强制可运行（fail-visible）
任务卡全文：Dev 仓
.task/feature-agent-quality-phase01.mdWhy: The live report's model field came back empty because the settings endpoint nests models under the user scope ({defaults, project, schema, user}), not at the top level; the before-baseline report had to be annotated by hand. Evidence: SELF - vet with -tags live passes; the scope shape verified against GET /api/settings on the qa instance. Impact: Future live reports carry the served model without manual annotation.