Files
Agent-System-Backend/eval
weichen 196c8c09f7
ci / vet-and-test (pull_request) Successful in 42s
Add the task-level A/B dual-run harness with demo.brief evidence (Q3 WP5)
Why: Q3 WP5 requires a same-batch dual-run harness (direct vs squad
compiled graph) that records differences without demanding byte-identical
output, feeding DecideSwitch evidence for the runtime.entry default-switch
decision.

Evidence: eval/ab_test.go mechanics (theme-based scoring, status/theme/
latency diff notes, P50, DecideSwitch wiring, RunFunc error tolerance,
empty batch); wfcompile TestABDemoBriefDualRun runs the demo.brief scenario
on both entries (direct = scripted single-agent answer; squad = real
RunCompiled with stub backend and judge) and logs the report: both sides
100/100, summary-form difference recorded, decision '切换条件已满足…合入前
需要产品确认'. Full backend suite green (SELF); CI authoritative (GATE).

Impact: new eval/ab.go harness (pure mechanics, no runtime dependency);
no product default is flipped - runtime.entry stays direct until product
confirmation per the DecideSwitch report semantics (evidence recorded in
the task card and PR).

Refs: Q3 plan WP5 (2026-10-07-agent-stack-q2q3-plans.md); task card
.task/feature-agent-stack-q3.md
2026-10-08 03:46:54 +00:00
..

交付物评测

仓内 testdata/ 是脱敏短样例,用来确认评分卡能区分“结论先行”和“栏目式笔记”。

真实会议材料不要放进仓库。放到 ~/.agent-system/eval/,或设置 EVAL_DIR。其中的 .md 会在 go test ./eval/ 里被打分并打印,缺失该目录时测试跳过。

cd modules/backend && go test ./eval/ ./internal/quality/ -count=1

两种入口的切换由 DecideSwitch 判断:squad 分数不低于 direct,并且直通路径多出来的首字延迟 P50 不超过 3 秒。没有同一评测集上的实测分数时,结果是不切换,runtime.entry 默认保持 direct。