Files
Agent-System-Backend/eval
weichen 06f5835211
ci / vet-and-test (pull_request) Successful in 41s
Add estimate-marking awareness and a second fact-check round (E3)
Why: The E1 live comparison was the only metric that got worse
(unverifiedFacts 7 -> 12): the gate gave one correction round and then
shipped bare numbers. The design (E3) calls for number sourcing with a
mechanically checkable marking escape: numbers labeled as
targets/assumptions are honest planning quantities, not fabricated
facts, so the gate must let them pass - and the model must be told that
marking is the sanctioned way out.

Why: The marking decision is clause-scoped, not a fixed window - a
window wrongly blessed "3,500" because a neighboring clause said
"target 85%" (caught by the existing test). The second round only
fires when the unverified set actually changed, because the
continuation budget is shared across all final gates and a repeat
block on identical tokens starves the downstream gates (also caught by
an existing test).

Evidence: go test ./... green; new anchors TestMarkedAsEstimate,
TestUnverifiedFactsMarksPass, TestFactCheckGateRounds; the live eval
gains an unverifiedUnmarkedFacts field sharing the same marking rule.

Impact: base.deliverables v6 adds the hard number-sourcing rule;
research/explore/general sub-agent profiles v+1 require citations for
key numbers; fact gate: marked-in-clause numbers pass, round 2
escalates to "last interception" with the marking instruction, then
passes through with the check line. Eval report keeps the strict
series and adds the E3-series field.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md (E3,
D4); Dev task feature/agent-e3-evidence-gate.
2026-10-10 16:08:19 +00:00
..

交付物评测

仓内 testdata/ 是脱敏短样例,用来确认评分卡能区分“结论先行”和“栏目式笔记”。

真实会议材料不要放进仓库。放到 ~/.agent-system/eval/,或设置 EVAL_DIR。其中的 .md 会在 go test ./eval/ 里被打分并打印,缺失该目录时测试跳过。

cd modules/backend && go test ./eval/ ./internal/quality/ -count=1

两种入口的切换由 DecideSwitch 判断:squad 分数不低于 direct,并且直通路径多出来的首字延迟 P50 不超过 3 秒。没有同一评测集上的实测分数时,结果是不切换,runtime.entry 默认保持 direct。