weichen
ef060d4ee5
ci / vet-and-test (pull_request) Successful in 40s
Why: User rulings 4/5 plus the P2-P4 live evidence. Keyword scoring dimensions (readability, actionability) kept failing valid documents - normal Chinese section headings flagged as column titles, loops of 4 corrections - because they grade writing preferences, not verifiable facts. Unverified numbers (~10-13 per run) remained the largest quality leak through every phase. And the verifier runs on the same model as the worker (lightweight falls back to chat), so "independent" verification was self-review. Separately, svc seeding after boot never registered the tavily tools until a restart because keys were snapshotted at startup. Changes: quality.HardFacts (lint actually ran + zero errors) replaces the full ScoreBytes projection in CheckArtifacts - keyword dimensions remain in ScoreBytes for eval visibility only. New fact-consistency check (runtime.factCheck, default on): extract percentages/dates/large numbers from deliverable text, trace each against the material corpus (attachment text + tool evidence); misses get exactly one correction round (fix, drop, or mark as target/estimate), never a loop. runVerifier skips same-source model verification unless runtime.verifierSameModel is set (hard facts + fact check still run). Search tools now register lazily, reading tavily keys per call (settings/seeding take effect immediately). Evidence: SELF - TestHardFactsIgnoresKeywordDimensions, TestHardFactsKeepsLintUnavailableBlocking, TestUnverifiedFactsAndNotice, TestFactCheckContinuationOnce (fact note fires, corrected run passes); go vet + go test ./... pass; live before/after on the golden task set recorded on the phase05 task card per the policy clause. Impact: Gate stops rejecting valid documents on stylistic grounds; invented figures get one forced correction round; same-model verification no longer pretends to be independent; tavily config is hot. No contract or storage changes.
55 lines
1.6 KiB
Go
55 lines
1.6 KiB
Go
package quality
|
|
|
|
import (
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
// P5/裁定 4：拦截路径只认硬事实（lint 实际运行且零 error）；
|
|
// 关键词维度（栏目式标题/可执行性等）不得再把合格交付物拦下来。
|
|
func TestHardFactsIgnoresKeywordDimensions(t *testing.T) {
|
|
dir := t.TempDir()
|
|
// 评分整体会因「结论先行 20/可读性 25」判负的文本——硬事实口径必须放行
|
|
//（无 lint 或 lint 通过时）。
|
|
path := filepath.Join(dir, "报告.md")
|
|
if err := os.WriteFile(path, []byte("# 概述\n\n正文内容。\n"), 0o644); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
full := ScoreFile(path)
|
|
if full.Passed {
|
|
t.Fatalf("keyword dimensions should still fail the full score (fixture sanity): %+v", full)
|
|
}
|
|
hard := HardFacts("报告.md", []byte("# 概述\n\n正文内容。\n"))
|
|
if !hard.Passed {
|
|
t.Fatalf("hard facts must ignore keyword dimensions: %+v", hard.Violations)
|
|
}
|
|
if len(hard.Lint) > 0 {
|
|
for _, issue := range hard.Lint {
|
|
if issue.Severity == "error" {
|
|
t.Fatalf("lint errors must still block: %+v", issue)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// lint 不可运行（技能根已配置）仍是硬事实失败——fail-visible 保持。
|
|
func TestHardFactsKeepsLintUnavailableBlocking(t *testing.T) {
|
|
SetLintEnvRoot(t.TempDir())
|
|
t.Cleanup(func() { SetLintEnvRoot("") })
|
|
hard := HardFacts("报告.md", []byte("# 概述\n正文。\n"))
|
|
if hard.Passed {
|
|
t.Fatalf("lint-not-run must stay blocking: %+v", hard)
|
|
}
|
|
found := false
|
|
for _, v := range hard.Violations {
|
|
if strings.Contains(v, "自检未运行") {
|
|
found = true
|
|
}
|
|
}
|
|
if !found {
|
|
t.Fatalf("expected 自检未运行, got %v", hard.Violations)
|
|
}
|
|
}
|