Add the multi-turn eval format and self-describing corpora (I1 WP6) #8

Merged
weichen merged 1 commits from feature/agent-stack-i1 into main 2026-10-07 08:22:33 +00:00
1 Commits
Author SHA1 Message Date
weichen 1d0fcbd1ba Add the multi-turn eval format and self-describing corpora
ci / vet-and-test (pull_request) Successful in 18s
ci / docker-checkout-hostgateway (pull_request) Successful in 0s
Why: I1 WP6 - eval needs per-turn command expectations to carry the
SetSlot+Start acceptance case, and corpora must reproduce standalone
in intentctl (the guard case has been red there since before I1
because the guard spec only existed as a test builtin).

Evidence: go vet clean; go test ./... all ok; intentctl eval passes
11/11 on the v1 corpus (guard case fixed via self-declaration) and
1/1 on the new two-turn SetSlot+Start corpus.

Impact: new intent/corpus.go (CorpusCase/CorpusTurn/CommandExpect,
RunCorpusCase with chained expectations); intentctl eval consumes
turns and corpus-declared builtin_specs and domains; report.generate
spec and corpus_multiturn.json fixtures added. v1 corpus format and
conformance unchanged.

Refs: Documents proposal 2026-10-07-agent-stack-n1q1i1-plans.md I1 WP6
2026-10-07 08:12:50 +00:00