Files

T

weichenandCursor 4187bcfc9e docs(ops): release Wave 2B integration slot and archive short-term

Smoke passed on merge; record N7 product-fail classification evidence
from the in-flight nightly suite without claiming same-image promotion.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-09-16 14:32:10 +00:00

5.7 KiB

Raw Blame History

Wave 2B PU5B：Nightly Real PI + Promotion Receipt 测试报告

一、测试背景 (Background)

Wave 2（PU4）已合入 dev 并补齐 U8 / full-none 真实 PI 证据。发布加固缺口仍在：PR CI 不跑真实 PI；晋升入口缺少不可变 receipt；基础设施失败易被误记为 PASS。本轮落地夜间入口、receipt 写/校验（含 timed waiver）与 promote-to-test / release-to-main 门禁钩子。不声称 same-image digest 晋升已完成。

二、测试目标 (Test Objectives)

缺 secret / runner 不可达 → infra_fail receipt，job/脚本非零退出（禁止写成 PASS）
产品断言失败 → product_fail，晋升拒绝
PASS → 落盘 receipt；无 receipt / 过期 waiver → promotion-receipt-check 非零退出
workflow concurrency cancel-in-progress；PR ci.yml 仍不跑真实 PI
Artifact 矩阵行保持 partial（same-commit + digest 可追溯）

三、测试方案与环境 (Test Plan & Environment)

项	值
OS	Linux 7.0.0-31-generic
分支	`feature/wave2b-nightly-promotion-receipt`
Runner	`http://127.0.0.1:18865`（MiniMax-M2.7-highspeed）
隔离 PG	`127.0.0.1:5434/agent_wave2_pi`
脚本	`scripts/run-nightly-real-pi.sh`、`promotion-receipt-{write,check}.sh`、`promote-to-test.sh`、`release-to-main.sh`
Workflow	`.github/workflows/nightly-real-pi.yml`（schedule + workflow_dispatch）
Receipt 目录	`.doc/test-reports/assets/promotion-receipts/`

用例矩阵：

ID	场景	期望
N1	`--dry-infra-fail` / 缺 secret	`verdict=infra_fail`，exit 1
N2	fixture PASS commit	check OK
N3	fixture product_fail	check BLOCKED
N4	fixture infra_fail 无 waiver	check BLOCKED
N5	infra_fail + 有效 waiver 绑定 PASS	check OK
N6	无 receipt commit	check BLOCKED
N7	本机真实 suite（可选墙钟）	pass / product_fail / infra_fail 分类落盘

四、预期结果 (Expected Results)

Receipt 最小字段：verdict、commit、recorded_at、runner_url、model_id、protocol、image_digests、suite、failure_class、evidence_refs
Waiver：owner、reason、expires_at、binds_receipt（须指向 PASS）
git-workflow-helper promote-to-test 校验 dev HEAD；release-to-main 校验 test HEAD

五、实际测试结果与判定 (Actual Results & Evaluation)

N1 — infra 负例（本机 dry）

测试输入 (Input)：./scripts/run-nightly-real-pi.sh --dry-infra-fail
实际输出 (Actual Output)：

==> nightly real-PI suite commit=82780222373d runner=http://127.0.0.1:18865
forced infra failure (--dry-infra-fail)
wrote promotion receipt: .../82780222373de1ded853e3fe5a8f2208133bc7c7.json
wrote infra_fail receipt (dry)

评价判定 (Evaluation / Verdict)：PASS（verdict=infra_fail，exit 1；非绿）
性能与耗时 (Duration & Latency)：<1s

对应 receipt 摘录见 assets；晋升门禁对该 commit：

promotion-receipt-check: BLOCKED — infra_fail receipt freezes promotion (...); provide timed waiver or re-run nightly

N2–N6 — fixture 门禁

用例	commit fixture	实际	判定
N2 PASS	`aaaa…`	`OK — PASS receipt`	PASS
N3 product	`cccc…`	`BLOCKED — product_fail`	PASS
N4 infra	`dddd…`	`BLOCKED — infra_fail … freeze`	PASS
N5 waiver	`dddd…` + waiver→`eeee…` PASS	`OK — audited waiver … until 2099-01-01`	PASS
N6 missing	`bbbb…`	`BLOCKED — missing promotion receipt`	PASS

N7 — 本机真实 suite（合入后墙钟跑）

测试输入：TEST_DATABASE_URL=…/agent_wave2_pi + RUNNER_AUTH_TOKEN + ./scripts/run-nightly-real-pi.sh（commit 67d73359fd13）
实际输出 (Actual Output)（摘录）：

=== RUN   TestMultiIntentU8_RealPI_ConfirmGenerate_PG
    ... PI run status=quality_failed want completed
--- FAIL: TestMultiIntentU8_RealPI_ConfirmGenerate_PG (352.98s)
=== RUN   TestWorkflowGovernance_PiAgent_PG_FullNoneMixedChain
    ✓ [1. activate] ... ✓ [2. full intake] ... ✓ [3. advance V0.1] ok

评价判定 (Evaluation / Verdict)：U8 段已出现产品侧 quality_failed（非 secret/runner 不可达）。整包结束后脚本应落 product_fail receipt 并 exit 1——符合「产品失败拒绝晋升、不得写成 PASS」；full/none 段仍继续跑作对照。控制台：assets/2026-09-16-wave2b-nightly-promotion-receipt/nightly-console.txt
性能与耗时：U8 FAIL ≈ 353s；full/none 长链另行计时
合入冒烟：go vet PASS；洁净环境 go test -short ./... PASS；/health schema 47 ok

六、自动化测试套件汇总 (Test Suite Execution)

合入冒烟（清污染环境变量）：

unset AGENT_TIMEOUT KB_DATABASE_URL KB_EMBEDDING_DIM
go vet ./...
go test -short ./...

PR CI 不变；新增 nightly workflow 不进入 PR 路径。

七、测试结论与后续优化建议 (Conclusion & Next Steps)

结论：Wave 2B 脚本门禁与 nightly 入口已交付；infra/product 分类与 waiver 链可审计；晋升钩子在无 PASS receipt 时非零退出。
明确未完成：compose 按 digest pull 的 same-image 晋升流水线仍未做 → FEATURE-MATRIX「Artifact 同 digest 晋升」保持 partial。
后续：配置 Actions RUNNER_AUTH_TOKEN + NIGHTLY_TEST_DATABASE_URL（或自托管 runner）；Wave 3 候选（Playwright 双标签 / tier）。

证据路径

Receipts / fixtures：.doc/test-reports/assets/promotion-receipts/
Short-term：.doc/ops/short-term/2026-09-16-wave2b-nightly-promotion-receipt.md

5.7 KiB Raw Blame History Unescape Escape

Wave 2B PU5B：Nightly Real PI + Promotion Receipt 测试报告

一、 测试背景 (Background)

二、 测试目标 (Test Objectives)

三、 测试方案与环境 (Test Plan & Environment)

四、 预期结果 (Expected Results)

五、 实际测试结果与判定 (Actual Results & Evaluation)