diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index f1bd7df6..90c945af 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -5,7 +5,7 @@ ## 焦点 -1. **Wave 2 done**:已合入 `dev` + smoke/rebuild PASS;Wave 2B(nightly/receipt)另开 +1. **Wave 2B**:nightly PI + promotion receipt(`feature/wave2b-nightly-promotion-receipt`) 2. **Wave 1 done**:PU5A race + PU3 IAM fail-closed 已合入 `dev` 3. **运维**:夜间 CI `RUN_PI_E2E=1`;promotion receipt(Wave 2B) @@ -13,6 +13,7 @@ | owner | task | branch | worktree | mutex | status | |-------|------|--------|----------|-------|--------| +| Cursor | 2026-09-16-wave2b-nightly-promotion-receipt | feature/wave2b-nightly-promotion-receipt | - | ci | in_progress | | Cursor | 2026-09-16-wave2-real-pi-evidence | feature/wave2-real-pi-evidence | - | intent-plan | done | | Cursor | 2026-09-16-wave1-race-tempdir-cleanup | fix/race-tempdir-cleanup | - | ci | done | | Cursor | 2026-09-16-wave1-iam-fail-closed | fix/iam-store-fail-closed | - | authz | done | diff --git a/.doc/ops/short-term/2026-09-16-wave2b-nightly-promotion-receipt.md b/.doc/ops/short-term/2026-09-16-wave2b-nightly-promotion-receipt.md new file mode 100644 index 00000000..da94c53f --- /dev/null +++ b/.doc/ops/short-term/2026-09-16-wave2b-nightly-promotion-receipt.md @@ -0,0 +1,48 @@ +--- +title: "Wave 2B PU5B:nightly PI + promotion receipt" +status: in_progress +created: 2026-09-16 +updated: 2026-09-16 +owner: Cursor +branch: feature/wave2b-nightly-promotion-receipt +worktree: "-" +mutex: ci +--- + +# Wave 2B PU5B:nightly PI + promotion receipt + +## 目标 + +落地夜间真实 PI 回归入口、不可变 promotion receipt,以及晋升脚本强制校验;same-image digest 晋升本轮仅做到 same-commit + digest 可追溯。 + +## 范围 + +- promotion receipt 写/校验 + waiver +- nightly-real-pi.yml + run-nightly-real-pi.sh +- git-workflow-helper / promote-to-test / release-to-main 门禁钩子 +- 报告与 FEATURE-MATRIX / remediation 同步 + +## 非目标 + +- compose 按 digest pull 的同镜像晋升流水线 +- Playwright 双标签;升 MULTI_INTENT tier + +## 步骤 + +1. [x] 认领 + 开分支 +2. [x] receipt 写/校验脚本 +3. [x] nightly workflow + 本机脚本 +4. [x] 晋升钩子 +5. [ ] 报告 + 合入 dev + +## 验收标准 + +- [x] 缺 secret/runner → infra_fail,晋升拒绝 +- [x] product_fail → 晋升拒绝 +- [x] pass receipt → 晋升可过;无 receipt → 非零退出 +- [x] 报告归档(`.doc/test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md`) + +## 相关链接 + +- Remediation PU5B: `../../remediation/2026-09-16-feature-matrix-production-readiness-plan.md` +- Test report: `../../test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md` diff --git a/.doc/remediation/2026-09-16-feature-matrix-production-readiness-plan.md b/.doc/remediation/2026-09-16-feature-matrix-production-readiness-plan.md index 51f4e6bc..04097012 100644 --- a/.doc/remediation/2026-09-16-feature-matrix-production-readiness-plan.md +++ b/.doc/remediation/2026-09-16-feature-matrix-production-readiness-plan.md @@ -184,7 +184,7 @@ flowchart TB | PI change-report narrative | deterministic report 已有 | 产品 go/no-go;只读 narrative、fallback 永久保留 | 当前 `deferred`;获批 short-term 开始后转 `partial` | | Frontend workbench | 当前声明范围均有证据 | 修正文档状态;增强另列 | `done` | | CI / E2E | PR CI、PG、backup、web 已有 | nightly real PI;TempDir race flake | `partial` | -| Artifact promotion | 同 commit 可追溯,环境内仍可能重建 | 一次构建、digest 晋升、promotion receipt | 当前 `planned`;PU5B 开始后转 `partial` | +| Artifact promotion | 同 commit 可追溯,环境内仍可能重建;receipt + 晋升钩子已落地 | 一次构建、digest 晋升流水线未完成 | 当前 `partial`(Wave 2B);不得声称 same-image | | Reliable events | 核心 live 且旧 hook 退出 | 与 FAV 拆行 | `done` | | FAV | 明确 off | 独立审计与 go/no-go | `deferred` | @@ -414,7 +414,7 @@ flowchart TB - [x] PU4 的 multi-intent U8 与 full/none 真实 PI/browser/restart/cross-user 证据独立归档;失败未被文档改写掩盖。 (2026-09-16:Wave2 报告已签;Playwright 双标签仍 gated,API 刷新/重启已签;不升 tier) - [x] PU5A 的 TempDir race flake 被真正关闭;PU5B 的 nightly job、promotion receipt 和人工豁免链能区分 infrastructure/product failure。 - (2026-09-16:PU5A 已关闭;PU5B 仍开放) + (2026-09-16:PU5A 已关闭;PU5B **nightly+receipt+晋升钩子**已交付——见 [Wave2B 报告](../test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md)。**同镜像 digest 晋升**未完成,矩阵 Artifact 行保持 partial。) - [ ] PU6–PU8 已分别完成产品 go/no-go;未获批项保持 deferred,获批项另建 short-term 后才进入实现与 rollout DoD。 - [ ] 任何真实 PI 工作均满足 R13 的最小出站、租户绑定、脱敏、凭证、留存和泄漏负例。 - [ ] deterministic summary/change-report、legacy 单意图、B1–B4 和三环境隔离均无回归。 diff --git a/.doc/status/FEATURE-MATRIX.md b/.doc/status/FEATURE-MATRIX.md index 1e6f0f7a..6e75e8b8 100644 --- a/.doc/status/FEATURE-MATRIX.md +++ b/.doc/status/FEATURE-MATRIX.md @@ -63,8 +63,8 @@ | 前端工作台 | 三栏 + 块文档 + SSE;错误卡/进度/busy;版本冲突信封;demo seed 默认关;终止 preview | done | non_blocker | [workbench](../test-reports/2026-09-08-three-column-workbench-and-block-document-test-report.md);[双标签](../test-reports/assets/2026-09-15-conversation-intent-dual-tab/screenshots/tab-b-busy.png);[demo](../test-reports/2026-09-16-demo-seed-pollution-test-report.md);[B4](../test-reports/2026-09-16-frontend-gate-b4-test-report.md)(项目 6363) | **before**:空「缺口」却 partial → **after**:本行 done。其他 409 消费方信封复用另列 hardening backlog | `web` | | PI Runner / 配置链 | pi-agent + MiniMax + typed-result-v1 | done | non_blocker | [piagent](../test-reports/2026-09-13-codex-audit-p0-remediation-test-report.md);[container-env](../test-reports/2026-09-13-container-env-configuration-test-report.md);[typed-result](../test-reports/2026-09-15-pi-typed-result-v1-test-report.md) | — | `pi-agent` | | 三环境运维 | dev/test/main + 共享 KB;对话/业务库隔离 | done | non_blocker | [PROJECT.md](../../PROJECT.md) §4;CURRENT 环境快照 schema **47** | — | `infra` | -| CI / E2E | PR CI、PG 集成、治理 PG、已签真实 PI 门禁 | partial | 加固(阻止**下一次**晋升,不自动否定已发布) | [BUG-08](../test-reports/2026-09-16-ci-governance-pg-bug08-test-report.md);[PI 三工作流](../test-reports/2026-09-16-pi-three-workflow-e2e-test-report.md);[B3](../test-reports/2026-09-16-chat-refresh-gate-test-report.md);[B4](../test-reports/2026-09-16-frontend-gate-b4-test-report.md) | 夜间 `RUN_PI_E2E=1` + 失败分类;TempDir race **已修**([PU5A 报告](../test-reports/2026-09-16-wave1-race-tempdir-cleanup-test-report.md));PR CI 不依赖外部模型 | `ci` | -| Artifact 同 digest 晋升 | 候选 commit 一次构建、按 digest 晋升、promotion receipt | planned | 加固 | [计划 PU5B](../remediation/2026-09-16-feature-matrix-production-readiness-plan.md) | 落地前只可声称 same-commit + 各环境 digest 可追溯,不得声称 same-image promotion | `infra` | +| CI / E2E | PR CI、PG 集成、治理 PG、已签真实 PI 门禁;**nightly** schedule/`workflow_dispatch` + infra/product 分类 receipt | partial | 加固(阻止**下一次**晋升,不自动否定已发布) | [BUG-08](../test-reports/2026-09-16-ci-governance-pg-bug08-test-report.md);[PI 三工作流](../test-reports/2026-09-16-pi-three-workflow-e2e-test-report.md);[B3](../test-reports/2026-09-16-chat-refresh-gate-test-report.md);[B4](../test-reports/2026-09-16-frontend-gate-b4-test-report.md);[Wave2B nightly/receipt](../test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md) | 关闭条件:nightly 入口 + 失败分类 + promotion-receipt 门禁已交付;TempDir race **已修**;PR CI 不依赖外部模型。Actions secret/runner 需环境配置 | `ci` | +| Artifact 同 digest 晋升 | 候选 commit 一次构建、按 digest 晋升、promotion receipt | partial | 加固 | [计划 PU5B](../remediation/2026-09-16-feature-matrix-production-readiness-plan.md);[Wave2B 报告](../test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md) | **本轮**:same-commit + receipt 内 `image_digests` 可追溯 + 晋升钩子校验 PASS/waiver。**未完成**:compose 按 digest pull 的 same-image 流水线 — 不得声称 same-image promotion | `infra` | | 可靠事件核心 | PG dispatcher、notify/advance/projection live、生产闭包 hook 退出、Phase2 选择性解耦已合入 | done | non_blocker | [E0A–E2](../test-reports/2026-09-15-event-e0a-production-scope-contract-test-report.md) 等;[复盘](../ops/research/2026-09-15-track-e-retrospective.md);[shared live](../test-reports/2026-09-15-event-notify-shared-live-test-report.md);[phase2 closeout](../test-reports/2026-09-15-phase2-closeout-test-report.md) | **before**:与 FAV off 混成 partial → **after**:FAV 独立行。选择性后续解耦按收益触发 | `architecture-docs` / `events` | | Final Artifact Vault(FAV) | 不可变终态制品库;`FAV_DEGRADED_READ=false` | deferred | deferred_enhancement | [Track E 复盘](../ops/research/2026-09-15-track-e-retrospective.md);CURRENT 旁支纪律 | 独立审计与 go/no-go;**禁止**整支合入 `feature/final-artifact-vault` | `events` | diff --git a/.doc/test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md b/.doc/test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md new file mode 100644 index 00000000..a0fcb8a9 --- /dev/null +++ b/.doc/test-reports/2026-09-16-wave2b-nightly-promotion-receipt-test-report.md @@ -0,0 +1,105 @@ +# Wave 2B PU5B:Nightly Real PI + Promotion Receipt 测试报告 + +## 一、 测试背景 (Background) + +Wave 2(PU4)已合入 `dev` 并补齐 U8 / full-none 真实 PI 证据。发布加固缺口仍在:PR CI 不跑真实 PI;晋升入口缺少不可变 receipt;基础设施失败易被误记为 PASS。本轮落地夜间入口、receipt 写/校验(含 timed waiver)与 `promote-to-test` / `release-to-main` 门禁钩子。**不**声称 same-image digest 晋升已完成。 + +## 二、 测试目标 (Test Objectives) + +- 缺 secret / runner 不可达 → `infra_fail` receipt,job/脚本非零退出(禁止写成 PASS) +- 产品断言失败 → `product_fail`,晋升拒绝 +- PASS → 落盘 receipt;无 receipt / 过期 waiver → `promotion-receipt-check` 非零退出 +- workflow `concurrency` cancel-in-progress;PR `ci.yml` 仍不跑真实 PI +- Artifact 矩阵行保持 **partial**(same-commit + digest 可追溯) + +## 三、 测试方案与环境 (Test Plan & Environment) + +| 项 | 值 | +|----|-----| +| OS | Linux 7.0.0-31-generic | +| 分支 | `feature/wave2b-nightly-promotion-receipt` | +| Runner | `http://127.0.0.1:18865`(MiniMax-M2.7-highspeed) | +| 隔离 PG | `127.0.0.1:5434/agent_wave2_pi` | +| 脚本 | `scripts/run-nightly-real-pi.sh`、`promotion-receipt-{write,check}.sh`、`promote-to-test.sh`、`release-to-main.sh` | +| Workflow | `.github/workflows/nightly-real-pi.yml`(schedule + workflow_dispatch) | +| Receipt 目录 | `.doc/test-reports/assets/promotion-receipts/` | + +用例矩阵: + +| ID | 场景 | 期望 | +|----|------|------| +| N1 | `--dry-infra-fail` / 缺 secret | `verdict=infra_fail`,exit 1 | +| N2 | fixture PASS commit | check OK | +| N3 | fixture product_fail | check BLOCKED | +| N4 | fixture infra_fail 无 waiver | check BLOCKED | +| N5 | infra_fail + 有效 waiver 绑定 PASS | check OK | +| N6 | 无 receipt commit | check BLOCKED | +| N7 | 本机真实 suite(可选墙钟) | pass / product_fail / infra_fail 分类落盘 | + +## 四、 预期结果 (Expected Results) + +- Receipt 最小字段:`verdict`、`commit`、`recorded_at`、`runner_url`、`model_id`、`protocol`、`image_digests`、`suite`、`failure_class`、`evidence_refs` +- Waiver:`owner`、`reason`、`expires_at`、`binds_receipt`(须指向 PASS) +- `git-workflow-helper promote-to-test` 校验 **dev HEAD**;`release-to-main` 校验 **test HEAD** + +## 五、 实际测试结果与判定 (Actual Results & Evaluation) + +### N1 — infra 负例(本机 dry) + +- **测试输入 (Input)**:`./scripts/run-nightly-real-pi.sh --dry-infra-fail` +- **实际输出 (Actual Output)**: + +```text +==> nightly real-PI suite commit=82780222373d runner=http://127.0.0.1:18865 +forced infra failure (--dry-infra-fail) +wrote promotion receipt: .../82780222373de1ded853e3fe5a8f2208133bc7c7.json +wrote infra_fail receipt (dry) +``` + +- **评价判定 (Evaluation / Verdict)**:PASS(`verdict=infra_fail`,exit 1;非绿) +- **性能与耗时 (Duration & Latency)**:<1s + +对应 receipt 摘录见 assets;晋升门禁对该 commit: + +```text +promotion-receipt-check: BLOCKED — infra_fail receipt freezes promotion (...); provide timed waiver or re-run nightly +``` + +### N2–N6 — fixture 门禁 + +| 用例 | commit fixture | 实际 | 判定 | +|------|----------------|------|------| +| N2 PASS | `aaaa…` | `OK — PASS receipt` | PASS | +| N3 product | `cccc…` | `BLOCKED — product_fail` | PASS | +| N4 infra | `dddd…` | `BLOCKED — infra_fail … freeze` | PASS | +| N5 waiver | `dddd…` + waiver→`eeee…` PASS | `OK — audited waiver … until 2099-01-01` | PASS | +| N6 missing | `bbbb…` | `BLOCKED — missing promotion receipt` | PASS | + +### N7 — 本机真实 suite + +- **测试输入**:`TEST_DATABASE_URL=…/agent_wave2_pi` + `RUNNER_AUTH_TOKEN` + `./scripts/run-nightly-real-pi.sh` +- **说明**:墙钟取决于 PI;结果与日志落入 `promotion-receipts/evidence--/` 与 `.json`。若 Actions 缺 secret,workflow 同样写 `infra_fail` 并 fail(不得 skip 冒充绿)。 +- **评价判定**:以当次 receipt `verdict` 为准;本报告以 N1–N6 关闭脚本/门禁契约,真实 suite 由合入后夜间/手动 dispatch 持续执行。 + +## 六、 自动化测试套件汇总 (Test Suite Execution) + +合入冒烟(清污染环境变量): + +```bash +unset AGENT_TIMEOUT KB_DATABASE_URL KB_EMBEDDING_DIM +go vet ./... +go test -short ./... +``` + +PR CI 不变;新增 nightly workflow 不进入 PR 路径。 + +## 七、 测试结论与后续优化建议 (Conclusion & Next Steps) + +- **结论**:Wave 2B 脚本门禁与 nightly 入口已交付;infra/product 分类与 waiver 链可审计;晋升钩子在无 PASS receipt 时非零退出。 +- **明确未完成**:compose 按 digest pull 的 **same-image** 晋升流水线仍未做 → FEATURE-MATRIX「Artifact 同 digest 晋升」保持 **partial**。 +- **后续**:配置 Actions `RUNNER_AUTH_TOKEN` + `NIGHTLY_TEST_DATABASE_URL`(或自托管 runner);Wave 3 候选(Playwright 双标签 / tier)。 + +## 证据路径 + +- Receipts / fixtures:`.doc/test-reports/assets/promotion-receipts/` +- Short-term:`.doc/ops/short-term/2026-09-16-wave2b-nightly-promotion-receipt.md`