From 2269cabf4353314d8b05c3ea8992fb3345960959 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 13:43:54 +0000 Subject: [PATCH 01/11] docs(arch): archive document reliability pilot plan and claim docgov MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Land the revised modular-monolith scheme on the implementation branch so U1–U4 can proceed against a frozen design, and register the docgov mutex without occupying the integration slot. Co-authored-by: Cursor --- .../2026-09-14-modular-monolith-decoupling.md | 62 ++ ...9-14-module-decoupling-remediation-plan.md | 55 ++ ...026-09-14-module-decoupling-plan-review.md | 98 +++ .doc/audit/INDEX.md | 8 + .doc/ops/CURRENT.md | 10 +- ...4-module-decoupling-refactor-acceptance.md | 74 ++ .../2026-09-14-selective-module-decoupling.md | 47 ++ .../2026-09-14-document-reliability-pilot.md | 61 ++ ...026-09-14-durable-event-fav-integration.md | 60 ++ ...6-09-14-module-decoupling-refactor-plan.md | 677 ++++++++++++++++++ .doc/remediation/INDEX.md | 1 + .doc/remediation/gap-matrix.md | 1 + .doc/status/FEATURE-MATRIX.md | 1 + ...26-09-14-module-decoupling-architecture.md | 422 +++++++++++ .doc/system-design/README.md | 21 + 15 files changed, 1596 insertions(+), 2 deletions(-) create mode 100644 .doc/adr/2026-09-14-modular-monolith-decoupling.md create mode 100644 .doc/archive/ops/2026-09-14-module-decoupling-remediation-plan.md create mode 100644 .doc/audit/2026-09-14-module-decoupling-plan-review.md create mode 100644 .doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md create mode 100644 .doc/ops/long-term/2026-09-14-selective-module-decoupling.md create mode 100644 .doc/ops/short-term/2026-09-14-document-reliability-pilot.md create mode 100644 .doc/ops/short-term/2026-09-14-durable-event-fav-integration.md create mode 100644 .doc/remediation/2026-09-14-module-decoupling-refactor-plan.md create mode 100644 .doc/system-design/2026-09-14-module-decoupling-architecture.md diff --git a/.doc/adr/2026-09-14-modular-monolith-decoupling.md b/.doc/adr/2026-09-14-modular-monolith-decoupling.md new file mode 100644 index 00000000..d337fd64 --- /dev/null +++ b/.doc/adr/2026-09-14-modular-monolith-decoupling.md @@ -0,0 +1,62 @@ +# ADR：以纵向绞杀方式实施模块化单体解耦 + +## 状态 + +提议（2026-09-14；架构与实施计划已修订，等待用户确认后进入首个实现里程碑） + +## 背景 + +`internal/server`、`internal/bizdev`、共享 `UserProjectContext`、工作区路径和多表写入形成隐式耦合。 +历史受控正文 404/409、刷新后投影缺失和任务终态不完整说明，单纯搬包不会修复事实源、事务与恢复边界。 + +## 决策 + +1. **保持模块化单体**：不拆微服务；进程内调用使用窄端口，跨进程可靠性只用于持久事件和 FAV。 +2. **纵向绞杀**：先在现有包边界插入端口和适配器，迁完一条业务链并验证后再移动或重命名包；禁止“先全量搬包、以后再接口化”。 +3. **契约分层**：外部 API DTO 归 `server/api`；持久 schema、公共事件和稳定共享词汇进入按领域划分的 contracts；一对一内部接口由消费者拥有。 +4. **命令与事务同属一个应用用例**：一个业务命令只有一个事务 owner；跨表属主通过同一不透明事务作用域协作，业务状态、账本、当前指针、staging 和 outbox 原子提交。 +5. **单一写者不等于单独事务**：表属主控制 SQL 和不变量；应用 UnitOfWork 控制跨属主提交边界。属主端口不得私自嵌套提交。 +6. **并发正确性以数据库为准**：用户编辑使用乐观版本/ETag；状态转换使用行锁和条件更新；项目级不变量可使用事务级 advisory lock；进程内 mutex 只能优化，不能作为多副本正确性依据。 +7. **锁顺序固定**：同一事务按 project/workflow instance → document record → document version/snapshot → task run → outbox/staging 的顺序取锁；事务内禁止 LLM、对象存储、网络调用和长文件 I/O。 +8. **可靠事件采用 transactional outbox**：事件按至少一次投递设计，具有稳定 ID、schema version、环境/项目范围、聚合序号、因果链和幂等键；逐消费者记录 delivery、重试、租约和死信。 +9. **工作区是可清理投影**：跨模块只传不可伪造 Workspace ID 和受限 artifact key；物理路径只在 workspace/squadrun 适配器内解析和校验。 +10. **最终成果进入 FAV**：PostgreSQL 保存业务关系和授权，FAV 只保存已确认终态的精确 payload、manifest、回执和删除状态，workspace 不承担最终成果事实源。 +11. **授权逐动作校验**：认证适配器构造不可由请求覆盖的 Principal；HTTP、CLI、后台订阅者和恢复流程都按 action/resource/current state fail closed。 +12. **先完成文档可靠性试点**：只有受控正文刷新、重启、并发提交、投影缺失降级和授权负例通过后，才批准 chat、squad、编排层和传输层扩展。 +13. **迁移采用 expand/migrate/contract**：允许新增 Goose migration；不得修改已发布迁移；回滚只保证回到最近的兼容阶段，不能任意独立 revert 早期依赖。 +14. **协议改造独立立项**：统一错误信封和 PI Runner `result.json` 不作为核心模块解耦前置条件；它们使用独立兼容与 E2E 验收。 + +## 并发与失败语义 + +- 普通聚合状态转换默认 `READ COMMITTED` + 明确行锁/条件更新；涉及跨聚合唯一不变量时才使用 Serializable，并对 `40001`/`40P01` 重跑完整事务。 +- outbox worker 使用小批次 `FOR UPDATE SKIP LOCKED` 认领，处理发生在事务外；完成、续租和失败回填使用 fencing token 防止过期 worker 覆盖新 owner。 +- 每个写命令必须同时有 idempotency key、expected version 或两者；相同 key 不同 payload 返回 typed conflict。 +- 锁等待、事务时长、重试次数、队列积压和死信必须可观测；超过预算时失败返回，不无限等待。 + +## 授权与恢复语义 + +- Principal 至少携带 actor、环境、认证来源和请求关联 ID；项目、会话、文档、阶段与 duty 从数据库权威关系重新解析。 +- 持久事件中的 actor 只用于审计，不能替代订阅者对当前资源状态的授权检查。 +- FAV 降级读取要求运行库授权图完整；授权图缺失时禁止面向用户返回内容。 +- 恢复只允许进入 scratch identity；promotion 需要系统管理员与数据责任人确认，并通过各属主端口提交，禁止恢复 CLI 直写运行表。 + +## 后果 + +- 首个实现里程碑的改动面小于原 Wave 1–6，但会先增加 UnitOfWork、事件投递和授权基础设施。 +- “单表唯一写者”检查改为生产 SQL 写权限检查，并允许 migration、测试和经批准运维脚本存在表名。 +- `feature/final-artifact-vault` 是候选依赖，不直接假定可合入；必须先按当前 dev 重放测试、迁移和权限审计。 +- 包物理移动、错误信封、SSE 和 PI Runner 协议可延后，不阻塞文档一致性整改。 + +## 替代方案 + +- **一次性横向搬包**：拒绝。冲突面大，不能先证明用户故障减少。 +- **仅增加 arch-check**：拒绝。门禁不能替代事务、锁和授权契约。 +- **拆微服务**：拒绝。当前问题是模块边界和数据一致性，不需要引入分布式事务与跨服务部署成本。 +- **继续把 workspace 作为恢复来源**：拒绝。工作区属于可清理计算现场，不能成为最终事实源。 + +## 相关文档 + +- [模块解耦架构](../system-design/2026-09-14-module-decoupling-architecture.md) +- [模块解耦整改与重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md) +- [方案复核](../audit/2026-09-14-module-decoupling-plan-review.md) +- [文档版本边界 ADR](./2026-09-13-document-version-boundary.md) diff --git a/.doc/archive/ops/2026-09-14-module-decoupling-remediation-plan.md b/.doc/archive/ops/2026-09-14-module-decoupling-remediation-plan.md new file mode 100644 index 00000000..2ba98905 --- /dev/null +++ b/.doc/archive/ops/2026-09-14-module-decoupling-remediation-plan.md @@ -0,0 +1,55 @@ +--- +title: "模块解耦架构与重构计划修订" +status: done +created: 2026-09-14 +updated: 2026-09-14 +owner: Codex +branch: feature/module-decoupling-remediation +worktree: .worktrees/feature/module-decoupling-remediation +mutex: architecture-docs +--- + +# 模块解耦架构与重构计划修订 + +## 目标 + +把模块解耦方案修订为可以分段实施、验证和回滚的架构与重构计划，补齐事务、并发互斥、授权、可靠事件、FAV、数据迁移和真实页面验收。 + +## 范围 + +- 修订模块化单体架构设计与 ADR。 +- 新增详细整改与重构计划。 +- 建立首个文档可靠性纵向试点及后续里程碑任务。 +- 更新 CURRENT、FEATURE-MATRIX 和相关索引。 + +## 非目标 + +- 本任务不修改业务代码、数据库迁移或部署配置。 +- 本任务不合并 `feature/final-artifact-vault`，只定义其集成依赖和边界。 +- 本任务不执行共享容器验证，也不占用集成槽位。 + +## 步骤 + +1. [x] 对照当前源码、已接受 ADR、FAV 分支和外部权威资料复核原方案。 +2. [x] 修订架构设计与 ADR。 +3. [x] 编写详细整改计划、里程碑任务和验收清单。 +4. [x] 更新协作 SSOT 与索引并完成文档一致性检查。 + +## 验收标准 + +- [x] 计划明确事务所有权、锁顺序、事件投递语义、授权主体和恢复边界。 +- [x] 首个独立交付边界直接覆盖受控文档刷新、重启、投影缺失和授权负例。 +- [x] 每个后续里程碑有依赖、文件范围、并发风险、测试矩阵、停走门和回滚边界。 +- [x] FAV、迁移语料库和完整备份的职责不重叠，工作区不再承担最终成果事实源。 + +## 相关链接 + +- Checklist: [../../ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md](../../ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md) +- Handoff: 不需要；实现阶段使用两个 planned short-term 接续 +- Test report: 本任务仅产出计划，不生成业务验收报告 + +## 证据 / 备注 + +- 原方案位于主工作区未跟踪文件；本任务在独立 worktree 重建并修订，未覆盖原工作区。 +- `feature/final-artifact-vault` 已有实现与测试证据，但尚未合入本任务基线，实施前必须重新审计分支差异。 +- 文档校验：本地相对链接、R/KTD/U ID 唯一性、Mermaid fence 和 `git diff --check` 均通过。 diff --git a/.doc/audit/2026-09-14-module-decoupling-plan-review.md b/.doc/audit/2026-09-14-module-decoupling-plan-review.md new file mode 100644 index 00000000..d7dcbd64 --- /dev/null +++ b/.doc/audit/2026-09-14-module-decoupling-plan-review.md @@ -0,0 +1,98 @@ +--- +title: "模块解耦方案复核" +status: done +created: 2026-09-14 +updated: 2026-09-14 +scope: architecture, transactions, concurrency, authorization, events, artifact recovery +--- + +# 模块解耦方案复核 + +## 结论 + +模块化单体、单一写者、瘦传输层和应用编排层的方向成立，但原方案不能直接批准 Wave 1–6。 +原方案把事务、事件、工作区和最终成果四条边界拆开设计，可能在包依赖减少后产生新的数据不一致、跨租户授权旁路和不可恢复投影。 +整改应先打通一条真实文档纵向链路，再按证据决定是否扩展到 chat、squad、SSE 和 PI Runner。 + +## 已确认问题 + +### A1. 单一写者拆分会破坏跨表原子性 + +`internal/docrevision/pg_mutate.go` 当前在一个事务中写版本账本并投影 `document_records` 根指针。 +原方案要求 docrevision 改调 docgov/freedoc 普通端口，却没有定义跨属主事务或命令所有权。 +如果各端口各自提交，会出现版本账本、正文 head、治理状态和 outbox 部分成功。 + +### A2. 现有 outbox 不是可泛化的可靠事件总线 + +`internal/docrevision/service.go` 在版本事务完成后调用 `OnOutbox`，并忽略回调错误。 +`outbox_events` 只有全局 `published_at`，没有逐消费者确认、租约、重试、死信和幂等收件箱。 +拟议的 `Publish(ctx, event)` 也没有事务作用域，不能兑现“与业务同事务”。 + +### A3. 工作区仍被当成事实源 + +原 `WorkspaceRef` 暴露 `Root`、`OutputDir` 和 `ManifestPath`，`OutputContract` 继续接受裸路径。 +这不能从类型上阻止路径拼接、跨工作区访问或链接逃逸。 +docquery 若继续在运行时从 `output.md` 选择正文，会保留导致历史 404/409 的 split-brain 根因。 + +### A4. 最终成果保险库未进入模块边界 + +`feature/final-artifact-vault` 已设计并实现终态归档、只读降级和 scratch 恢复，但尚未合入当前 dev 基线。 +原解耦方案没有为 artifactvault 分配端口、事件、授权和恢复边界，后续容易由 docquery 或 CLI 重建旁路。 + +### A5. contracts 可能成为新的高扇入万能包 + +原方案把内部接口、HTTP DTO、事件、错误码和共享枚举集中到一个包，并要求所有 DTO 版本化。 +这会把语义耦合搬到 `contracts` 和 `app`，使普通内部调整承担不必要的 ADR 成本。 + +### A6. 并发控制只覆盖了进程内互斥 + +当前代码同时存在 `sync.Mutex`、行锁、乐观 ETag、唯一幂等键和 `pg_advisory_xact_lock`。 +原方案没有规定哪些锁用于正确性、锁顺序、超时、重试边界和多副本行为。 +进程内锁只能优化单实例竞争，不能作为跨实例正确性条件。 + +### A7. 授权主体和后台执行边界缺失 + +“在应用层入口统一授权”没有定义可信 Principal 的来源，也没有覆盖 CLI、事件订阅者、恢复工具和后台任务。 +若端口继续接受调用方填写的 `UserID`、`ProjectID` 或裸 `DocRef`，可能形成 confused-deputy 或跨项目访问。 + +### A8. 横向 Wave 缺少最小可交付边界 + +原方案先物理搬包，再接口化，最后才处理单写者和事件语义。 +这会重复修改高扇出代码，并在真正验证边界之前冻结大量无消费者契约。 +错误信封和 PI Runner 协议与文档一致性解耦没有直接依赖，不应阻塞核心路线。 + +### A9. 迁移与回滚承诺不成立 + +可靠多消费者 outbox、FAV 集成和必要的状态字段需要增量 migration，因此“数据零迁移”不成立。 +后续里程碑会依赖前序契约，不能任意独立 revert；只能按 expand/migrate/contract 阶段和兼容窗口回退。 + +### A10. 架构检查范围会产生误报 + +表名和路径字面量合法存在于 Goose migration、fixture、测试和运维脚本。 +R5/R6 应检查生产运行时的越权写入和路径解析，不应以全仓 grep 的零出现作为完成条件。 + +## 外部规则核对 + +- PostgreSQL 行锁和事务级 advisory lock 会在事务结束时释放，适合短期业务互斥；session 级 advisory lock 不跟随回滚，不用于本方案的业务锁。[PostgreSQL explicit locking](https://www.postgresql.org/docs/17/explicit-locking.html) +- `SKIP LOCKED` 适合多消费者队列表，不适合作为普通一致性查询语义。[PostgreSQL SELECT](https://www.postgresql.org/docs/current/sql-select.html) +- Serializable 或 deadlock 重试必须重跑完整事务，包括决定 SQL 和参数的业务逻辑。[PostgreSQL serialization failure handling](https://www.postgresql.org/docs/16/mvcc-serialization-failure-handling.html) +- 授权必须默认拒绝，并在每次请求或后台动作上重新校验，不能信任客户端传入的身份和资源归属。[OWASP Authorization Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) + +## 整改结论 + +1. 暂停批准原 Wave 1–6，保留只读架构基线。 +2. 先定义事务、锁、授权、事件和持久成果契约。 +3. 首个交付只覆盖受控文档读取与终态提交纵向链。 +4. FAV、迁移语料和完整备份作为三个互补层接入。 +5. 通过真实页面刷新、重启、并发、权限和恢复停走门后，再批准其他模块拆分。 + +## 证据入口 + +- `internal/docrevision/pg_mutate.go` +- `internal/docrevision/service.go` +- `internal/workspace/resolver.go` +- `internal/docduty/revoke.go` +- `internal/taskrun/manager.go` +- `deploy/postgres/goose/00018_2026-09-12-outbox-notifications-stage-advance.sql` +- `.doc/adr/2026-09-13-document-version-boundary.md` +- `feature/final-artifact-vault` 分支中的 FAV ADR、整改方案与测试报告 diff --git a/.doc/audit/INDEX.md b/.doc/audit/INDEX.md index ab5873db..5e5bc69c 100644 --- a/.doc/audit/INDEX.md +++ b/.doc/audit/INDEX.md @@ -13,6 +13,14 @@ ## 核心文档（按阅读顺序推荐） +### 模块解耦方案复核（2026-09-14） + +**文件**: [2026-09-14-module-decoupling-plan-review.md](./2026-09-14-module-decoupling-plan-review.md) + +**结论**: 模块化单体方向成立，但原横向 Wave 缺少跨属主事务、可靠事件、多副本互斥、逐动作授权和 FAV 边界；应先完成受控文档可靠性纵向试点，再按证据扩面。 + +**整改**: [模块解耦与文档可靠性重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md) + ### 项目 143 workflow-plan 409 与三权全有或全无（2026-09-14） **文件**: [2026-09-14-workflow-plan-409-frozen-and-optional-duties.md](./2026-09-14-workflow-plan-409-frozen-and-optional-duties.md) diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index 37098ad7..836387b5 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -5,13 +5,16 @@ ## 焦点 -1. **CHAT-INTENT**: [对话意图运行时状态](../audit/2026-09-13-conversation-intent-runtime-state.md)（`fix/conversation-intent-state`；与文档账本运行时无关） -2. **CHAT**: [对话历史刷新](../audit/2026-09-13-chat-history-refresh-disappears.md)（认领仍 `in_progress`；gap 行已 PASS，以代码/handoff 为准收敛） +1. **DOC-RELIABILITY**: [文档可靠性纵向试点 M0–M1](../ops/short-term/2026-09-14-document-reliability-pilot.md)（`refactor/document-reliability-pilot`；`mutex=docgov`；不占集成槽、不 rebuild） +2. **CHAT-INTENT**: [对话意图运行时状态](../audit/2026-09-13-conversation-intent-runtime-state.md)（`fix/conversation-intent-state`；与文档账本运行时无关） +3. **CHAT**: [对话历史刷新](../audit/2026-09-13-chat-history-refresh-disappears.md)（认领仍 `in_progress`；gap 行已 PASS，以代码/handoff 为准收敛） ## 并行认领 | owner | task | branch | worktree | mutex | status | |-------|------|--------|----------|-------|--------| +| Cursor | document-reliability-pilot | refactor/document-reliability-pilot | .worktrees/refactor-document-reliability-pilot | docgov | in_progress | +| Codex | module-decoupling-remediation-plan | feature/module-decoupling-remediation | .worktrees/feature/module-decoupling-remediation | architecture-docs | done | | verifier_antigravity | controlled-content-ui-acceptance | fix/controlled-content-ui-acceptance | .worktrees/fix-controlled-content-ui-acceptance | document-content-ui-acceptance | done | | implementer_cursor | controlled-content-canonical-hash | fix/controlled-content-canonical-hash | .worktrees/fix-controlled-content-canonical-hash | docgov | done | | Codex | project143-controlled-content-404-diagnosis | fix/project143-controlled-content-404-diagnosis | - | docgov-content-diagnosis | done | @@ -51,6 +54,7 @@ > 新任务开始前登记。`branch` 必须为 `feature/` 或 `fix/`，**禁止** `dev` / `test` / `main`。同一 `mutex` 同时仅一个 `in_progress`。 > workflow-plan 409 与治理鉴权 G1–G5 已收口（见 [test-report](../test-reports/2026-09-14-governance-action-level-authz-test-report.md)）。禁止终止或改写项目 143。 +> `document-reliability-pilot` 仅占 `docgov` 编码；合 `dev` + `env-service rebuild` 必须另占集成槽位。M1 报告未签字前不得 rebuild。 ## 集成槽位 @@ -84,6 +88,8 @@ | DOCREV 管理测试 | [../test-reports/2026-09-13-document-revision-management-test-report.md](../test-reports/2026-09-13-document-revision-management-test-report.md) | | DOCREV 运行时 UI Round20 | [../test-reports/2026-09-14-document-revision-runtime-ui-test-report.md](../test-reports/2026-09-14-document-revision-runtime-ui-test-report.md) | | 纯软件项目自举设计 | [../system-design/2026-09-14-software-project-bootstrap-workflow.md](../system-design/2026-09-14-software-project-bootstrap-workflow.md) | +| 模块化单体解耦架构 | [../system-design/2026-09-14-module-decoupling-architecture.md](../system-design/2026-09-14-module-decoupling-architecture.md) | +| 模块解耦整改计划 | [../remediation/2026-09-14-module-decoupling-refactor-plan.md](../remediation/2026-09-14-module-decoupling-refactor-plan.md) | | SSE/幂等审计 | [../audit/2026-09-13-document-revision-sse-idempotency.md](../audit/2026-09-13-document-revision-sse-idempotency.md) | | 质量门禁自污染审计 | [../audit/2026-09-13-document-revision-quality-gate-self-pollution.md](../audit/2026-09-13-document-revision-quality-gate-self-pollution.md) | | 当前文档路由审计 | [../audit/2026-09-13-document-revision-active-doc-routing.md](../audit/2026-09-13-document-revision-active-doc-routing.md) | diff --git a/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md b/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md new file mode 100644 index 00000000..38f408e0 --- /dev/null +++ b/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md @@ -0,0 +1,74 @@ +--- +title: "模块解耦重构验收清单" +status: planned +created: 2026-09-14 +updated: 2026-09-14 +owner: TBD +--- + +# 模块解耦重构验收清单 + +## M0 架构与基线 + +- [ ] contracts 分类、事务 owner、表属主、锁顺序、事件信封、Principal 和 FAV 边界均有唯一规范来源。 +- [ ] arch-check 分离 production import、SQL write、path 和 event registration；例外有 owner 与到期日。 +- [ ] 当前违规基线可重复生成，新违规立即阻断。 + +## M1 文档可靠性 + +- [ ] 人工编辑、agent commit、approve/reject 的跨表写入同事务。 +- [ ] fault injection 任一点失败均无部分提交。 +- [ ] 同 ETag 并发写只有一个赢家；不同文档不被全局锁串行。 +- [ ] 1、2、4 app replicas 结果一致。 +- [ ] GET/refresh/reload 零写入，projection 缺失不伪装成 404。 +- [ ] 项目 143 等价 hazard 与新项目真实页面刷新、重登、restart、rebuild PASS。 +- [ ] 匿名、非成员、跨用户、跨项目、错误 stage/duty、授权失败负例 PASS。 + +## M2 事件与 FAV + +- [ ] 业务终态、staging 和 outbox 同事务。 +- [ ] 每个 consumer 有独立 delivery/inbox/attempt/lease/dead-letter。 +- [ ] 1、2、4 dispatcher 无重复可见副作用和永久饥饿。 +- [ ] crash-before/after-side-effect/ack 全部可恢复。 +- [ ] stale fencing、跨项目事件、错误 producer、坏 digest 均拒绝。 +- [ ] FAV 中间结果、Prompt、工具轨迹和 secret 准入率为零。 +- [ ] DB 授权成功后才允许 degraded read；授权图缺失 fail closed。 +- [ ] tombstone/删除账本 union-deny PASS。 + +## M3 TaskRun、workspace 与数据生命周期 + +- [ ] RunManifest 规范字节、SHA、成员和 storage identity 持久化。 +- [ ] 重复 finalize、SSE 断线和 cancel race 不重复消息或 artifact。 +- [ ] WorkspaceRef 无物理路径，路径遍历和链接逃逸负例 PASS。 +- [ ] 旧 lease writer 被 fencing token 拒绝。 +- [ ] workspace 清空后最终成果仍可定位和校验。 +- [ ] backfill 可暂停、续跑和重放；异常进入 quarantine。 +- [ ] 八条 migration/reload/recovery 轨道 PASS。 +- [ ] 数据销毁前完整备份、corpus、FAV coverage 和人工批准全部满足。 + +## M4–M6 选择性扩展 + +- [ ] 每个子切片在原包边界先获得行为收益，再决定物理搬包。 +- [ ] chatlog/contextasm 保持授权可见集合和计数一致。 +- [ ] app 是唯一组合根，subscriber 重复启动不重复注册。 +- [ ] UserProjectContext 不再被多个模块字段级直改。 +- [ ] server 无 SQL、路径拼接和业务闭包。 +- [ ] 对应架构规则全部 block，过期例外和 flags 已删除。 + +## 证据分层 + +- [ ] Unit/stub 证据。 +- [ ] PostgreSQL integration 证据。 +- [ ] race/concurrency/load 证据。 +- [ ] fault injection/crash 证据。 +- [ ] 真实 PI 证据。 +- [ ] 真实浏览器证据。 +- [ ] restart/rebuild 证据。 +- [ ] cross-user/project authorization negative 证据。 +- [ ] scratch recovery 与完整 DB restore 证据。 + +## 关闭条件 + +- [ ] test report 回链具体 U-ID、commit、环境 identity 和 fixture/dataset。 +- [ ] CURRENT、FEATURE-MATRIX、gap-matrix 与任务状态一致。 +- [ ] 未实施后续仍为 planned/deferred，不用局部测试冒充全计划完成。 diff --git a/.doc/ops/long-term/2026-09-14-selective-module-decoupling.md b/.doc/ops/long-term/2026-09-14-selective-module-decoupling.md new file mode 100644 index 00000000..11458be0 --- /dev/null +++ b/.doc/ops/long-term/2026-09-14-selective-module-decoupling.md @@ -0,0 +1,47 @@ +--- +title: "选择性模块解耦 M4–M6" +status: planned +created: 2026-09-14 +updated: 2026-09-14 +owner: TBD +mutex: architecture +review_cycle: milestone +--- + +# 选择性模块解耦 M4–M6 + +## 目标 + +在文档可靠性、可靠事件和 FAV 已证明收益后，按 U10–U12 分别收敛 chatlog/contextasm、应用组合根、共享上下文和传输层。 + +## 为什么是长期任务 + +这些改动不直接修复当前受控正文事实源问题，且会触及仍在进行的 chat/intent 工作。 +M1–M3 未通过前执行会扩大冲突面，并提前冻结未经真实消费者验证的契约。 + +## 触发条件 + +- M1–M3 的 test report 全部 PASS。 +- 同类缺陷触达文件数和运行故障率表明继续解耦具有收益。 +- `chat`、`intent`、`bizdev/coordinator`、`server/stream` mutex 按子切片释放。 +- 每个子切片有独立 owner、回滚点和行为验收，不批准一个巨型 Wave。 + +## 候选子任务 + +1. U10 chatlog/contextasm 原地端口化。 +2. U11a 唯一 app composition root。 +3. U11b 按事件逐个替换闭包桥。 +4. U11c 逐 use case 迁移 ExecuteTask/executeStageInternal。 +5. U11d UserProjectContext snapshot/update 收权。 +6. U12 server/SSE 边界和架构门禁收口。 + +## 不包含 + +- 错误信封统一。 +- PI Runner result.json 协议。 +- 微服务拆分。 + +## 相关链接 + +- [模块解耦整改与重构计划](../../remediation/2026-09-14-module-decoupling-refactor-plan.md) +- [模块化单体解耦架构](../../system-design/2026-09-14-module-decoupling-architecture.md) diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md new file mode 100644 index 00000000..6983f3cd --- /dev/null +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -0,0 +1,61 @@ +--- +title: "文档可靠性纵向试点 M0–M1" +status: in_progress +created: 2026-09-14 +updated: 2026-09-14 +owner: Cursor +branch: refactor/document-reliability-pilot +worktree: .worktrees/refactor-document-reliability-pilot +mutex: docgov +--- + +# 文档可靠性纵向试点 M0–M1 + +## 目标 + +在不横向搬包的前提下，完成 U1–U4，使受控文档在并发提交、页面刷新、重登、restart、rebuild 和 projection 缺失后仍从数据库账本返回唯一授权正文。 + +## 范围 + +- U1 架构基线与 production writer/path 门禁。 +- U2 应用 UnitOfWork 和跨属主事务能力。 +- U3 人工编辑、agent commit、批准/驳回事务化。 +- U4 授权后的文档统一读模型。 +- 项目 143 等价脱敏 hazard 与新项目真实 PI/UI 验收。 + +## 非目标 + +- 不启用 FAV degraded read。 +- 不拆 chatlog/contextasm/squadrun 包。 +- 不统一错误信封，不修改 PI Runner 协议。 +- 不清理历史 dev 数据。 + +## 步骤 + +1. [x] 从最新 `dev`（`a8f9451`）创建实现分支，登记 CURRENT 并确认 `docgov` mutex 已释放。 +2. [ ] 为当前跨表提交、刷新读取、ETag/idempotency 和权限负例补 characterization。 +3. [ ] 实现 U1–U2，验证任意属主写入/commit 故障整体回滚。 +4. [ ] 实现 U3，删除提交后 outbox 正确性回调。 +5. [ ] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 +6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 +7. [ ] 写 test report，更新验收清单、FEATURE-MATRIX、CURRENT 和 gap-matrix。 + +## 验收标准 + +- [ ] 同一 ETag 并发编辑只有一个合法赢家，账本/head/outbox 无孤儿。 +- [ ] 任一事务故障点不会留下部分提交。 +- [ ] 项目 143 等价 hazard 和新项目刷新/重启目标 404/409/projection missing 为 0。 +- [ ] 匿名、非成员、跨用户、跨项目、错误 stage/duty 和授权依赖失败均 fail closed。 +- [ ] 1、2、4 个 app 实例的最终状态一致。 +- [ ] M1 test report 给出继续 M2 或停止扩面的明确结论。 + +## 相关链接 + +- Checklist: [../checklists/2026-09-14-module-decoupling-refactor-acceptance.md](../checklists/2026-09-14-module-decoupling-refactor-acceptance.md) +- Handoff: 实施时创建 +- Test report: 实施后创建 `.doc/test-reports/-document-reliability-pilot.md` +- Plan: [../../remediation/2026-09-14-module-decoupling-refactor-plan.md](../../remediation/2026-09-14-module-decoupling-refactor-plan.md) + +## 证据 / 备注 + +只有本任务真实验收通过后，才允许启动 durable event/FAV 与横向模块扩展。 diff --git a/.doc/ops/short-term/2026-09-14-durable-event-fav-integration.md b/.doc/ops/short-term/2026-09-14-durable-event-fav-integration.md new file mode 100644 index 00000000..bad952dd --- /dev/null +++ b/.doc/ops/short-term/2026-09-14-durable-event-fav-integration.md @@ -0,0 +1,60 @@ +--- +title: "可靠事件与 FAV 集成 M2–M3" +status: planned +created: 2026-09-14 +updated: 2026-09-14 +owner: TBD +branch: refactor/durable-event-fav-integration +worktree: "-" +mutex: event-delivery +--- + +# 可靠事件与 FAV 集成 M2–M3 + +## 目标 + +完成 U5–U9，使投影、通知、RunManifest 和最终成果在多 worker、崩溃、重放和 workspace 清理后仍可恢复且不越权。 + +## 范围 + +- opaque workspace capability、lease 和 fencing。 +- durable outbox、逐消费者 delivery/inbox、重试和 dead letter。 +- 审计并集成 `feature/final-artifact-vault` 的适用实现。 +- taskrun finalize、RunManifest 持久化、FAV coverage 和迁移 corpus。 + +## 非目标 + +- 不盲目整分支 cherry-pick FAV。 +- 不启用跨授权图的 vault 读取。 +- 不自动销毁 dev/main 数据。 +- 不执行 U10–U12。 + +## 步骤 + +1. [ ] 读取 M1 test report，确认继续 M2 的 go/no-go 已通过。 +2. [ ] 重新分配 Goose 序号并审计 U6/FAV schema 组合。 +3. [ ] 实现 U5–U6，多 app/worker 和 crash matrix 先通过。 +4. [ ] 逐文件审计 FAV 分支，按 U7 边界迁移合格实现。 +5. [ ] 实现 U8，收敛 taskrun terminal finalize 与 RunManifest。 +6. [ ] 实现 U9 的 checkpoint backfill、corpus、quarantine 和 cleanup gates。 +7. [ ] 完成真实 PI、浏览器、workspace 清理、scratch recovery 和 rebuild 演练。 + +## 验收标准 + +- [ ] 多订阅者有独立 delivery，失败者不阻塞其他订阅者。 +- [ ] worker crash/retry/replay 不造成重复可见副作用。 +- [ ] workspace 清空后，verified 最终文档/回复仍可定位并校验。 +- [ ] 授权图缺失、跨项目、tombstone、坏 digest 和 stale fencing 全部拒绝。 +- [ ] clean/oldest/previous/hazard/import/reload/rollback/artifact recovery 八轨有独立证据。 +- [ ] 任何数据销毁仍需完整备份、corpus、FAV coverage 和用户批准。 + +## 相关链接 + +- Checklist: [../checklists/2026-09-14-module-decoupling-refactor-acceptance.md](../checklists/2026-09-14-module-decoupling-refactor-acceptance.md) +- Handoff: 实施时创建 +- Test report: 实施后按 M2、M3 分开创建 +- Plan: [../../remediation/2026-09-14-module-decoupling-refactor-plan.md](../../remediation/2026-09-14-module-decoupling-refactor-plan.md) + +## 证据 / 备注 + +本任务依赖 M1，不得与 M1 同时修改 docgov/docrevision 的事务主链。 diff --git a/.doc/remediation/2026-09-14-module-decoupling-refactor-plan.md b/.doc/remediation/2026-09-14-module-decoupling-refactor-plan.md new file mode 100644 index 00000000..e9bfc070 --- /dev/null +++ b/.doc/remediation/2026-09-14-module-decoupling-refactor-plan.md @@ -0,0 +1,677 @@ +--- +title: "模块解耦与文档可靠性重构计划" +status: planned +created: 2026-09-14 +updated: 2026-09-14 +type: refactor +owner: TBD +deepened: 2026-09-14 +--- + +# 模块解耦与文档可靠性重构计划 + +## Goal Capsule + +- **Objective:** 项目文档、任务最终产物和最终智能体回复在并发写入、页面刷新、服务重启、工作区清理和受控恢复后仍能按权限定位到唯一、可校验的内容。 +- **Means:** 采用模块化单体、纵向绞杀、应用 UnitOfWork、数据库互斥、transactional outbox、opaque workspace capability 和 FAV。 +- **Authority:** 本计划受 [模块化单体解耦 ADR](../adr/2026-09-14-modular-monolith-decoupling.md)、[文档版本边界 ADR](../adr/2026-09-13-document-version-boundary.md) 和 [工作流运行时权威 ADR](../adr/2026-09-12-workflow-runtime-authority.md) 约束。 +- **Stop condition:** 任一里程碑出现数据不变量无法在单事务维持、授权只能通过放宽实现、旧数据无法安全迁移，或真实页面故障率没有下降时，停止扩面并回到设计评审。 +- **Execution profile:** 高风险、跨模块、迁移和授权敏感；characterization-first，逐纵向切片发布。 +- **Delivery:** 每个里程碑独立 short-term、独立分支和独立验收报告；本计划本身不授权合并、部署或删数据。 + +--- + +## Product Contract + +### Summary + +重构先解决文档事实源、事务与恢复链，再扩展其他模块。 +架构改造必须用用户可见的刷新、重启、并发和授权结果证明价值，不能只用包数量或代码行数证明完成。 + +### Problem Frame + +系统当前同时依赖 PostgreSQL 账本、`document_records` 指针、workspace `output.md` 和若干内存状态。 +这些状态在提交、刷新和重启时没有统一事务与恢复协议，导致历史上出现正文 404/409、projection missing、缓存竞争和任务完成后文档仍为空。 + +原解耦方案提出 contracts、单一写者和事件总线，但没有解释单一写者如何维持现有跨表原子性，也没有为多个 event subscriber、跨实例互斥和 FAV 恢复定义可执行协议。 + +### Requirements + +**事实源与持久成果** + +- R1. PostgreSQL immutable ledger 是文档版本与业务关系的权威，workspace 只作为可重建投影。 +- R2. FAV 只归档服务端确认的终态文档、终态助手回复、类型化 RunManifest 和必要附件，中间结果准入率为零。 +- R3. 数据库授权图完整但正文不可用时，系统可按 feature flag 从已验证 FAV 只读降级；授权图不完整时必须拒绝。 +- R4. GET、刷新和重载不得隐式修复数据库、覆盖 head 或创建归档。 + +**事务与并发** + +- R5. 一个业务命令只有一个事务 owner；业务状态、账本、当前指针及该里程碑已启用的 staging/outbox 必须原子提交。 +- R6. 单一写者只约束 SQL 和不变量所有权，不允许各属主各自提交导致跨表半完成。 +- R7. 多副本正确性必须依赖数据库行锁、事务级 advisory lock、唯一约束、expected version 和 fencing token；进程内 mutex 不能成为唯一正确性条件。 +- R8. 所有写命令必须具有 idempotency key、expected version 或两者；相同 key 不同 payload 必须返回冲突。 +- R9. 事务不得包含 LLM、PI Runner、对象存储、HTTP/SSE 或长文件 I/O。 + +**事件与恢复** + +- R10. 持久事件采用至少一次投递、逐消费者 delivery、幂等 inbox、租约、fencing token、重试和死信。 +- R11. 同一聚合事件按 aggregate version 有序，不同聚合允许并发；失败消费者不阻塞其他消费者。 +- R12. 旧 workspace、旧数据库和 FAV 回填必须可断点续跑、可重复、可审计，异常进入 quarantine。 +- R13. 恢复只进入 scratch identity，promotion 需要系统管理员和数据责任人确认，并通过属主端口提交。 + +**授权与隔离** + +- R14. HTTP、CLI、worker、event subscriber 和恢复工具必须使用服务端构造的 Principal，并按 action/resource/current state 逐动作授权。 +- R15. 客户端或事件 payload 中的 user、project、role、duty、backend 和路径不能作为授权事实。 +- R16. WorkspaceRef 只能暴露不可伪造 ID、scope 和 fencing token；物理路径不得跨模块传播。 +- R17. 路径解析必须拒绝绝对路径、`..`、跨项目 ID、symlink/hardlink 逃逸和 storage identity 不匹配。 +- R18. 事件、日志、metrics、manifest 和 result 文件不得包含 secret、认证材料、Prompt 或不必要正文。 + +**交付与验证** + +- R19. 首个独立交付边界是受控文档可靠性纵向试点，未通过停走门前不启动横向包搬移。 +- R20. 每个里程碑必须分别提供 unit、PostgreSQL、并发/故障注入、真实 PI、真实浏览器、restart/rebuild 和跨用户负例证据；不适用项必须说明原因。 +- R21. 迁移采用 expand/migrate/contract，允许新的 Goose migration，禁止修改已发布 migration。 +- R22. 数据清理必须同时满足完整备份恢复、migration corpus、FAV coverage、精确删除 manifest 和用户批准，不得因“已有最终成果”跳过其他门禁。 + +### Success Criteria + +- 项目 143 等价脱敏 hazard 和新建项目两条链路在刷新、重登、restart 与 rebuild 后正文成功率为 100%，目标 404/409/projection missing 为 0。 +- 同一文档并发写只有一个合法赢家；失败请求不产生孤儿 snapshot、错 head、重复消息、重复通知或重复 artifact。 +- 每个应发布终态都存在 publish request；已验证 artifact 可在 workspace 删除后按稳定逻辑 ID 定位并校验 SHA。 +- 匿名、非成员、跨用户、跨项目、错误 duty、错误 stage、篡改事件和授权依赖失败全部 fail closed。 +- 1、2、4 个应用实例和 dispatcher 下，状态与副作用结果一致；性能阈值由基线报告确认后固化，不能通过降低一致性达标。 +- 旧 schema corpus、legacy-v0 workspace 和 previous-release corpus 均可升级、重载和回滚到最近兼容点。 + +### Scope Boundaries + +**本计划包含** + +- 文档、taskrun、workspace、artifactvault、可靠事件和授权基础设施。 +- chatlog/contextasm、squadrun、app 组合根和 UserProjectContext 的后续选择性扩展。 +- 数据生命周期、迁移 corpus、回填、quarantine 和 scratch 恢复的集成门禁。 + +**Deferred to Follow-Up Work** + +- HTTP 错误信封统一。 +- PI Runner stdout/result.json 协议替换。 +- 全量 SSE transport 重写。 +- 微服务拆分和跨服务消息基础设施。 +- main 数据自动清理和未来 tenant 实体。 + +--- + +## Planning Contract + +### Key Technical Decisions + +- KTD1. **纵向绞杀优先。** 先在原包边界增加窄端口和适配器，迁完消费者并通过行为验收后再物理移动包。 +- KTD2. **事务 owner 与表属主分离。** 应用用例拥有事务，领域模块拥有 SQL；同一 `pgx.Tx` 通过不透明 TxScope 绑定 repository。 +- KTD3. **显式锁策略。** 普通状态转换使用 `READ COMMITTED`、行锁和条件更新；项目级不变量使用事务级 advisory lock；Serializable 只用于经过证明的复杂不变量。[PostgreSQL explicit locking](https://www.postgresql.org/docs/17/explicit-locking.html) +- KTD4. **固定锁顺序。** project/workflow instance → document root → version/snapshot/turn → task run → staging/outbox/delivery。 +- KTD5. **事务外副作用。** 网络、LLM、对象存储、通知和投影由提交后的持久事件驱动。 +- KTD6. **至少一次事件。** outbox 不承诺 exactly-once；event ID、consumer inbox 和业务唯一约束共同提供幂等结果。 +- KTD7. **opaque workspace capability。** 跨模块不传物理路径，所有读写使用 scope-bound ID、artifact key 和 fencing token。 +- KTD8. **三层权威。** PostgreSQL 管业务关系与授权，workspace 管临时现场，FAV 管已验证终态 payload/manifest/receipt/tombstone。 +- KTD9. **授权逐动作重算。** Principal 来自受信适配器；状态改变在取得资源锁后重读当前授权事实，subscriber 的 actor 快照只用于审计。[OWASP Authorization Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) +- KTD10. **依赖感知回滚。** 每个里程碑只保证回到最近兼容点；进入 contract 阶段后须按逆依赖顺序回滚。 +- KTD11. **按证据收紧门禁。** arch-check 先记录基线，每条规则在对应纵向切片清零后转 block，不按日历统一切换。 +- KTD12. **FAV 分支先审后用。** `feature/final-artifact-vault` 的现有实现作为候选，不直接视为 dev 已有能力；集成前重放迁移、测试、安全和并发审计。 + +### High-Level Technical Design + +#### Component topology + +```mermaid +flowchart TB + Transport[HTTP SSE CLI] --> UseCases[application use cases] + Jobs[workers and recovery] --> UseCases + UseCases --> Policy[authorization policy] + UseCases --> Unit[UnitOfWork] + Unit --> DG[docgov owner] + Unit --> DR[docrevision owner] + Unit --> TR[taskrun owner] + Unit --> OB[outbox and staging] + UseCases --> WSC[workspace capability] + OB --> Dispatch[delivery dispatcher] + Dispatch --> Projection[projection subscriber] + Dispatch --> AV[artifactvault subscriber] + Dispatch --> Notify[notification subscriber] +``` + +#### Write sequence + +```mermaid +sequenceDiagram + participant C as Caller + participant A as App use case + participant P as Policy + participant DB as PostgreSQL + participant O as Outbox worker + participant V as FAV + C->>A: command + idempotency + expected version + A->>P: authorize principal/action/resource + P-->>A: allow or deny + A->>DB: begin and acquire locks in canonical order + A->>DB: ledger + root + staging + outbox + DB-->>A: commit + A-->>C: committed result + O->>DB: claim delivery + O->>V: publish immutable artifact + V-->>O: verified receipt + O->>DB: idempotent ack +``` + +#### Event delivery lifecycle + +```mermaid +stateDiagram-v2 + [*] --> pending + pending --> leased + leased --> delivered + leased --> retryable_failed + leased --> dead_letter + retryable_failed --> leased + leased --> pending: lease expired + delivered --> [*] +``` + +#### Migration lifecycle + +```mermaid +flowchart TB + Expand[Expand schema and compatible readers] --> Shadow[Shadow validation] + Shadow --> Backfill[Checkpointed backfill] + Backfill --> Switch[Switch writer] + Switch --> Observe[Observe and restart test] + Observe --> Contract{Go or rollback?} + Contract -->|go| Remove[Remove old path] + Contract -->|rollback| Previous[Return to prior compatible point] +``` + +### Concurrency and Lock Matrix + +| Resource/operation | Correctness primitive | Conflict result | Retry policy | Forbidden while held | +|---|---|---|---|---| +| Project duty/heal/revoke | transaction advisory lock keyed by project | wait then typed timeout | whole idempotent tx for deadlock only | network, LLM, filesystem copy | +| Workflow stage advance | instance row lock + expected config revision | stale-state conflict | caller refresh; no blind retry | external execution | +| Controlled document edit | document row + open version row + ETag | version conflict with current head metadata | user/client rebases | projection and notification | +| Agent candidate commit | candidate/document/version locks + idempotency key | stale candidate or idempotency conflict | exact same request may replay | PI/LLM call | +| TaskRun claim/finalize | unique idempotency + conditional status/version update | existing run or state conflict | same digest returns prior result | message/SSE side effect | +| Workspace write | DB lease + fencing token + atomic file commit | stale lease/token | reacquire then re-resolve state | arbitrary path access | +| Outbox delivery | row lease + `SKIP LOCKED` + fencing token | another worker owns item | retry at next schedule | subscriber work inside claim tx | +| FAV publish | artifact idempotency unique key + no-replace object write | existing verified artifact or digest conflict | same digest resumes | reading mutable workspace | + +### Authorization Matrix + +| Surface | Principal source | Resource facts | Required checks | Failure behavior | +|---|---|---|---|---| +| HTTP/SSE | authenticated session/middleware | DB project/session/document/workflow | membership, owner, duty, stage, action | 401/403/typed conflict; no data | +| CLI ops | service identity + operator session | explicit environment/dataset/resource manifest | capability, exact scope, dry-run where destructive | fail closed and audit | +| Event subscriber | registered service identity; event actor for audit only | DB current aggregate + event scope/digest | producer allowlist, project/resource match, current policy | dead letter/quarantine | +| FAV degraded read | current authenticated Principal | DB relation + vault manifest/tombstone | normal read permission first, then integrity | no fallback if auth graph incomplete | +| Recovery promotion | recovery credential + admin + data steward approval | scratch identity + signed plan + conflict map | two-party approval, no unresolved conflicts | refuse promotion | +| Agent backend | app-resolved BackendRef | project policy + InputBundle sensitivity | backend allowlist, least data, secret injection boundary | reject before dispatch | + +### Delivery dependency graph + +```mermaid +flowchart TB + M0[M0 baseline and contract correction] --> M1[M1 document reliability pilot] + M1 --> M2[M2 durable events and FAV integration] + M2 --> M3[M3 taskrun and workspace finalization] + M3 --> M4[M4 selective domain decoupling] + M4 --> M5[M5 app composition and context ownership] + M5 --> M6[M6 transport boundary cleanup] + M2 --> D[Data corpus and lifecycle rollout] + D --> M3 +``` + +### Assumptions + +- 当前 dev 基线为 schema 33；任何实现分支必须在创建 migration 时重新查询最新序号。 +- `feature/final-artifact-vault` 尚未合 dev，现有代码与测试只能作为候选实现和证据来源。 +- 当前无独立 tenant 实体，权限范围沿用 environment + project membership/admin；未来 tenant 模型另立 ADR。 +- 性能容量目标需要在 M1 压测后固化；本计划先固定正确性和观测项，不伪造吞吐数字。 + +--- + +## Phased Delivery + +| 里程碑 | 交付边界 | 预计范围 | 进入条件 | 退出/停走门 | +|---|---|---|---|---| +| M0 | 基线与契约纠偏 | 文档/门禁/characterization | 在途 mutex 不冲突 | 事务/锁/授权/事件/FAV 契约评审通过 | +| M1 | 文档可靠性纵向试点 | docflow/docgov/docrevision/docquery | M0 完成，独占 docgov | 刷新、重启、并发、权限和故障注入全 PASS | +| M2 | 持久事件与 FAV | outbox/delivery/artifactvault | M1 数据边界稳定 | 多 worker、重放、归档、降级读取和 tombstone PASS | +| M3 | taskrun/workspace 终态 | taskrun/squadrun/workspace | M2 verified receipt 可用 | 清 workspace 后最终产物仍可定位，重复 finalize 无副作用 | +| M4 | 选择性领域端口化 | chatlog/contextasm/squadrun | M1–M3 复盘证明收益 | 每个子切片独立回归；无纯搬包 | +| M5 | 组合根与上下文收权 | app/chatflow/stageflow/UserProjectContext | M4 稳定 | 每个 use case 单 owner；闭包按事件逐个替换 | +| M6 | 传输收口 | server/SSE | M5 完成 | server 无 SQL/编排/路径；R1–R7 对应规则全 block | + +M0–M1 是首个批准范围。 +M2–M6 需要在前一里程碑验收报告后重新 go/no-go，不构成一次性整体授权。 + +--- + +## Implementation Units + +### Unit Index + +| ID | 标题 | 主要文件 | 依赖 | +|---|---|---|---| +| U1 | 固化架构基线和门禁 | `scripts/arch-check.sh`, `.doc/system-design/module-boundaries.yaml` | 无 | +| U2 | 建立应用 UnitOfWork | `internal/app/uow`, owner repositories | U1 | +| U3 | 文档命令原子化 | docgov/docrevision/docflow | U2 | +| U4 | 文档授权读模型 | docquery/server/authz | U1, U3 | +| U5 | Workspace capability | workspace/squadrun | U1 | +| U6 | Durable event delivery | outbox/dispatcher/inbox | U2 | +| U7 | FAV 集成与降级读取 | artifactvault/docflow/docquery | U3, U4, U6 | +| U8 | TaskRun/RunManifest 终态 | taskrun/squadrun/workspace | U5, U6, U7 | +| U9 | 数据回填与迁移 corpus | dataset/FAV tooling | U6–U8 | +| U10 | chatlog/contextasm 端口化 | ctxmgr/chat/contextasm | M1–M3 go/no-go | +| U11 | 组合根和 use case 收权 | app/server/bizdev/coordinator | U10 | +| U12 | 传输边界与最终门禁 | server/stream/CI | U11 | + +### U1. 固化架构基线和可执行门禁 + +- **Goal:** 建立不会把现有违规合法化、也不会误报 migration/fixture 的边界基线。 +- **Requirements:** R19–R21；KTD1、KTD11。 +- **Dependencies:** 无；必须避开 `intent`、`chat` 等在途 mutex，当前只占 `arch`。 +- **Files:** `scripts/arch-check.sh`、`.doc/system-design/module-boundaries.yaml`、`internal/archtest/arch_test.go`、`internal/archtest/arch_test_test.go`、`.doc/audit/-arch-check-baseline.md`。 +- **Approach:** + 1. 分离 production import、production SQL write、path construction、event producer/consumer 四类规则。 + 2. 表名扫描只识别 production Go 中的写语句；Goose、测试、fixture 和受控运维脚本进入显式类别，不以全仓零出现为目标。 + 3. 为每项既有违规记录 owner、原因、截止里程碑和到期日；新违规立即 block。 + 4. 只为已完成纵向切片的规则切换 block，不能以“两周后统一 block”代替清零。 +- **Execution note:** 先对当前依赖和 SQL 写者生成可重复 characterization 报告，再改门禁。 +- **Patterns to follow:** `go list` 依赖图、现有 AGENTS mutex、Goose 不可改写规则。 +- **Test scenarios:** + - 合法的 Goose/fixture 表名不触发生产写者违规。 + - 非属主 production 包新增 `UPDATE document_records` 时测试失败并定位文件。 + - 路径通过字符串拼接、`filepath.Join` 和间接常量构造时均被检测。 + - 无过期日例外、过期例外、新增越权 event producer 均阻断。 +- **Verification:** 基线报告可重放；新增违规阻断；现有违规数只能下降。 + +### U2. 建立应用 UnitOfWork 和事务能力 + +- **Goal:** 在单一写者边界内维持跨表业务不变量。 +- **Requirements:** R5–R9；KTD2–KTD5。 +- **Dependencies:** U1。 +- **Files:** `internal/app/uow/uow.go`、`internal/app/uow/pg_uow.go`、`internal/app/uow/pg_uow_test.go`、`internal/docgov/repository.go`、`internal/docrevision/repository.go`、`internal/taskrun/repository.go`、`internal/freedoc/repository.go`。 +- **Approach:** + 1. UnitOfWork 开启、提交和回滚唯一 `pgx.Tx`，向每个属主提供绑定该 transaction 的窄 repository capability。 + 2. TxScope 不暴露通用 `Exec` 给应用层；只有属主 repository 能执行自己的 SQL。 + 3. 禁止 repository 在已有 TxScope 内嵌套 transaction 或提前 commit。 + 4. 实现统一的完整事务重试 wrapper，只重试明确的 serialization/deadlock 错误，并保持 idempotency context。 + 5. 为锁超时、事务时长、重试和 rollback reason 加结构化指标，不记录业务正文。 +- **Test scenarios:** + - 第二个属主写入失败时第一个属主写入回滚。 + - outbox/staging 写入失败时业务状态和账本均回滚。 + - commit 失败时不会返回成功或执行提交后副作用。 + - `40001`/`40P01` 重跑完整事务并最终只产生一个业务结果。 + - version conflict、permission denied、payload digest conflict 不进入自动重试。 + - 尝试跨属主执行任意 SQL 无法通过类型边界或架构测试。 +- **Verification:** fault injection 覆盖 begin、每个 owner write、outbox、commit；无部分提交。 + +### U3. 文档提交纵向事务化 + +- **Goal:** 将人工编辑、agent commit、批准/驳回和根指针投影收敛到文档用例事务。 +- **Requirements:** R1、R5–R9、R19；KTD2–KTD5。 +- **Dependencies:** U2；实施时独占 `docgov` mutex。 +- **Files:** `internal/app/docflow/commands.go`、`internal/app/docflow/commands_pg_test.go`、`internal/docgov/engine.go`、`internal/docgov/lease.go`、`internal/docrevision/pg_mutate.go`、`internal/docrevision/service.go`、`internal/docrevision/document_commit_concurrency_pg_test.go`、`internal/server/document_revision_handlers.go`。 +- **Approach:** + 1. 为每个文档命令明确 aggregate、expected head、idempotency key 和授权 action。 + 2. 版本账本与 `document_records` 指针先在同一 UoW 写入；M1 保留现有通知兼容适配，但任何核心正确性不得依赖提交后的 `OnOutbox` 回调。 + 3. 按 KTD4 锁定 document root 与 open version；状态转换采用条件更新。 + 4. 提交返回 ledger/root 已持久化结果；workspace projection 和通知由提交后事件处理。 + 5. 保留现有 ETag 语义，统一 agent、人工与批准路径的冲突分类。 +- **Execution note:** 先补当前行为的 PG characterization，再移动写入所有权；不得先删除旧事务路径。 +- **Patterns to follow:** `internal/docrevision/pg_mutate.go` 当前同事务投影、`internal/docgov/lease.go` expected SHA、现有 document revision idempotency tests。 +- **Test scenarios:** + - 同一 ETag 两个用户并发编辑，只有一个成功，另一个得到当前 head。 + - 人工编辑与 agent commit 并发，账本 event_no 连续且 head 指向赢家。 + - approve 与 reject 并发，只允许一个合法终态。 + - candidate commit 在 root/version/outbox 任一故障点整体回滚。 + - 相同 idempotency key 和相同 digest 返回原结果；不同 digest 返回冲突。 + - 锁超时返回 typed retryable busy，不泄露 SQL 或路径。 + - 两个 app 实例执行同一请求，结果与单实例一致。 +- **Verification:** 文档四类命令的跨表不变量由 PostgreSQL 测试证明；`OnOutbox` 不再承担正确性。 + +### U4. 建立授权后的文档统一读模型 + +- **Goal:** 正文读取不再依赖 handler 拼路径或把一致性错误伪装成 404。 +- **Requirements:** R1、R3、R4、R14、R15、R19；KTD1、KTD8、KTD9。 +- **Dependencies:** U1、U3；实施时按 CURRENT 排队使用 `docgov` 和 `authz` mutex。 +- **Files:** `internal/app/docflow/query.go`、`internal/app/docflow/query_test.go`、`internal/docquery/service.go`、`internal/docquery/service_pg_test.go`、`internal/server/project_doc_handlers.go`、`internal/server/document_revision_authz_test.go`、`web/src/api/projectDocuments.ts`、`web/src/lib/controlledDocHydrate.ts`、`web/e2e/controlled-document-reload.spec.ts`。 +- **Approach:** + 1. Principal 由认证适配器构造，DocRef 不携带可覆盖 Principal 的 user/role/duty。 + 2. 先解析数据库 resource→project、workflow、stage 和 duty，再读取 ledger/current pointer。 + 3. workspace projection 只作为校验缓存；缺失时不阻止读取数据库 immutable content。 + 4. ledger/root、projection 和 FAV 的差异映射为 typed consistency state，不再统一变成 404。 + 5. FAV 降级在 U7 接入前保持关闭；接口先保留明确 extension point。 + 6. 列表和版本查询强制分页，BatchGet 首期上限 100；批量授权按 Principal/project 分组并使用有界并发。 +- **Test scenarios:** + - 正常 projection、projection 缺失、projection hash 漂移、ledger 缺失分别返回正确结果或 typed error。 + - GET、刷新、重登不产生数据库写入、workspace 文件或归档事件。 + - 匿名、非成员、跨用户会话、跨项目、错误 stage/duty 均拒绝且不泄露文档存在性。 + - 授权依赖超时/错误时 fail closed。 + - BatchGet 的 100/101 边界、重复引用、跨项目混合引用和限流拒绝不泄露资源存在性。 + - 项目 143 等价 hazard 和新项目在页面刷新、restart、rebuild 后正文一致。 +- **Verification:** 控制台和服务器目标 404/409/projection missing 为零；所有来源分支有服务端与浏览器证据。 + +### U5. Workspace capability、lease 与文件提交 + +- **Goal:** 从类型和运行时同时禁止跨模块裸路径与跨租户工作区访问。 +- **Requirements:** R1、R7、R9、R16–R18；KTD7。 +- **Dependencies:** U1。 +- **Files:** `internal/workspace/types.go`、`internal/workspace/resolver.go`、`internal/workspace/capability.go`、`internal/workspace/capability_test.go`、`internal/workspace/lease_pg_test.go`、`internal/squad/workspace_adapter.go`、`internal/bizdev/service.go`、`internal/server/project_doc_handlers.go`。 +- **Approach:** + 1. WorkspaceRef 删除 Root/OutputDir/ManifestPath，只保留 scope-bound ID、schema version 和 fencing token。 + 2. workspace service 提供限定 artifact key 的 read、atomic writer、commit projection 和 inspect 能力。 + 3. ID 与 lease 绑定 environment、project、run 和 storage identity；所有写入校验 fencing token。 + 4. 使用 realpath/descriptor-based 检查，拒绝 `..`、绝对路径、symlink/hardlink 和根外 rename。 + 5. legacy-v0 workspace 只读；首次受控 Resume 校验后原子升级 v1，并保留兼容备份一个发布周期。 +- **Test scenarios:** + - `..`、绝对路径、编码后的分隔符、空 key、跨 project Workspace ID 全部拒绝。 + - symlink 和 hardlink 在校验后替换的 TOCTOU 场景仍不能逃逸。 + - 旧 lease writer 在新 lease 签发后因 fencing token 过期而失败。 + - 两个 app 实例并发提交同一 projection，reader 只看到完整 marker。 + - v0 读取、升级失败回滚、未知 major 版本拒绝。 +- **Verification:** production 包中 workspace 物理路径只存在于 workspace/squadrun adapter；多实例测试不依赖 `sync.Mutex`。 + +### U6. Durable outbox、逐消费者 delivery 与 replay + +- **Goal:** 让后台投影、通知和 FAV 发布在崩溃与水平扩容下可恢复。 +- **Requirements:** R5、R9–R11、R14、R18、R21；KTD5、KTD6。 +- **Dependencies:** U2；需要新的 additive Goose migration。 +- **Files:** `deploy/postgres/goose/000xx_*durable_event_delivery.sql`、`internal/events/envelope.go`、`internal/events/outbox.go`、`internal/events/dispatcher.go`、`internal/events/inbox.go`、`internal/events/dispatcher_pg_test.go`、`internal/events/replay.go`、`cmd/events/main.go`。 +- **Approach:** + 1. 保留业务 outbox record，新增 event envelope、每消费者 delivery/inbox、attempt、lease、next_attempt_at 和 dead-letter 数据。 + 2. `FOR UPDATE SKIP LOCKED` 只用于小批次 claim；worker 在 claim transaction 外执行 subscriber。 + 3. ack/fail 使用 delivery version/fencing token，过期 worker 不能覆盖新 lease owner。 + 4. 同 aggregate 依据 aggregate version 串行，不同 aggregate 并行；缺号进入等待或 quarantine。 + 5. retry policy 区分 transient、permanent、authorization/integrity failures；后两者默认 dead letter/quarantine。 + 6. replay CLI 先 dry-run，必须限定 consumer/event/project/time/id 范围并记录 operator、reason 和计划 hash。 + 7. 逐个迁移并删除 `OnOutbox` 与闭包桥；切换前 shadow 对账，切换后旧回调不再注册。 +- **Test scenarios:** + - 两个和四个 worker 并发 claim，没有同一 lease 的双 owner。 + - worker 在副作用前、后、ack 前崩溃，重启后最终只有一个可见业务结果。 + - 一个 subscriber 永久失败不阻塞其他 subscriber。 + - lease 过期后旧 worker ack 被 fencing token 拒绝。 + - 相同 event 重放被 inbox 去重；相同 aggregate 乱序事件不越过缺失版本。 + - 伪造 producer、跨项目 scope、payload digest 不符进入 quarantine。 + - dead-letter replay 未授权、无 dry-run 或范围过宽时拒绝。 +- **Verification:** 删除旧单一 `published_at` 语义前完成双读对账；pending age、attempt、lease steal、dead letter 可观测。 + +### U7. FAV 分支审计、终态发布与授权降级读取 + +- **Goal:** 将最终成果与 workspace 清理解耦，同时不形成第三套业务真相源。 +- **Requirements:** R1–R4、R10–R15、R18、R21、R22；KTD8、KTD9、KTD12。 +- **Dependencies:** U3、U4、U6；先审计 `feature/final-artifact-vault` 相对当前 dev 的完整差异。 +- **Files:** `internal/artifactvault/`、`deploy/postgres/goose/000xx_*final_artifact*.sql`、`internal/app/docflow/artifact.go`、`internal/docquery/service.go`、`internal/server/vault.go`、`cmd/artifact-vault/`、`internal/artifactvault/*_test.go`。 +- **Approach:** + 1. 重用通过审计的 FAV eligibility、staging、publisher、manifest、tombstone 和 scratch recovery；不直接 cherry-pick 未复核的组合根/handler 改动。 + 2. 重新分配最新 Goose 序号，并验证 migration 与 U6 delivery schema 的组合和 Down 边界。 + 3. 文档终态 transaction 冻结精确 payload/SHA 或 immutable blob ref 并写 publish request；publisher 不重读 workspace 或可变消息行。 + 4. docquery 只在正常数据库授权成功后使用 FAV；manifest 只做 scope/integrity 交叉校验，不能授予权限。 + 5. union-deny：运行删除账本或 vault tombstone 任一侧声明删除即拒绝读取/恢复。 +- **Execution note:** 现有 FAV 分支包含业务代码和 migration，必须先 rebase/merge audit，再按本计划拆分原子提交。 +- **Test scenarios:** + - committed/sealed/approved 文档按准入表归档；ready/rejected/stale/quality_failed 和 workspace 草稿为零。 + - publish 重复、对象已存在、每个 fsync/rename/marker 崩溃点都可恢复。 + - workspace 删除后，授权用户可读 verified payload；未授权用户看不到 artifact 存在性。 + - DB 授权关系缺失、manifest scope 不符、坏 SHA、tombstone、vault 不可用均 fail closed 或返回明确 degraded error。 + - GET 零数据库写入、零 workspace 写入、零隐式恢复。 + - 跨项目相同 SHA 不共享删除或密钥边界。 +- **Verification:** FAV test report 与当前 dev 重放通过；`FAV_DEGRADED_READ` 默认关闭，直到 M2 全部负例通过。 + +### U8. TaskRun、RunManifest 与 workspace 清理门禁 + +- **Goal:** 任务终态和最终产物在重复 finalize、SSE 断线和 workspace 清理后保持一致。 +- **Requirements:** R1、R2、R5–R12、R16–R18、R22；KTD5–KTD8。 +- **Dependencies:** U5、U6、U7。 +- **Files:** `internal/taskrun/manager.go`、`internal/taskrun/store.go`、`internal/taskrun/finalize.go`、`internal/taskrun/finalize_pg_test.go`、`internal/squad/manifest.go`、`internal/app/stageflow/execute.go`、`internal/server/task_finalize.go`、`internal/server/task_run_idempotent_sse_test.go`。 +- **Approach:** + 1. RunManifest 核心字段类型化并持久化规范字节、SHA、workspace/storage identity 和成员列表。 + 2. taskrun terminal finalize transaction 写状态、assistant terminal turn、manifest、artifact staging 和 outbox。 + 3. SSE done 只投影已提交状态；客户端断线与重试复用 task run idempotency key。 + 4. workspace 清理只依赖已冻结 staging 和 policy；需长期保留的最终成果还必须取得 verified receipt。 + 5. backend 由服务端 BackendRef policy 解析，InputBundle 携带数据分类和最小权限范围。 +- **Test scenarios:** + - 同一 finalize 重复调用返回同结果，不重复消息、事件、manifest 或 artifact。 + - finalize 与 cancel 并发只有一个合法终态。 + - SSE 在 commit 前后断线并重试，历史中只有一个 assistant turn。 + - workspace 在 finalize 后立即清理，publisher 仍从 staging 归档原 SHA。 + - manifest digest、storage identity、artifact member 不一致阻止 verified/cleanup。 + - 未授权 backend、敏感 InputBundle、凭证回显和 result 残留全部拒绝或清理。 +- **Verification:** “完成但没有 durable manifest/publish request”和“已显示完成但正文仍模板”两类状态不可产生。 + +### U9. 存量回填、migration corpus 与数据生命周期 + +- **Goal:** 只保留可解释的演示/回归数据，并持续验证旧数据迁移、重载和受控恢复。 +- **Requirements:** R2、R12、R13、R21、R22；KTD8、KTD10。 +- **Dependencies:** U6–U8。 +- **Files:** `cmd/dataset/`、`internal/dataset/`、`internal/artifactvault/backfill.go`、`scripts/backup-system-db.sh`、`scripts/restore-system-db-drill.sh`、`.doc/test-fixtures/`、`.doc/test-reports/`。 +- **Approach:** + 1. 建立 dataset registry，分类 pinned、demo、regression、migration baseline、ephemeral 和 quarantine。 + 2. backfill 只枚举 immutable document snapshot、approved/commit event 和 completed chat turn，禁止扫描 workspace。 + 3. 大表按稳定主键 checkpoint 分批处理；多个 worker 使用 `SKIP LOCKED` 或互斥 shard，限速避免挤占在线连接池。 + 4. 每个 batch 记录 source schema/commit/storage identity、范围、成功、quarantine 和重试水位。 + 5. dev 先蓝绿建立新数据根，导入 6–10 个批准 dataset；旧库/状态根只读 quarantine 7–14 天后，需用户再次批准才销毁。 + 6. 每个 release 验证 clean、oldest-supported、previous、hazard、import、reload、rollback 和 artifact recovery 八条轨道。 +- **Test scenarios:** + - backfill 中断、重复和并发新终态时分母/分子可复算且不重复 artifact。 + - legacy-v0 workspace、旧 schema、高 ID、projection missing/fork、缺 snapshot 分别升级或 quarantine。 + - restore 强制 scratch，拒绝 dev/test/main DSN、错误 storage identity、篡改 manifest 和缺权限凭证。 + - FAV coverage、完整备份 restore 或 migration corpus 任一失败都阻止 destroy。 + - dev 选择性导入后项目关系、文件 SHA、sequence、授权和 reload 一致。 +- **Verification:** 数据清理有精确 manifest、可恢复位置、观察窗口和批准记录；不修改历史 Goose。 + +### U10. chatlog/contextasm 原地端口化 + +- **Goal:** 在不重复付费搬包的前提下分离对话记录和 prompt 上下文装配。 +- **Requirements:** R7、R14、R15、R19、R20；KTD1、KTD9。 +- **Dependencies:** M1–M3 go/no-go；必须等待当前 `chat` mutex 释放。 +- **Files:** `internal/ctxmgr/`、`internal/chat/`、`internal/app/chatflow/`、`internal/ctxmgr/chat_auth_pg_test.go`、`internal/server/chat_history_auth_test.go`。 +- **Approach:** + 1. 先定义 chatflow 所需的消费者侧 ChatLog 和 ContextAssembler ports。 + 2. 在现有 ctxmgr 内提供 adapters，迁移调用者并保持表所有权不变。 + 3. 统计接口 fan-in、DTO 转换和同类改动触达文件数；收益成立后再物理拆为 chatlog/contextasm。 + 4. 所有读取携带 Principal/action，不接受请求体 user/project 作为权限事实。 +- **Test scenarios:** + - 刷新前后同一授权消息集合的正文、分类和计数一致。 + - 跨用户、跨项目、已退出成员和服务端后台导出负例。 + - context assembly 只读取允许集合，摘要/缓存不放大不可见消息。 + - 原地 adapter 与最终拆包实现的 contract tests 等价。 +- **Verification:** 同类对话变更触达面相对基线下降；拆包前已获得独立行为收益。 + +### U11. 组合根迁移、事件替换和共享上下文收权 + +- **Goal:** 让每个跨模块业务流程具有唯一应用用例 owner，而不是集中在 server/bizdev 闭包和共享指针。 +- **Requirements:** R5、R7、R9–R11、R14、R15、R19、R20；KTD1、KTD5、KTD6、KTD9。 +- **Dependencies:** U10；按四个独立子切片执行，不建立一个巨型 Wave。 +- **Files:** `internal/app/app.go`、`internal/app/chatflow/`、`internal/app/stageflow/`、`internal/app/docflow/`、`internal/server/server.go`、`internal/bizdev/service.go`、`internal/coordinator/`、对应 contract/integration tests。 +- **Approach:** + 1. U11a 只把构造和生命周期迁到唯一 app composition root。 + 2. U11b 一次只替换一种闭包事件；每种事件单独迁移、对账和回滚。 + 3. U11c 逐个迁移 ExecuteTask/executeStageInternal 用例，不同时修改无关领域。 + 4. U11d 将 UserProjectContext 改为只读 snapshot + 类型化 ContextUpdate，单一 use case 应用增量。 +- **Test scenarios:** + - app start/stop/restart 不丢 worker，重复启动不创建重复 subscriber。 + - 每个闭包新旧路径 shadow 结果一致，切换后旧路径无调用。 + - ContextUpdate 并发应用使用版本检查，不发生字段级 lost update。 + - server、CLI 和 worker 调用同一 use case 得到一致授权和事务结果。 +- **Verification:** 每个子切片独立发布、独立回滚；server 不再保存业务闭包，UserProjectContext 不被多方直接修改。 + +### U12. 传输边界收口和最终架构门禁 + +- **Goal:** 在领域行为稳定后缩减 server/stream 耦合并全面启用已清零规则。 +- **Requirements:** R14、R18–R21；KTD1、KTD11。 +- **Dependencies:** U11。 +- **Files:** `internal/server/`、`internal/stream/transport/`、`scripts/arch-check.sh`、`.doc/system-design/module-boundaries.yaml`、CI 配置、`web/e2e/`。 +- **Approach:** + 1. server handler 只保留参数、Principal、use case 调用和 wire serialization。 + 2. SSE transport 只编码中性 progress/event frame，不拥有业务状态或重试决策。 + 3. 对已清零的 import、SQL writer、path、producer/consumer 规则逐项转 block。 + 4. 错误信封和 PI Runner 协议保持独立 follow-up，不进入本单元。 +- **Test scenarios:** + - HTTP、SSE 断线重连和 CLI 对同一业务状态观察一致。 + - server 新增领域实现 import、SQL 或路径拼接时 CI 失败。 + - SSE 慢客户端、断开和重复连接不阻塞业务 transaction 或 dispatcher。 + - 全套授权、重启和多实例回归保持通过。 +- **Verification:** R1–R7 的修订版规则全 block；server 行数只作观察指标，不作替代行为验收。 + +--- + +## Verification Contract + +### Evidence layers + +| 层级 | 必须证明 | 不得替代 | +|---|---|---| +| Unit | 状态机、错误分类、schema、路径和 policy | PostgreSQL 原子性 | +| PostgreSQL integration | 事务回滚、锁、幂等、migration、并发 | 浏览器用户体验 | +| Fault injection | 每个 commit/delivery/publish 崩溃窗口 | 真实进程 restart | +| Concurrency/load | 多 app/worker、热点与非热点资源、积压和公平性 | 正确性断言 | +| Real PI | 真实 agent 生成、manifest、finalize、归档 | stub 或固定假输出 | +| Browser | 页面创建/打开/编辑/刷新/重登、错误与 degraded 状态 | API 直接调用 | +| Restart/rebuild | app、worker、workspace root、存储 identity 重载 | health 单点检查 | +| Authorization negative | 匿名/跨用户/跨项目/阶段/duty/后台/recovery | 前端隐藏按钮 | +| Recovery | 完整 DB restore、migration corpus、FAV scratch 恢复 | 仅列出 dump 内容 | + +### Repository gates + +- `go vet ./...` +- `go test -short ./...` +- 目标模块 PostgreSQL 集成和 race 测试；若全包 race 存在已知 TempDir flake，报告必须分离目标模块结果和基建债。 +- `scripts/arch-check.sh` 对当前里程碑已收紧规则为 block，对未迁移规则输出基线差异。 +- `web` 单元测试与 production build。 +- dev 容器必须 rebuild 后检查 `/health` 与 `/ready`，不能用 restart 证明新代码已部署。 +- 真实 PI、真实浏览器、post-restart 和多用户负例按独立证据分栏。 + +### Concurrency test matrix + +| 维度 | 场景 | 必须断言 | +|---|---|---| +| App replicas | 1/2/4 | 结果和不变量一致，进程内 mutex 关闭仍正确 | +| Dispatcher replicas | 1/2/4 | 无双 owner ack，无重复可见副作用，无永久饥饿 | +| Resource shape | 同一文档热点/不同文档并行 | 热点正确冲突，非热点不被全局锁串行化 | +| Failure point | before commit/after commit/before ack/after side effect | 无丢事件，重复交付幂等 | +| Lock behavior | timeout/deadlock/serialization | 分类和完整事务重试正确，无无限等待 | +| Lease behavior | expiry/renew/steal/stale fencing | 旧 owner 无法覆盖新 owner | +| Load | steady/burst/backlog recovery | 记录 p50/p95/p99、锁等待、连接池、oldest event age | + +### Authorization test matrix + +每个状态改变和读取接口至少覆盖：匿名、已登录非成员、错误项目成员、会话非 owner、错误 duty、错误 stage、过期 lease、跨环境 ID、服务身份越权、授权依赖失败和系统管理员合法路径。 +FAV 额外覆盖 manifest project 不符、tombstone、删除账本不一致和授权图缺失。 +状态改变必须覆盖授权检查后的成员撤销、duty 变化、stage 推进和 lease 失效，证明用例在取得资源锁后重新验证当前事实。 + +### Migration and rollback matrix + +| 阶段 | Forward proof | Rollback proof | +|---|---|---| +| Expand | 新 schema 与旧代码兼容 | migration Down 或旧代码继续运行 | +| Shadow | 新旧读取差异为零或有解释的 quarantine | 关闭 flag 回旧路径 | +| Backfill | checkpoint、幂等、并发新写一致 | 停止 worker，不撤销已安全扩展列 | +| Switch | 新 writer 唯一、旧 writer 只读 | flag 回最近兼容 writer | +| Observe | restart/rebuild/浏览器/PI/指标稳定 | 切回旧 reader/writer 与只读 quarantine 数据 | +| Contract | 引用扫描和用户批准 | contract 后只能逆序部署兼容版本，不能单 commit revert | + +--- + +## Risk Analysis and Mitigation + +| 风险 | 触发信号 | 缓解和停止条件 | +|---|---|---| +| contracts 成为新万能包 | fan-in、共享 DTO、ADR 数持续上升 | 消费者侧端口；按域拆公共契约；超过阈值回评边界 | +| 跨属主事务过大 | 锁等待、事务 p99、deadlock 上升 | 缩小原子集合；事务外副作用；固定锁序；停止扩面 | +| advisory lock 形成全局瓶颈 | 非热点资源也排队 | 只对项目级不变量使用；key 分区；检查 `pg_locks` | +| outbox backlog 放大 | oldest age/attempt/dead letter 超阈值 | 独立 worker 连接预算、批次和退避；按 aggregate 并行 | +| poison event 无限重试 | 同 event 高频失败 | typed permanent failure、dead letter、受控 replay | +| FAV 成为授权旁路 | DB 关系缺失仍返回内容 | 正常授权先行、union-deny、默认关闭 degraded read | +| 工作区链接逃逸 | realpath/scope mismatch | opaque ID、descriptor 检查、fencing、负例测试 | +| 回填压垮在线库 | lock wait/replication lag/连接池饱和 | checkpoint、小批次、限速、独立连接预算、可暂停 | +| 双读长期不收口 | exception 数和差异长期不降 | 明确 owner、到期日和 go/no-go；禁止无限兼容 | +| 物理搬包造成大冲突 | import churn 大于行为收益 | 原地端口化；行为稳定后再移动；按 mutex 排队 | +| FAV 分支漂移 | migration 冲突、dev API 已变化 | rebase audit，不整分支盲合；按单元摘取并重测 | +| 数据清理误删 | coverage/backup/corpus 任一未知 | 默认不删；蓝绿迁移；quarantine 观察；用户二次批准 | + +--- + +## Operational and Rollout Notes + +### Feature flags + +- 新 docquery reader、单写路径、dispatcher consumer、FAV degraded read 和 legacy workspace reader 分别使用独立 flag。 +- flag 只选择已兼容路径，不能跳过授权、完整性或终态准入。 +- 每个 flag 有 owner、默认值、环境范围、启用前置和删除里程碑。 + +### Resource budgets + +- app transaction、dispatcher、backfill 和 FAV publisher 使用分离的连接池或明确配额，避免后台积压耗尽在线请求连接。 +- worker batch、lease、heartbeat、retry 和 concurrency 由配置管理；M1/M2 基线报告确定初值和告警阈值。 +- 对同 aggregate 设置最大在途数 1；跨 aggregate 并行度按数据库与 vault 指标调节。 + +以下是首轮试点的安全起始值，不是产品 SLO；压测报告必须记录调整依据： + +| 参数 | 起始值 | 调整约束 | +|---|---|---| +| 写事务 lock wait | 最多 2 秒且不超过请求剩余 deadline | 不允许无限等待；热点冲突优先返回 typed busy | +| serialization/deadlock retry | 最多 3 次，指数退避并带 jitter | 只重跑完整幂等事务 | +| dispatcher claim batch | 50 | claim transaction 只认领，不执行 subscriber | +| delivery lease/heartbeat | 30 秒 / 10 秒 | heartbeat 失败后旧 worker 不得 ack | +| delivery attempts | 12 次后 dead letter | permanent/auth/integrity failure 不做 12 次盲重试 | +| retry backoff ceiling | 15 分钟 | oldest age 仍受告警和人工处置约束 | +| backfill batch | 每批 200 个 aggregate | 按 lock wait、replication lag 和在线池占用自动降速 | +| backfill workers | dev 初始 1，验证后最多 4 | 每个 shard/checkpoint 不重叠 | + +### Observability + +- Dashboard：transaction/lock、document consistency、outbox delivery、workspace lease、FAV publication、authorization deny、backfill/quarantine。 +- Alert：oldest event age、dead letter、storage identity mismatch、projection mismatch、FAV coverage gap、恢复/删除拒绝。 +- Trace 使用 correlation/causation ID 串联 HTTP/use case/transaction/event/subscriber，但不得携带正文或 secret。 + +### Documentation lifecycle + +- 每个里程碑开始前新建 short-term 和 CURRENT claim。 +- 每个里程碑结束生成 test report，回链本计划 U-ID 和验收清单。 +- 完成项移入 archive；未解决问题进入 audit/remediation，不通过口头说明关闭。 + +--- + +## Sources and Research + +- [模块解耦方案复核](../audit/2026-09-14-module-decoupling-plan-review.md)记录当前源码中的事务、outbox、workspace 和授权缺口。 +- `internal/docrevision/pg_mutate.go` 是现有账本与根指针同事务模式，U2–U3 不得降低其原子性。 +- `internal/docduty/revoke.go` 已使用事务级 project advisory lock，是 KTD3 的本地先例。 +- `internal/taskrun/manager.go`、`internal/taskrun/store.go` 已有进程内幂等锁与数据库唯一键，U8 需要把正确性完全落到跨实例可用的数据库协议。 +- `feature/final-artifact-vault` 已实现候选 FAV 能力；U7 必须按当前 dev 重新审计，而不是把分支状态当成当前发布事实。 +- [PostgreSQL `SKIP LOCKED`](https://www.postgresql.org/docs/current/sql-select.html)只用于队列式 claim，不用于普通一致性读取。 +- [PostgreSQL serialization failure handling](https://www.postgresql.org/docs/16/mvcc-serialization-failure-handling.html)要求 serialization/deadlock 重试覆盖完整事务逻辑。 +- [OWASP Business Logic Security](https://cheatsheetseries.owasp.org/cheatsheets/Business_Logic_Security_Cheat_Sheet.html)支持服务端重新推导身份、所有权和安全相关状态。 + +--- + +## Definition of Done + +- [ ] U1–U9 完成并通过 M0–M3 的逐里程碑 go/no-go；U10–U12 只在收益复盘批准后执行。 +- [ ] 文档提交不存在跨表半完成，所有提交后副作用来自持久事件。 +- [ ] 多实例、多 worker、崩溃和 lease 过期场景保持相同业务结果。 +- [ ] workspace 不再是正文或最终 manifest 的唯一事实源，裸路径不跨模块传播。 +- [ ] FAV 只含终态成果，中间结果准入率为零，授权缺失时不可降级读取。 +- [ ] 项目 143 等价 hazard 与新项目完成真实 PI、页面刷新、重登、restart/rebuild 和跨用户负例。 +- [ ] clean/oldest/previous/hazard/import/reload/rollback/artifact recovery 八条迁移轨通过。 +- [ ] arch-check 规则按纵向切片清零后全部 block，例外有 owner 和到期日。 +- [ ] CURRENT、FEATURE-MATRIX、gap-matrix、ADR、系统设计、test report 和 checklist 状态一致。 +- [ ] 放弃方案、临时代码、双写旁路、过期 flags 和无消费者契约已移除，不留在最终 diff。 + +## Related Links + +- Audit: [模块解耦方案复核](../audit/2026-09-14-module-decoupling-plan-review.md) +- Architecture: [模块化单体解耦架构](../system-design/2026-09-14-module-decoupling-architecture.md) +- ADR: [模块化单体解耦](../adr/2026-09-14-modular-monolith-decoupling.md) +- Planning task: [本次计划修订任务](../archive/ops/2026-09-14-module-decoupling-remediation-plan.md) +- Acceptance: [重构验收清单](../ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md) +- FAV dependency: `feature/final-artifact-vault` 中的 `.doc/remediation/2026-09-14-final-artifact-vault-remediation.md` +- Data lifecycle dependency: `feature/final-artifact-vault` 中的 `.doc/remediation/2026-09-14-runtime-data-lifecycle-and-migration-corpus-remediation.md` diff --git a/.doc/remediation/INDEX.md b/.doc/remediation/INDEX.md index d948d641..cfc6e762 100644 --- a/.doc/remediation/INDEX.md +++ b/.doc/remediation/INDEX.md @@ -3,6 +3,7 @@ | 文档 | 说明 | |------|------| | [gap-matrix.md](./gap-matrix.md) | 发布缺口 SSOT | +| [2026-09-14-module-decoupling-refactor-plan.md](./2026-09-14-module-decoupling-refactor-plan.md) | 纵向绞杀、UnitOfWork、高并发互斥、可靠事件、逐动作授权、FAV、迁移与恢复的详细重构计划 | | [2026-09-14-workflow-plan-409-frozen-and-optional-duties-remediation.md](./2026-09-14-workflow-plan-409-frozen-and-optional-duties-remediation.md) | 项目设置 409 冻结态守卫 + 非必需文档三权全有或全无 + 成员/门禁/权限一致性 | | [2026-09-14-controlled-content-canonical-hash-remediation.md](./2026-09-14-controlled-content-canonical-hash-remediation.md) | 项目 143 正文 canonical/raw 哈希漂移、错误语义与重复 hydration 整改 | | [2026-09-13-document-revision-reviewer-duty-sync.md](./2026-09-13-document-revision-reviewer-duty-sync.md) | UI-01 已分配 B=reviewer，受控文档 reviewer_id 硬编码 2，矩阵降为 viewer | diff --git a/.doc/remediation/gap-matrix.md b/.doc/remediation/gap-matrix.md index 249e4218..f9f37351 100644 --- a/.doc/remediation/gap-matrix.md +++ b/.doc/remediation/gap-matrix.md @@ -70,6 +70,7 @@ | DOCREV-ACTIVE-DOC-ROUTING | UI-05「当前受控文档」被法规关键词改路由到 `feasibility_reg`，intake 无 pending | **PASS** | [audit](../audit/2026-09-13-document-revision-active-doc-routing.md)；Round20 UI-05 pending 停在 intake V2 | **不改弱 E2E**；不删证据 | | DOCREV-QUALITY-STATUS-FP | 行动计划表状态列「待续」被门禁当敷衍，两轮 repair 后 quality_failed 80 | **PASS** | [audit](../audit/2026-09-13-document-revision-quality-status-false-positive.md)；Round20 UI-03 | **不删禁词、不放宽正文**；不删证据 | | GOVERNANCE-ACTION-AUTHZ | 文档动作按职责鉴权；未激活 202-blocked；关闭创建直通 | **PASS** | [test-report](../test-reports/2026-09-14-governance-action-level-authz-test-report.md)；[audit 异态](../audit/2026-09-14-planning-with-workflow-key-anomaly.md) | 不改 83 条 planning+key 夹具；none 长 PI 审批链另开 | +| DOC-RELIABILITY-PILOT | 受控文档写路径跨表非同事务、读路径被 projection 绑架、授权 TOCTOU；刷新/重启后可能 404/409/projection-missing | OPEN | [复核](../audit/2026-09-14-module-decoupling-plan-review.md)；[重构计划](./2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md) | M0–M1=U1–U4；未占槽不得 rebuild；M2+ 需独立 go/no-go | --- diff --git a/.doc/status/FEATURE-MATRIX.md b/.doc/status/FEATURE-MATRIX.md index f56e25b4..abec2c5f 100644 --- a/.doc/status/FEATURE-MATRIX.md +++ b/.doc/status/FEATURE-MATRIX.md @@ -27,6 +27,7 @@ | PI Runner / 配置链 | pi-agent + MiniMax 真实链路 | done | [test piagent](../test-reports/2026-09-13-codex-audit-p0-remediation-test-report.md)；[container-env](../test-reports/2026-09-13-container-env-configuration-test-report.md) | — | `pi-agent` | | 三环境运维 | dev/test/main + 共享 KB | done | [PROJECT.md](../../PROJECT.md) §4；四环境 healthy | — | `infra` | | CI / E2E | PG 集成 + 真实 PI E2E | partial | [gap-matrix BUG-08](../remediation/gap-matrix.md)；[test pg-ci](../test-reports/2026-09-12-pg-ci-soak-e2e-test-report.md)；[运行时 UI Round20](../test-reports/2026-09-14-document-revision-runtime-ui-test-report.md) | CI `TEST_DATABASE_URL` 隔离库仍缺；文档运行时真实 UI 已按 UI_PHASE 分文件且 post-restart 断言最终 A1 正文。`-race` 全包并行 TempDir flake 为基建残留 | `ci` | +| 模块解耦与可靠事件 | 文档可靠性纵向试点、跨属主事务、多副本互斥、FAV 与选择性模块化 | planned | [架构](../system-design/2026-09-14-module-decoupling-architecture.md)；[ADR](../adr/2026-09-14-modular-monolith-decoupling.md)；[重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md) | 方案已入库。M0–M1=U1–U4 为首个批准范围；M2+ 需 M1 go/no-go | `docgov` | --- diff --git a/.doc/system-design/2026-09-14-module-decoupling-architecture.md b/.doc/system-design/2026-09-14-module-decoupling-architecture.md new file mode 100644 index 00000000..43444f69 --- /dev/null +++ b/.doc/system-design/2026-09-14-module-decoupling-architecture.md @@ -0,0 +1,422 @@ +# 模块化单体解耦架构 + +> 状态：**设计已修订，首个文档可靠性试点待实施** +> +> 决策：[模块化单体解耦 ADR](../adr/2026-09-14-modular-monolith-decoupling.md) +> +> 实施：[模块解耦整改与重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md) + +## 1. 目标与边界 + +本设计把 Agent-System 收敛为模块化单体，使项目文档在刷新、服务重启、并发修改和工作区清理后仍能从稳定逻辑标识定位到正确正文。 +模块边界必须同时约束依赖、事务、并发、授权、文件和最终成果；只减少 Go import 或移动目录不算解耦完成。 + +### 1.1 目标 + +1. 每个业务命令有唯一应用用例、事务 owner 和幂等语义。 +2. 每张运行表和每类文件有唯一写入属主，但跨表不变量仍能在同一事务提交。 +3. HTTP、SSE、CLI、worker 和恢复工具经过同一授权决策，不信任调用方填写的身份或资源归属。 +4. workspace 只保存计算现场和可重建投影；最终文档、最终回复和 manifest 进入 FAV。 +5. 事件在崩溃、重复投递、多订阅者和水平扩容下可追踪、可重试、可隔离。 +6. 首个纵向试点直接消除受控正文刷新/重启后 404、409 和 projection missing 类故障。 + +### 1.2 非目标 + +- 不拆微服务，不引入分布式事务协调器。 +- 不在本轮统一全部 HTTP 错误响应。 +- 不在本轮替换 PI Runner stdout/result 协议。 +- 不把所有历史数据永久保留，也不把生产数据复制为测试 fixture。 +- 不允许 FAV 绕过数据库授权或自动重建活动工作流、lease、pending task。 + +## 2. 目标架构 + +```mermaid +flowchart TB + Client[Web or CLI] --> Adapter[server adapters] + Worker[Background workers] --> App[application use cases] + Adapter --> App + App --> Authz[authorization policy] + App --> UoW[UnitOfWork] + App --> ReadModels[read use cases] + UoW --> Owners[domain table owners] + Owners --> PG[(PostgreSQL)] + UoW --> Outbox[(outbox and staging)] + Outbox --> Dispatcher[event dispatcher] + Dispatcher --> Subscribers[idempotent subscribers] + Subscribers --> Vault[artifactvault] + App --> Workspace[workspace capability service] + Workspace --> Scratch[(scratch workspace)] + ReadModels --> PG + ReadModels --> Vault +``` + +### 2.1 分层职责 + +| 层 | 职责 | 禁止 | +|---|---|---| +| `internal/server` | 参数解析、认证信息转 Principal、调用应用用例、序列化 HTTP/SSE | SQL、事务、路径拼接、跨模块编排、后台重试 | +| `internal/app/*flow` | 用例编排、授权、UnitOfWork、命令幂等、事件发布 | 领域 SQL、物理路径、传输格式 | +| 消费者侧 ports | 表达一个用例需要的最小能力 | 聚合所有模块方法的万能接口 | +| 领域模块 | 业务不变量、表/文件写入所有权、事务绑定 repository | import 其他领域实现、开启不受控嵌套事务 | +| read model | 授权后的跨模块只读聚合 | 业务状态变更、修复式 GET 写入 | +| 基础设施 | PostgreSQL、CAS、FAV、workspace、事件 dispatcher | 推断业务终态或自行授予权限 | + +### 2.2 契约分类 + +`internal/contracts` 不再作为所有 DTO 的公共仓库。 + +| 契约 | 所有位置 | 版本规则 | +|---|---|---| +| HTTP/SSE wire DTO | `internal/server/api`、`internal/stream/transport` | 对外兼容，breaking change 走 ADR | +| 一对一调用端口 | 消费者 use case 包内 | 随用例演化，不带 JSON tag，不自动走 ADR | +| 公共领域事件 | `internal/contracts/events/` | 有 schema version；生产者和多个消费者共同评审 | +| 持久文件/manifest schema | 所属模块，如 `artifactvault`、`workspace` | major/minor 兼容；未知 major fail closed | +| 稳定共享词汇 | `internal/contracts/vocabulary` | 仅无行为枚举和值对象；禁止依赖业务实现 | +| 错误分类 | 领域错误留领域包；wire 映射留 server | 不把数据库/路径/第三方原始错误暴露给客户端 | + +接口默认由消费者拥有，应用组合根只注入实现。 +只有三个以上消费者确实共享相同语义时，才提升为公共端口。 + +## 3. 模块边界 + +| 模块 | 唯一职责与写入所有权 | 可依赖 | 禁止 | +|---|---|---|---| +| `app/docflow` | 文档读取、提交、批准、投影和归档用例；相关事务 owner | 消费端口、authz、uow | SQL、物理路径 | +| `app/stageflow` | 阶段推进和任务执行用例 | 消费端口、authz、uow | bizdev 门面、HTTP DTO | +| `app/chatflow` | 一轮对话识别、协调、消息落账和任务触发 | 消费端口、authz、uow | 直接写共享上下文字段 | +| `docgov` | 文档治理状态、角色、lease、`document_records` 写者 | 自有 repository、稳定词汇 | 直调 docrevision 实现、读 workspace 文件 | +| `docrevision` | immutable snapshot/version/revision/event/report 账本写者 | 自有 repository | 直接写 `document_records`、`freeform_documents` | +| `freedoc` | 自由文档根表和发布状态写者 | 自有 repository | 被 docrevision 反向更新 | +| `docquery` | 授权后文档列表、正文、版本聚合读取 | 只读端口、artifact read fallback | 任何写入、目录扫描、授权推断 | +| `chatlog` | `chat_*` 表族读写和会话/消息/turn 查询 | 自有 repository | prompt 装配、外部导出策略 | +| `contextasm` | 记忆、摘要、召回和 prompt 上下文组装 | 消费者侧 ChatLog/KB read ports | HTTP、SQL 越界写入 | +| `intent` | 纯规则意图分类 | vocabulary | bizdev、server、IO | +| `taskrun` | task run 状态机、claim/finalize、幂等写者 | 自有 repository | server 补协议、workspace 路径 | +| `squadrun` | 在受限 workspace capability 内执行角色与步骤 | agent/template/quote 消费端口 | DB、SSE、业务授权、任意路径 | +| `workspace` | Workspace ID 签发、物理路径解析、projection 生命周期 | fsutil、storage identity | 业务终态、最终归档 | +| `artifactvault` | 终态 payload/manifest/receipt/tombstone 权威 | 独立存储、registry | 活动业务关系、运行授权、workspace 扫描 | +| `notification` | 通知偏好、投递和状态 | 事件消费、repository | 业务状态推进 | +| `app` | 唯一组合根、worker 生命周期、事件订阅注册 | 所有构造函数 | 被 server 反向装配 | + +模块“可依赖”只表示通过端口调用，不表示允许 import 对方实现包。 + +## 4. 事务与单一写者 + +### 4.1 两种所有权 + +- **表属主**决定 SQL、约束和单表不变量。 +- **事务 owner**决定一个业务命令需要哪些属主共同提交。 + +两者不能合并为“每个模块各开一个事务”。 +应用用例开启 `UnitOfWork`，取得绑定同一 `pgx.Tx` 的 repository capabilities；领域服务只能使用这些 capabilities，不能 commit、rollback 或启动嵌套事务。 + +### 4.2 文档提交原子集合 + +一次受控文档提交必须原子包含： + +1. 校验 Principal、项目成员、duty、lease 和 expected head。 +2. 锁定 `document_records` 与当前 open version。 +3. 写 immutable content blob/snapshot/revision/change event。 +4. 更新版本 head 与 `document_records` 当前指针/哈希。 +5. 冻结应归档 payload 或不可变 blob 引用。 +6. 写领域事件和 FAV publish request。 +7. 提交后再进行 workspace 投影、通知和外部存储发布。 + +任一步失败，步骤 2–6 全部回滚。 +投影或发布失败不会回滚业务提交，而是进入可重试队列并在 UI 暴露 degraded 状态。 + +### 4.3 事务隔离选择 + +| 场景 | 机制 | 原因 | +|---|---|---| +| 单文档编辑/批准/提交 | `READ COMMITTED` + `SELECT ... FOR UPDATE` + expected head | 竞争范围明确，冲突可向用户解释 | +| 同幂等键并发创建 | 唯一约束 + payload digest 校验 | 数据库决定唯一赢家 | +| 跨表项目级 duty/heal | `pg_advisory_xact_lock(project_id)` + 固定行锁顺序 | 复用现有项目级互斥，事务结束自动释放 | +| 复杂跨聚合唯一不变量 | `SERIALIZABLE`，完整事务有限重试 | 仅在显式证明需要时使用 | +| outbox 多 worker claim | `FOR UPDATE SKIP LOCKED` + lease/fencing token | 提高队列吞吐，不作为普通读取语义 | + +### 4.4 锁顺序 + +所有跨属主事务按下列顺序获得锁： + +1. project 或 workflow instance 事务级 advisory/row lock； +2. document/freeform/chat aggregate root； +3. version/snapshot/turn； +4. task run； +5. staging/outbox/delivery。 + +禁止在持锁期间调用 LLM、PI Runner、对象存储、HTTP、SSE 或执行大文件复制。 +需要外部工作时采用 prepare → 外部执行 → compare-and-commit；commit 阶段重新验证 expected version。 + +### 4.5 超时与重试 + +- 每个写事务设置独立 context deadline 和数据库 `lock_timeout`；初始预算由配置管理，不在 handler 硬编码。 +- `40001` 与 `40P01` 只允许重跑完整幂等事务，采用带抖动退避和有限次数。 +- expected-version 冲突、权限拒绝、唯一键 payload 不同不是瞬时错误，不自动重试。 +- 事务重试期间不得重复发送消息、SSE、通知或对象存储写入；这些副作用只能由提交后的 outbox 驱动。 + +## 5. 并发协议 + +### 5.1 文档并发 + +客户端读取正文时获得 `ETag = head_snapshot_id:head_revision_no`。 +PATCH、agent commit、approve 和 projection repair 都必须携带 expected head 或稳定 idempotency key。 +旧 ETag 返回 typed `VERSION_CONFLICT`，响应包含当前 head 元数据但不泄露无权正文。 + +### 5.2 TaskRun 并发 + +`Claim(idempotency_key, request_digest)` 通过唯一约束保证只创建一个 run。 +相同 key、相同 digest 返回既有 run；相同 key、不同 digest 返回 `IDEMPOTENCY_CONFLICT`。 +状态迁移使用 `WHERE status = expected AND version = expected_version`，影响行数为零即重新读取并分类冲突。 +幂等键按 environment、project、actor、action 和客户端 key 作用域化，并与规范化 request digest 绑定。 +幂等记录的保留期不得短于客户端重试窗口、事件重放窗口和对应业务记录的审计期。 + +### 5.3 Workspace 并发 + +Workspace ID 由服务端签发并绑定 environment、project、run、storage identity 和生命周期。 +同一 workspace 同时只有一个写 lease；续租返回 fencing token，所有写操作校验 token。 +本地 `sync.Mutex` 可减少同进程竞争，但跨副本 lease 和 token 才是正确性依据。 +文件写入采用同目录临时文件、校验、原子 rename 和目录同步；reader 只读取完整 projection marker。 + +### 5.4 FAV 并发 + +逻辑 artifact 的幂等键由终态事件、资源、payload SHA 和 policy version 决定。 +对象写入使用 content-addressed key 和 no-replace 语义；blob、manifest 全部校验完成后最后写 commit marker。 +重复 publisher 可共享已有 blob，但只有一个 registry 状态迁移为 verified。 + +### 5.5 查询与背压 + +docquery 的列表和版本历史必须分页，BatchGet 首期硬上限 100 个引用；超过上限返回 typed validation error，不自动拆成无界并发查询。 +批量读取先按 Principal 和 project scope 分组授权，再使用有界并发和数据库 statement deadline。 +API 对 actor/project/action 维度限流，后台 backfill 和 dispatcher 使用独立连接预算，不能耗尽在线读写连接池。 +SSE 慢客户端只影响传输缓冲；不能持有业务 transaction、数据库锁或阻止事件持久化。 + +## 6. 可靠事件 + +### 6.1 事件信封 + +每个持久事件至少包含： + +```text +event_id +event_name +schema_version +environment +project_id +aggregate_type / aggregate_id / aggregate_version +actor_ref +correlation_id / causation_id +idempotency_key +payload_digest +occurred_at +``` + +事件不携带 secret、认证 token、Prompt、正文或任意文件路径。 +大 payload 先冻结到不可变 staging，事件只保存引用和 digest。 + +### 6.2 投递模型 + +```mermaid +sequenceDiagram + participant UC as Use case transaction + participant DB as PostgreSQL + participant W as Dispatcher worker + participant S as Subscriber + UC->>DB: business state + staging + outbox + DB-->>UC: commit + W->>DB: claim delivery with SKIP LOCKED + W->>S: deliver event and fencing token + S->>DB: idempotent side effect and inbox receipt in one transaction + S-->>W: success or typed failure + W->>DB: ack, retry schedule, or dead letter +``` + +- 每个订阅者有独立 delivery 状态，不能共享单一 `published_at`。 +- 投递语义为至少一次；消费者以 event ID/inbox 唯一约束幂等。 +- 同一 aggregate 按 aggregate version 有序；不同 aggregate 可并行。 +- claim 按 `next_attempt_at, event_id` 排序并采用 aging，防止热点失败事件令旧事件永久饥饿。 +- 数据库 subscriber 的业务副作用与 inbox receipt 必须同事务；外部副作用使用业务幂等键并在回执落库前允许安全重放。 +- transient failure 指数退避；永久失败进入 dead letter，不能阻塞其他订阅者。 +- worker 认领带 lease、attempt 和 fencing token；过期 worker 无权提交 ack。 +- replay 必须指定事件范围、目标消费者、操作者和原因，并写审计记录。 +- delivery/outbox 的清理只能在所有必需消费者完成、重放窗口结束且恢复 checkpoint 已覆盖后执行；审计类事件按独立保留策略处理。 + +## 7. Workspace、账本与 FAV + +### 7.1 三层权威 + +| 层 | 权威内容 | 清理规则 | +|---|---|---| +| PostgreSQL | 项目、权限、workflow、task、文档关系、版本账本、当前指针 | 依完整备份/PITR 和环境生命周期处理 | +| workspace/state | 执行现场、临时材料、可重建 projection | 终态冻结和归档门禁通过后按 TTL 清理 | +| FAV | 已归档 payload、manifest、receipt、tombstone | 按 retention/legal hold/delete ledger 处理 | + +### 7.2 Workspace capability + +跨模块 DTO 只包含： + +```text +workspace_id +project_scope +run_id +lease_fencing_token +schema_version +``` + +调用方使用 `ReadArtifact(key)`、`OpenAtomicWriter(key)`、`CommitProjection(key, digest)` 等受限端口。 +artifact key 必须是受控相对标识，禁止绝对路径、`..`、空段、跨作用域 ID、symlink 和 hardlink 逃逸。 + +### 7.3 RunManifest + +RunManifest 的规范字节、SHA、workspace ID、storage identity 和输出成员在 taskrun finalize 时持久化。 +`Evidence map[string]string` 和 `Gaps []string` 不作为核心协议；核心字段类型化，扩展只能进入带 namespace 和 schema 的 extensions。 +workspace 内 manifest 是可重建副本，不是唯一事实源。 + +### 7.4 FAV 接入 + +artifactvault 只接收服务端确认的终态文档、终态助手回复和必要附件。 +流式 chunk、候选、Prompt、工具轨迹、修复过程、临时报告和 workspace 草稿准入率必须为零。 +发布从终态事务的 staging/outbox 驱动，禁止目录扫描。 + +docquery 的读取优先级为： + +1. 校验 Principal 和数据库 resource→project 授权图； +2. 读取数据库 immutable ledger/current pointer； +3. 检查并校验 workspace projection，仅作为缓存； +4. 正文缺失但授权图完整时，可按 feature flag 从 FAV 只读降级； +5. 权限图缺失、vault tombstone、SHA 冲突或来源不一致时 fail closed。 + +GET 不得修复数据库或覆盖 head。 +修复和恢复是独立命令，并产生审计记录。 + +## 8. 授权架构 + +### 8.1 Principal + +Principal 由 HTTP session、服务身份或恢复凭证适配器构造，至少包含 actor ID、actor type、environment、认证来源、request/correlation ID 和明确 capabilities。 +`UserID`、`ProjectID`、role 和 duty 不能直接从业务请求体覆盖 Principal。 + +### 8.2 决策模型 + +授权决策输入为 `Principal + Action + Resource + CurrentState`。 + +| 动作类别 | 强制条件 | +|---|---| +| 文档读取 | 项目成员/管理员 + 分类可见性 + stage/duty 范围 | +| 文档编辑/提交 | 写权限 + 当前 duty + 合法状态 + 有效 lease + expected head | +| 批准/驳回 | 对应 action-level duty + 冻结实例配置 + 当前状态 | +| TaskRun 执行/恢复 | 项目权限 + 合法 workflow/stage + 服务端解析的 backend capability | +| FAV 降级读取 | 正常读取授权已成功 + artifact scope/hash/删除状态一致 | +| replay/backfill | 运维 capability + 精确范围 + dry-run + 审计 | +| scratch recovery promotion | 系统管理员 + 数据责任人确认 + scratch identity + 冲突计划无未决项 | + +应用用例可以在事务前做快速拒绝，但状态改变必须在取得资源锁后重新读取授权相关事实并再次判定，防止成员、duty、stage 或 lease 在检查后变化。 +授权服务不可用或关系不完整时默认拒绝。 +事件携带的 actor 只用于审计；订阅者产生持久或外部副作用前重新读取当前权威状态。 + +### 8.3 后端与数据最小化 + +调用方不能提交任意 `Backend string`。 +应用层按环境、项目策略和输入敏感级别解析 `BackendRef`，agent 适配器再注入受管凭证。 +凭证不得进入 DTO、日志、事件、manifest、result 文件或 FAV。 +InputBundle 只包含完成任务必需的材料，并标记来源、分类、允许用途和生命周期。 + +## 9. 架构门禁 + +### 9.1 静态规则 + +| 规则 | 检查对象 | 例外 | +|---|---|---| +| 领域实现互不 import | Go production packages | 消费端口和稳定 vocabulary | +| server 不 import repository/领域实现 | server production packages | app facade、wire DTO | +| 表写入唯一属主 | production SQL 的 INSERT/UPDATE/DELETE/MERGE | Goose、测试 fixture、批准的修复工具 | +| 工作区物理路径唯一属主 | production path construction/open/write | workspace/squadrun adapter | +| 事件发布者 allowlist | event name → producer package | 无运行时动态注册旁路 | +| 订阅者 allowlist | event name → consumer use case | replay 工具需显式 capability | + +例外必须记录 owner、原因、范围、到期日和移除里程碑。 +warn/block 不按固定“两周”切换；每条规则在对应纵向切片清零后单独转 block。 + +### 9.2 行为门禁 + +架构检查不能替代以下测试： + +- 同文档并发编辑、批准和 agent commit； +- 任意事务步骤失败时全量回滚； +- worker 崩溃、lease 过期、重复投递和多订阅者部分失败； +- workspace 删除、换根、重启和 projection 重建； +- FAV 降级读取与 scratch 恢复； +- 匿名、非成员、跨用户、跨项目、错误 duty、错误 stage、授权依赖失败； +- 真实浏览器刷新/重登/rebuild 与真实 PI 主链。 + +## 10. 可观测性与容量 + +### 10.1 必须指标 + +- 数据库事务时长、锁等待、deadlock、serialization retry、版本冲突率。 +- outbox pending/oldest age、delivery attempts、lease steal、dead letter 数。 +- projection missing/mismatch/repair、FAV degraded read、vault verification lag。 +- 授权 deny 按 action/reason 计数，不记录正文或敏感资源名。 +- workspace active lease、过期清理、fencing rejection、storage identity mismatch。 + +### 10.2 高并发基线 + +首个试点必须建立并保存基线，而不是直接承诺未经测量的容量数字： + +- 同一文档热点写入与不同文档并行写入分开压测。 +- 1、2、4 个 app 实例下验证进程内锁不影响正确性。 +- 1、2、4 个 dispatcher 下验证无重复可见副作用、无饥饿和有界积压。 +- p50/p95/p99 延迟、锁等待、事务重试、CPU/连接池和队列年龄进入 test report。 +- 正确性门禁固定；吞吐阈值在基线评审后写入环境配置与发布 SLO，不能靠降低一致性通过。 + +## 11. 迁移与兼容 + +采用 expand → migrate → contract： + +1. 新增兼容 schema、端口和读路径，不改变旧写入。 +2. 新写路径 shadow/dual-read 验证，禁止无审计双写。 +3. 按 checkpoint 幂等回填，异常进入 quarantine。 +4. 切主写者，保留旧 reader 一个明确发布窗口。 +5. 验证真实刷新/restart/rebuild、FAV coverage 和回滚。 +6. 删除旧路径前再次审计引用和数据差异。 + +旧 workspace 文件缺少 `schema_ver` 时按只读 legacy-v0 解析。 +首次受控 Resume 先校验并原子升级到 v1，保留可回滚备份；未知 major 版本 fail closed。 + +回滚只保证回到最近兼容点。 +已被下游里程碑消费的 schema 或契约必须按逆依赖顺序回滚，不能单独 revert 早期提交。 + +## 12. 首个交付边界 + +首个里程碑只覆盖受控文档可靠性： + +```mermaid +flowchart TB + Read[Authorized content read] --> Ledger[Immutable ledger] + Ledger --> Projection{Projection aligned?} + Projection -->|yes| Return[Return canonical content] + Projection -->|no| Vault{Authorized FAV fallback enabled?} + Vault -->|yes| Degraded[Return verified degraded content] + Vault -->|no| Conflict[Typed consistency error] + Commit[Edit or agent commit] --> Tx[Single UnitOfWork] + Tx --> Ledger + Tx --> Root[Document root pointer] + Tx --> Stage[Artifact staging] + Tx --> Event[Outbox] +``` + +只有这个切片完成刷新、重启、并发、权限和恢复验收后，才启动其他包的物理拆分。 + +## 13. 后续扩展顺序 + +1. 文档可靠性试点。 +2. taskrun finalize、RunManifest 持久化和 FAV 集成。 +3. 逐个事件替换闭包桥。 +4. chatlog/contextasm 原地端口化，再按证据移动包。 +5. squadrun capability 化和 backend policy。 +6. app 组合根迁移。 +7. UserProjectContext 增量收权。 +8. server/SSE 收口。 +9. 错误信封与 PI Runner 协议作为独立项目实施。 + +详细依赖、文件和测试场景见整改计划。 diff --git a/.doc/system-design/README.md b/.doc/system-design/README.md index 651ea9c6..4c88309f 100644 --- a/.doc/system-design/README.md +++ b/.doc/system-design/README.md @@ -99,6 +99,26 @@ --- +### 6. 模块化单体解耦架构 + +| 项 | 链接 | +|-----|------| +| 主报告 | [2026-09-14-module-decoupling-architecture.md](2026-09-14-module-decoupling-architecture.md) | +| ADR | [../adr/2026-09-14-modular-monolith-decoupling.md](../adr/2026-09-14-modular-monolith-decoupling.md) | +| 整改计划 | [../remediation/2026-09-14-module-decoupling-refactor-plan.md](../remediation/2026-09-14-module-decoupling-refactor-plan.md) | +| 一句话总结 | 先以文档可靠性纵向试点验证事务、锁、授权、事件与 FAV，再按证据扩展模块边界 | + +**核心要点**： + +- 单一写者负责 SQL 和不变量，应用 UnitOfWork 负责跨属主原子提交； +- 数据库行锁、事务级 advisory lock、唯一约束和 fencing token 支撑多副本并发； +- workspace 是可清理投影，FAV 只保存已确认终态成果； +- 事件采用逐消费者 delivery、幂等 inbox、租约、重试和 dead letter； +- HTTP、CLI、worker、subscriber 和恢复工具使用同一逐动作授权模型； +- 先原地端口化，行为稳定后才物理搬包。 + +--- + ## 🏗️ 在整体架构中的位置 ``` @@ -161,6 +181,7 @@ ├── 04-项目多工作流与智能产品经理-测试经理工作流-实施计划.md ├── 05-Phase0-基线确认与契约冻结.md └── 06-Phase7-全链路验收与发布基线.md +├── 2026-09-14-module-decoupling-architecture.md ← 模块化单体解耦 .tmp/ ← 中间分析文件 ├── intent-recognition/ From 312578ccd154647f64163c6798def48ddf2b61ff Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 13:49:54 +0000 Subject: [PATCH 02/11] chore(arch): classify production writers and path rules Replace whole-repo grep with four production-only gates so Goose and tests stop looking like writer debt, and so new SQL, path, or event crossings fail immediately. Co-authored-by: Cursor --- .doc/system-design/module-boundaries.yaml | 364 +++++++++++++++ go.mod | 6 +- go.sum | 32 +- internal/archtest/arch_test.go | 32 ++ internal/archtest/arch_test_test.go | 166 +++++++ internal/archtest/boundaries.go | 153 +++++++ internal/archtest/evaluate.go | 82 ++++ internal/archtest/scan.go | 424 ++++++++++++++++++ .../deploy/postgres/goose/00001_fixture.sql | 4 + .../mini/internal/docgov/owner_write.go | 5 + .../mini/internal/docgov/owner_write_test.go | 5 + .../mini/internal/server/event_producer.go | 7 + .../mini/internal/server/path_concat.go | 5 + .../mini/internal/server/path_const.go | 9 + .../mini/internal/server/path_join.go | 7 + .../mini/internal/server/rogue_write.go | 5 + scripts/arch-check.sh | 39 ++ 17 files changed, 1338 insertions(+), 7 deletions(-) create mode 100644 .doc/system-design/module-boundaries.yaml create mode 100644 internal/archtest/arch_test.go create mode 100644 internal/archtest/arch_test_test.go create mode 100644 internal/archtest/boundaries.go create mode 100644 internal/archtest/evaluate.go create mode 100644 internal/archtest/scan.go create mode 100644 internal/archtest/testdata/mini/deploy/postgres/goose/00001_fixture.sql create mode 100644 internal/archtest/testdata/mini/internal/docgov/owner_write.go create mode 100644 internal/archtest/testdata/mini/internal/docgov/owner_write_test.go create mode 100644 internal/archtest/testdata/mini/internal/server/event_producer.go create mode 100644 internal/archtest/testdata/mini/internal/server/path_concat.go create mode 100644 internal/archtest/testdata/mini/internal/server/path_const.go create mode 100644 internal/archtest/testdata/mini/internal/server/path_join.go create mode 100644 internal/archtest/testdata/mini/internal/server/rogue_write.go create mode 100755 scripts/arch-check.sh diff --git a/.doc/system-design/module-boundaries.yaml b/.doc/system-design/module-boundaries.yaml new file mode 100644 index 00000000..56d6db2b --- /dev/null +++ b/.doc/system-design/module-boundaries.yaml @@ -0,0 +1,364 @@ +# U1 architecture baseline. Import allowlist is a 2026-09-14 production go list snapshot. +# SQL write / path / event rules apply only to production Go (not Goose, *_test.go, or fixtures). +# Existing debt lives in exceptions with owner + milestone + expiry. New findings block immediately. + +imports: + agent: + - config + auth: + - config + authz: + - docduty + bizdev: + - authz + - config + - ctxmgr + - db + - docduty + - docgov + - docrevision + - freedoc + - kb + - projectcontext + - projectworkflow + - quote + - squad + - storage + - stream + - workflowdef + chat: + - bizdev + - coordinator + - ctxmgr + - docgov + - intent + - projectinstance + - replyexport + - squad + - stream + - taskrun + config: [] + coordinator: + - agent + - bizdev + - ctxmgr + - docgov + - intent + - kb + - projectinstance + - squad + - stream + ctxmgr: + - docgov + - replyexport + - storage + db: [] + docduty: [] + docgov: + - docduty + - docrevision + - fsutil + docrevision: + - fsutil + freedoc: + - ctxmgr + - docrevision + - replyexport + fsutil: [] + intent: + - bizdev + kb: + - config + notification: [] + observability: [] + projectcontext: + - squad + projectinstance: + - docduty + - docgov + - projectworkflow + - workflowdef + projectworkflow: + - auth + quote: [] + replyexport: [] + server: + - agent + - auth + - authz + - bizdev + - chat + - config + - coordinator + - ctxmgr + - docduty + - docgov + - docrevision + - freedoc + - intent + - notification + - observability + - projectinstance + - projectworkflow + - replyexport + - squad + - stream + - taskrun + - upload + - workflowdef + - workspace + squad: + - agent + - config + - quote + - stream + - template + storage: [] + stream: [] + taskrun: + - bizdev + - ctxmgr + - projectworkflow + - stream + template: [] + upload: [] + workflow: [] + workflow/runtime: + - projectinstance + - workflowdef + workflowdef: + - projectworkflow + - storage + - upload + - workflow + workspace: [] + +table_owners: + document_records: + - docgov + chat_messages: + - ctxmgr + chat_turns: + - ctxmgr + freeform_documents: + - freedoc + task_runs: + - taskrun + document_versions: + - docrevision + document_snapshots: + - docrevision + document_change_events: + - docrevision + +path_owners: + output.md: + - squad + - workspace + data/dev: + - config + - workspace + workspaces/: + - squad + - workspace + +event_producers: + outbox.notify: + - docrevision + document.agent_candidate_ready: + - docrevision + document.agent_version_committed: + - docrevision + document.downstream_impact_detected: + - docrevision + document.change_report_degraded: + - docrevision + +event_consumers: + outbox.notify: + - server + - notification + +exceptions: + - kind: sql_write + package: docduty + file: internal/docduty/revoke.go + symbol: document_records + owner: docduty + reason: duty slot revoke still writes reviewer/approver columns directly; fold into docgov repository during U3 + milestone: U3 + expires: "2026-09-28" + - kind: sql_write + package: docduty + file: internal/docduty/store.go + symbol: document_records + owner: docduty + reason: heal/store updates duty columns on document_records; move behind docgov repository in U3 + milestone: U3 + expires: "2026-09-28" + - kind: sql_write + package: docrevision + file: internal/docrevision/pg_store.go + symbol: document_records + owner: docrevision + reason: projectRootPointersTX updates controlled root pointers; U3 must keep this inside the shared UnitOfWork + milestone: U3 + expires: "2026-09-28" + - kind: sql_write + package: docrevision + file: internal/docrevision/pg_store.go + symbol: freeform_documents + owner: docrevision + reason: projectRootPointersTX also projects freeform heads; keep atomic in U3 then hand write ownership back + milestone: U3 + expires: "2026-09-28" + - kind: sql_write + package: projectinstance + file: internal/projectinstance/store.go + symbol: task_runs + owner: projectinstance + reason: workflow terminate cancels task_runs; U8 owns run finalization in the same UoW + milestone: U8 + expires: "2026-10-26" + - kind: path + package: agent + file: internal/agent/cursor.go + symbol: output.md + owner: agent + reason: runner still names the deliverable file; U5 replaces this with WorkspaceRef + milestone: U5 + expires: "2026-10-12" + - kind: path + package: agent + file: internal/agent/piagent.go + symbol: output.md + owner: agent + reason: PI runner contract still uses output.md; U5 capability must hide the literal + milestone: U5 + expires: "2026-10-12" + - kind: path + package: bizdev + file: internal/bizdev/service.go + symbol: data/dev + owner: bizdev + reason: default data-root example string; U5 resolver owns environment roots + milestone: U5 + expires: "2026-10-12" + - kind: path + package: bizdev + file: internal/bizdev/service.go + symbol: output.md + owner: bizdev + reason: orchestration still joins workspace output.md; U5 removes Root/OutputDir + milestone: U5 + expires: "2026-10-12" + - kind: path + package: coordinator + file: internal/coordinator/context.go + symbol: output.md + owner: coordinator + reason: coordinator reads projection path; U4/U5 must go through workspace capability + milestone: U5 + expires: "2026-10-12" + - kind: path + package: coordinator + file: internal/coordinator/dialogue_llm.go + symbol: data/dev + owner: coordinator + reason: prompt example embeds data/dev; U5 must not leak runtime roots + milestone: U5 + expires: "2026-10-12" + - kind: path + package: coordinator + file: internal/coordinator/document_summary.go + symbol: data/dev + owner: coordinator + reason: summary prompt embeds data/dev; U5 must not leak runtime roots + milestone: U5 + expires: "2026-10-12" + - kind: path + package: coordinator + file: internal/coordinator/engine.go + symbol: output.md + owner: coordinator + reason: engine joins deliverable path; U5 WorkspaceRef + milestone: U5 + expires: "2026-10-12" + - kind: path + package: docgov + file: internal/docgov/lease.go + symbol: output.md + owner: docgov + reason: SaveDocumentContent still writes projection files; U5 capability + U4 ledger-first read + milestone: U5 + expires: "2026-10-12" + - kind: path + package: docgov + file: internal/docgov/project_docs.go + symbol: output.md + owner: docgov + reason: project doc helpers still construct output.md; U5 + milestone: U5 + expires: "2026-10-12" + - kind: path + package: docrevision + file: internal/docrevision/projection.go + symbol: output.md + owner: docrevision + reason: InspectProjection reads workspace file; U4 treats this as optional cache only + milestone: U4 + expires: "2026-09-28" + - kind: path + package: docrevision + file: internal/docrevision/reconcile.go + symbol: output.md + owner: docrevision + reason: reconcile still repairs projection files; U5 + milestone: U5 + expires: "2026-10-12" + - kind: path + package: server + file: internal/server/document_revision_access.go + symbol: output.md + owner: server + reason: handler still reads workspace files; U4 must use docquery ledger + milestone: U4 + expires: "2026-09-28" + - kind: path + package: server + file: internal/server/document_revision_handlers.go + symbol: output.md + owner: server + reason: handler error text mentions output.md; U4/U5 remove path leakage + milestone: U4 + expires: "2026-09-28" + - kind: path + package: server + file: internal/server/server.go + symbol: workspaces/ + owner: server + reason: server still concatenates workspaces/ roots; U5 + milestone: U5 + expires: "2026-10-12" + - kind: path + package: server + file: internal/server/task_finalize.go + symbol: output.md + owner: server + reason: finalize still joins output.md; U8 verified receipt + milestone: U8 + expires: "2026-10-26" + - kind: path + package: storage + file: internal/storage/cas.go + symbol: data/dev + owner: storage + reason: CAS default example path; U5 resolver + milestone: U5 + expires: "2026-10-12" + +baseline_counts: + import: 0 + sql_write: 5 + path: 17 + event: 0 diff --git a/go.mod b/go.mod index e64429ef..0e920a39 100644 --- a/go.mod +++ b/go.mod @@ -4,9 +4,12 @@ go 1.27.1 require ( github.com/jackc/pgx/v5 v5.10.0 + github.com/pressly/goose/v3 v3.24.1 github.com/stretchr/testify v1.11.1 github.com/xuri/excelize/v2 v2.11.0 golang.org/x/crypto v0.53.0 + golang.org/x/text v0.42.0 + gopkg.in/yaml.v3 v3.0.1 ) require ( @@ -17,7 +20,6 @@ require ( github.com/kr/text v0.2.0 // indirect github.com/mfridman/interpolate v0.0.2 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect - github.com/pressly/goose/v3 v3.24.1 // indirect github.com/richardlehane/mscfb v1.0.7 // indirect github.com/richardlehane/msoleps v1.0.6 // indirect github.com/rogpeppe/go-internal v1.16.0 // indirect @@ -28,6 +30,4 @@ require ( go.uber.org/multierr v1.11.0 // indirect golang.org/x/net v0.56.0 // indirect golang.org/x/sync v0.23.0 // indirect - golang.org/x/text v0.42.0 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/go.sum b/go.sum index f06b3aee..0653a18e 100644 --- a/go.sum +++ b/go.sum @@ -2,6 +2,12 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM= github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg= github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo= @@ -14,12 +20,18 @@ github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/mfridman/interpolate v0.0.2 h1:pnuTK7MQIxxFz1Gr+rjSIx9u7qVjf5VOoM/u6BbAxPY= github.com/mfridman/interpolate v0.0.2/go.mod h1:p+7uk6oE07mpE/Ik1b8EckO0O4ZXiGAfshKBWLUM9Xg= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pressly/goose/v3 v3.24.1 h1:bZmxRco2uy5uu5Ng1MMVEfYsFlrMJI+e/VMXHQ3C4LY= github.com/pressly/goose/v3 v3.24.1/go.mod h1:rEWreU9uVtt0DHCyLzF9gRcWiiTF/V+528DV+4DORug= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/richardlehane/mscfb v1.0.7 h1:oeoiM0WE79vHwE8RpIYYvIAc8ajTH2mb6UZm55/+EB0= github.com/richardlehane/mscfb v1.0.7/go.mod h1:pe0+IUIc0AHh0+teNzBlJCtSyZdFOGgV4ZK9bsoV+Jo= github.com/richardlehane/msoleps v1.0.6 h1:9BvkpjvD+iUBalUY4esMwv6uBkfOip/Lzvd93jvR9gg= @@ -49,12 +61,10 @@ golang.org/x/image v0.38.0 h1:5l+q+Y9JDC7mBOMjo4/aPhMDcxEptsX+Tt3GgRQRPuE= golang.org/x/image v0.38.0/go.mod h1:/3f6vaXC+6CEanU4KJxbcUZyEePbyKbaLoDOe4ehFYY= golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o= golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec= -golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= -golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk= golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= -golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE= -golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4= +golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw= +golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI= golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= @@ -63,3 +73,17 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EV gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4= +modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U= +modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w= +modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4= +modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo= +modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E= +modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU= +modernc.org/sqlite v1.34.1 h1:u3Yi6M0N8t9yKRDwhXcyp1eS5/ErhPTBggxWFuR6Hfk= +modernc.org/sqlite v1.34.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k= +modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA= +modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/internal/archtest/arch_test.go b/internal/archtest/arch_test.go new file mode 100644 index 00000000..eb414381 --- /dev/null +++ b/internal/archtest/arch_test.go @@ -0,0 +1,32 @@ +package archtest + +import ( + "os" + "testing" + "time" +) + +func TestArchitectureBoundaries(t *testing.T) { + root, err := ModuleRoot() + if err != nil { + t.Fatal(err) + } + b, err := LoadBoundaries(DefaultBoundariesPath(root)) + if err != nil { + t.Fatal(err) + } + findings, err := Scan(ScanOptions{Root: root}, b) + if err != nil { + t.Fatal(err) + } + ev := Evaluate(findings, b, time.Now()) + if os.Getenv("ARCHTEST_DUMP") != "" { + t.Log(ev.Report()) + } + if ev.Failed() { + t.Fatal(ev.Report()) + } + if os.Getenv("ARCHTEST_STRICT") != "" && len(ev.Excepted) > 0 { + t.Fatalf("strict/block mode: %d baseline exceptions remain\n%s", len(ev.Excepted), ev.Report()) + } +} diff --git a/internal/archtest/arch_test_test.go b/internal/archtest/arch_test_test.go new file mode 100644 index 00000000..e453a3aa --- /dev/null +++ b/internal/archtest/arch_test_test.go @@ -0,0 +1,166 @@ +package archtest + +import ( + "path/filepath" + "runtime" + "testing" + "time" +) + +func miniRoot(t *testing.T) string { + t.Helper() + _, file, _, ok := runtime.Caller(0) + if !ok { + t.Fatal("caller") + } + return filepath.Join(filepath.Dir(file), "testdata", "mini") +} + +func fixtureBoundaries() *Boundaries { + return &Boundaries{ + Imports: map[string][]string{ + "server": {}, + "docgov": {}, + }, + TableOwners: map[string][]string{ + "document_records": {"docgov"}, + }, + PathOwners: map[string][]string{ + "output.md": {"squad", "workspace"}, + }, + EventProducers: map[string][]string{ + "outbox.notify": {"docrevision"}, + "document.agent_version_committed": {"docrevision"}, + }, + EventConsumers: map[string][]string{ + "outbox.notify": {"notification"}, + }, + BaselineCounts: map[string]int{}, + } +} + +func scanMini(t *testing.T) []Finding { + t.Helper() + findings, err := Scan(ScanOptions{Root: miniRoot(t)}, fixtureBoundaries()) + if err != nil { + t.Fatal(err) + } + return findings +} + +func TestGooseAndTestsAreNotProductionWriters(t *testing.T) { + for _, f := range scanMini(t) { + if f.Kind == KindSQLWrite && f.Package == "docgov" && f.File == "internal/docgov/owner_write_test.go" { + t.Fatalf("test file must not be a production writer: %+v", f) + } + if f.Kind == KindSQLWrite && filepath.Ext(f.File) == ".sql" { + t.Fatalf("goose/sql must not be a production writer: %+v", f) + } + } +} + +func TestNonOwnerProductionWriteIsBlockedAndNamed(t *testing.T) { + var hit *Finding + findings := scanMini(t) + for i, f := range findings { + if f.Kind == KindSQLWrite && f.Symbol == "document_records" && f.Package == "server" { + hit = &findings[i] + if f.File != "internal/server/rogue_write.go" { + t.Fatalf("expected rogue_write.go, got %s", f.File) + } + } + } + if hit == nil { + t.Fatal("expected UPDATE document_records in internal/server/rogue_write.go to be blocked") + } +} + +func TestOwnerProductionWriteIsAllowed(t *testing.T) { + for _, f := range scanMini(t) { + if f.Kind == KindSQLWrite && f.Package == "docgov" && f.File == "internal/docgov/owner_write.go" { + t.Fatalf("owner write should be allowed: %+v", f) + } + } +} + +func TestPathJoinConcatAndConstAreDetected(t *testing.T) { + want := map[string]bool{ + "internal/server/path_join.go": false, + "internal/server/path_concat.go": false, + "internal/server/path_const.go": false, + } + for _, f := range scanMini(t) { + if f.Kind != KindPath || f.Symbol != "output.md" { + continue + } + if _, ok := want[f.File]; ok { + want[f.File] = true + } + } + for file, seen := range want { + if !seen { + t.Errorf("path construction not detected in %s", file) + } + } +} + +func TestUnauthorizedEventProducerIsBlocked(t *testing.T) { + found := false + for _, f := range scanMini(t) { + if f.Kind == KindEvent && f.Package == "server" { + found = true + } + } + if !found { + t.Fatal("expected unauthorized OnOutbox producer in server to be blocked") + } +} + +func TestExceptionRequiresExpiryAndOwner(t *testing.T) { + b := fixtureBoundaries() + b.Exceptions = []Exception{{ + Kind: KindSQLWrite, Package: "server", File: "internal/server/rogue_write.go", + Symbol: "document_records", Reason: "no owner or expiry", + }} + ev := Evaluate(scanMini(t), b, time.Date(2026, 9, 14, 12, 0, 0, 0, time.UTC)) + if len(ev.BadExceptions) == 0 { + t.Fatal("expected missing owner/expires to block") + } + if len(ev.Unmatched) == 0 { + t.Fatal("invalid exception must not hide the finding") + } +} + +func TestExpiredExceptionBlocksImmediately(t *testing.T) { + b := fixtureBoundaries() + b.Exceptions = []Exception{{ + Kind: KindSQLWrite, Package: "server", File: "internal/server/rogue_write.go", + Symbol: "document_records", Owner: "server", Reason: "stale", + Milestone: "U3", Expires: "2026-09-01", + }} + ev := Evaluate(scanMini(t), b, time.Date(2026, 9, 14, 12, 0, 0, 0, time.UTC)) + if len(ev.BadExceptions) == 0 { + t.Fatal("expected expired exception to block") + } + found := false + for _, f := range ev.Unmatched { + if f.File == "internal/server/rogue_write.go" { + found = true + } + } + if !found { + t.Fatal("expired exception must surface the original finding") + } +} + +func TestClassifyGooseSQL(t *testing.T) { + root := miniRoot(t) + got := classifyFile(root, filepath.Join(root, "deploy", "postgres", "goose", "00001_fixture.sql")) + if got != classGoose { + t.Fatalf("classify goose: got %v want %v", got, classGoose) + } + got = classifyFile(root, filepath.Join(root, "internal", "docgov", "owner_write_test.go")) + if got != classTestOrFixture { + t.Fatalf("classify test: got %v want %v", got, classTestOrFixture) + } +} diff --git a/internal/archtest/boundaries.go b/internal/archtest/boundaries.go new file mode 100644 index 00000000..cf8d91b5 --- /dev/null +++ b/internal/archtest/boundaries.go @@ -0,0 +1,153 @@ +package archtest + +import ( + "fmt" + "os" + "path/filepath" + "runtime" + "strings" + "time" + + "gopkg.in/yaml.v3" +) + +const ( + KindImport = "import" + KindSQLWrite = "sql_write" + KindPath = "path" + KindEvent = "event" +) + +type Boundaries struct { + Imports map[string][]string `yaml:"imports"` + TableOwners map[string][]string `yaml:"table_owners"` + PathOwners map[string][]string `yaml:"path_owners"` + EventProducers map[string][]string `yaml:"event_producers"` + EventConsumers map[string][]string `yaml:"event_consumers"` + Exceptions []Exception `yaml:"exceptions"` + BaselineCounts map[string]int `yaml:"baseline_counts"` +} + +type Exception struct { + Kind string `yaml:"kind"` + Package string `yaml:"package"` + File string `yaml:"file"` + Symbol string `yaml:"symbol"` + Owner string `yaml:"owner"` + Reason string `yaml:"reason"` + Milestone string `yaml:"milestone"` + Expires string `yaml:"expires"` +} + +type Finding struct { + Kind string + Package string + File string + Symbol string + Detail string +} + +func (f Finding) Key() string { + return f.Kind + "|" + f.Package + "|" + filepath.ToSlash(f.File) + "|" + f.Symbol +} + +func ModuleRoot() (string, error) { + _, file, _, ok := runtime.Caller(0) + if !ok { + return "", fmt.Errorf("archtest: cannot locate caller") + } + dir := filepath.Dir(file) + for { + if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil { + return dir, nil + } + parent := filepath.Dir(dir) + if parent == dir { + return "", fmt.Errorf("archtest: go.mod not found") + } + dir = parent + } +} + +func LoadBoundaries(path string) (*Boundaries, error) { + b, err := os.ReadFile(path) + if err != nil { + return nil, err + } + var out Boundaries + if err := yaml.Unmarshal(b, &out); err != nil { + return nil, err + } + if out.Imports == nil { + out.Imports = map[string][]string{} + } + if out.TableOwners == nil { + out.TableOwners = map[string][]string{} + } + if out.PathOwners == nil { + out.PathOwners = map[string][]string{} + } + if out.EventProducers == nil { + out.EventProducers = map[string][]string{} + } + if out.EventConsumers == nil { + out.EventConsumers = map[string][]string{} + } + if out.BaselineCounts == nil { + out.BaselineCounts = map[string]int{} + } + return &out, nil +} + +func DefaultBoundariesPath(root string) string { + return filepath.Join(root, ".doc", "system-design", "module-boundaries.yaml") +} + +func allowedHas(list []string, needle string) bool { + for _, p := range list { + if strings.TrimSpace(p) == needle { + return true + } + } + return false +} + +func (e Exception) validate(now time.Time) error { + if strings.TrimSpace(e.Owner) == "" { + return fmt.Errorf("missing owner") + } + if strings.TrimSpace(e.Reason) == "" { + return fmt.Errorf("missing reason") + } + if strings.TrimSpace(e.Milestone) == "" { + return fmt.Errorf("missing milestone") + } + if strings.TrimSpace(e.Expires) == "" { + return fmt.Errorf("missing expires") + } + day, err := time.ParseInLocation("2006-01-02", e.Expires, time.UTC) + if err != nil { + return fmt.Errorf("invalid expires %q", e.Expires) + } + // expires is inclusive through that calendar day (UTC). + if !now.UTC().Before(day.Add(24 * time.Hour)) { + return fmt.Errorf("expired on %s", e.Expires) + } + return nil +} + +func (e Exception) matches(f Finding) bool { + if e.Kind != f.Kind { + return false + } + if e.Package != "" && e.Package != f.Package { + return false + } + if e.File != "" && filepath.ToSlash(e.File) != filepath.ToSlash(f.File) { + return false + } + if e.Symbol != "" && e.Symbol != f.Symbol { + return false + } + return true +} diff --git a/internal/archtest/evaluate.go b/internal/archtest/evaluate.go new file mode 100644 index 00000000..358b7953 --- /dev/null +++ b/internal/archtest/evaluate.go @@ -0,0 +1,82 @@ +package archtest + +import ( + "fmt" + "sort" + "strings" + "time" +) + +type Evaluation struct { + Unmatched []Finding + Excepted []Finding + BadExceptions []string + Counts map[string]int + CountRegressed []string +} + +func Evaluate(findings []Finding, b *Boundaries, now time.Time) Evaluation { + ev := Evaluation{Counts: map[string]int{}} + used := make([]bool, len(b.Exceptions)) + for i, ex := range b.Exceptions { + if err := ex.validate(now); err != nil { + ev.BadExceptions = append(ev.BadExceptions, fmt.Sprintf("exception[%d] %s %s %s: %v", i, ex.Kind, ex.File, ex.Symbol, err)) + } + } + for _, f := range findings { + ev.Counts[f.Kind]++ + matched := false + for i, ex := range b.Exceptions { + if !ex.matches(f) { + continue + } + if err := ex.validate(now); err != nil { + continue + } + used[i] = true + matched = true + ev.Excepted = append(ev.Excepted, f) + break + } + if !matched { + ev.Unmatched = append(ev.Unmatched, f) + } + } + sort.Slice(ev.Unmatched, func(i, j int) bool { return ev.Unmatched[i].Key() < ev.Unmatched[j].Key() }) + sort.Slice(ev.Excepted, func(i, j int) bool { return ev.Excepted[i].Key() < ev.Excepted[j].Key() }) + + for kind, baseline := range b.BaselineCounts { + if ev.Counts[kind] > baseline { + ev.CountRegressed = append(ev.CountRegressed, fmt.Sprintf("%s: %d > baseline %d", kind, ev.Counts[kind], baseline)) + } + } + sort.Strings(ev.CountRegressed) + return ev +} + +func (ev Evaluation) Failed() bool { + return len(ev.Unmatched) > 0 || len(ev.BadExceptions) > 0 || len(ev.CountRegressed) > 0 +} + +func (ev Evaluation) Report() string { + var b strings.Builder + fmt.Fprintf(&b, "archtest findings=%d excepted=%d unmatched=%d\n", + ev.Counts[KindImport]+ev.Counts[KindSQLWrite]+ev.Counts[KindPath]+ev.Counts[KindEvent], + len(ev.Excepted), len(ev.Unmatched)) + for _, kind := range []string{KindImport, KindSQLWrite, KindPath, KindEvent} { + fmt.Fprintf(&b, " %s=%d\n", kind, ev.Counts[kind]) + } + for _, msg := range ev.BadExceptions { + fmt.Fprintf(&b, "BAD_EXCEPTION %s\n", msg) + } + for _, msg := range ev.CountRegressed { + fmt.Fprintf(&b, "REGRESSION %s\n", msg) + } + for _, f := range ev.Unmatched { + fmt.Fprintf(&b, "NEW %s %s %s %s (%s)\n", f.Kind, f.Package, f.File, f.Symbol, f.Detail) + } + for _, f := range ev.Excepted { + fmt.Fprintf(&b, "BASELINE %s %s %s %s (%s)\n", f.Kind, f.Package, f.File, f.Symbol, f.Detail) + } + return b.String() +} diff --git a/internal/archtest/scan.go b/internal/archtest/scan.go new file mode 100644 index 00000000..6b89c219 --- /dev/null +++ b/internal/archtest/scan.go @@ -0,0 +1,424 @@ +package archtest + +import ( + "fmt" + "go/ast" + "go/parser" + "go/token" + "os" + "path/filepath" + "regexp" + "strings" +) + +const internalPrefix = "chat-agent-system/internal/" + +var ( + sqlWriteRe = regexp.MustCompile(`(?is)\b(INSERT\s+INTO|UPDATE|DELETE\s+FROM)\s+([a-zA-Z_][a-zA-Z0-9_]*)`) + eventNameRe = regexp.MustCompile(`document\.[a-z0-9_.]+`) +) + +type fileClass int + +const ( + classIgnore fileClass = iota + classProduction + classTestOrFixture + classGoose +) + +type ScanOptions struct { + Root string + // ExtraRoots are additional trees scanned as production Go (used by self-tests). + ExtraRoots []string + // ExtraSQL are extra files classified as goose/fixture SQL. + ExtraSQL []string +} + +func Scan(opts ScanOptions, b *Boundaries) ([]Finding, error) { + var findings []Finding + roots := append([]string{filepath.Join(opts.Root, "internal")}, opts.ExtraRoots...) + fset := token.NewFileSet() + for _, root := range roots { + if root == "" { + continue + } + if _, err := os.Stat(root); err != nil { + continue + } + err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() { + name := d.Name() + if name == "testdata" || name == "vendor" || name == ".git" { + return filepath.SkipDir + } + return nil + } + class := classifyFile(opts.Root, path) + if strings.HasSuffix(path, ".go") { + switch class { + case classProduction: + fs, perr := scanProductionGo(fset, opts.Root, path, b) + if perr != nil { + return perr + } + findings = append(findings, fs...) + case classTestOrFixture, classGoose, classIgnore: + // Goose / tests / fixtures may mention tables and paths; they are not production writers. + } + } + return nil + }) + if err != nil { + return nil, err + } + } + for _, sqlPath := range opts.ExtraSQL { + // Explicit proof that goose SQL is classified away from production writes. + if classifyFile(opts.Root, sqlPath) != classGoose && !strings.Contains(filepath.ToSlash(sqlPath), "goose") { + continue + } + } + return dedupe(findings), nil +} + +func classifyFile(repoRoot, path string) fileClass { + rel := path + if repoRoot != "" { + if r, err := filepath.Rel(repoRoot, path); err == nil { + rel = r + } + } + slash := filepath.ToSlash(rel) + base := filepath.Base(path) + if strings.Contains(slash, "/testdata/") || strings.HasPrefix(slash, "testdata/") { + if strings.Contains(slash, "/goose/") || strings.HasPrefix(base, "000") && strings.HasSuffix(base, ".sql") { + return classGoose + } + if strings.HasSuffix(base, "_test.go") || strings.Contains(slash, "/fixture") || strings.Contains(slash, "/fixtures/") { + if strings.HasSuffix(base, ".go") && !strings.HasSuffix(base, "_test.go") && strings.Contains(slash, "/prod/") { + return classProduction + } + return classTestOrFixture + } + } + if strings.HasPrefix(slash, "deploy/postgres/goose/") || strings.Contains(slash, "/postgres/goose/") { + return classGoose + } + if strings.HasSuffix(base, "_test.go") { + return classTestOrFixture + } + if strings.HasSuffix(base, ".go") && strings.HasPrefix(slash, "internal/") { + if strings.HasPrefix(slash, "internal/archtest/") { + return classIgnore + } + return classProduction + } + if strings.HasSuffix(base, ".go") && (strings.Contains(slash, "/prod/") || strings.Contains(slash, "fixtures/prod/")) { + return classProduction + } + return classIgnore +} + +func scanProductionGo(fset *token.FileSet, repoRoot, path string, b *Boundaries) ([]Finding, error) { + src, err := os.ReadFile(path) + if err != nil { + return nil, err + } + file, err := parser.ParseFile(fset, path, src, parser.SkipObjectResolution) + if err != nil { + return nil, fmt.Errorf("parse %s: %w", path, err) + } + pkg := packageID(repoRoot, path) + rel := relFile(repoRoot, path) + var out []Finding + out = append(out, scanImports(file, pkg, rel, b)...) + out = append(out, scanSQLWrites(string(src), pkg, rel, b)...) + out = append(out, scanPaths(file, pkg, rel, b)...) + out = append(out, scanEvents(file, string(src), pkg, rel, b)...) + return out, nil +} + +func packageID(repoRoot, path string) string { + rel := relFile(repoRoot, path) + slash := filepath.ToSlash(rel) + slash = strings.TrimPrefix(slash, "internal/") + dir := filepath.ToSlash(filepath.Dir(slash)) + if dir == "." { + return filepath.Base(filepath.Dir(path)) + } + return dir +} + +func relFile(repoRoot, path string) string { + if repoRoot == "" { + return filepath.ToSlash(path) + } + rel, err := filepath.Rel(repoRoot, path) + if err != nil { + return filepath.ToSlash(path) + } + return filepath.ToSlash(rel) +} + +func scanImports(file *ast.File, pkg, rel string, b *Boundaries) []Finding { + allowed, ok := b.Imports[pkg] + if !ok { + // Unknown production package is itself a new import-graph node. + return []Finding{{Kind: KindImport, Package: pkg, File: rel, Symbol: pkg, Detail: "package missing from import allowlist"}} + } + var out []Finding + for _, spec := range file.Imports { + path := strings.Trim(spec.Path.Value, `"`) + if !strings.HasPrefix(path, internalPrefix) { + continue + } + dep := strings.TrimPrefix(path, internalPrefix) + if dep == pkg { + continue + } + if !allowedHas(allowed, dep) { + out = append(out, Finding{ + Kind: KindImport, + Package: pkg, + File: rel, + Symbol: dep, + Detail: fmt.Sprintf("production import %s -> %s", pkg, dep), + }) + } + } + return out +} + +func scanSQLWrites(src, pkg, rel string, b *Boundaries) []Finding { + var out []Finding + for _, m := range sqlWriteRe.FindAllStringSubmatch(src, -1) { + table := strings.ToLower(m[2]) + owners, tracked := b.TableOwners[table] + if !tracked { + continue + } + if allowedHas(owners, pkg) { + continue + } + out = append(out, Finding{ + Kind: KindSQLWrite, + Package: pkg, + File: rel, + Symbol: table, + Detail: fmt.Sprintf("%s writes %s", strings.ToLower(strings.Join(strings.Fields(m[1]), " ")), table), + }) + } + return out +} + +func scanPaths(file *ast.File, pkg, rel string, b *Boundaries) []Finding { + consts := map[string]string{} + ast.Inspect(file, func(n ast.Node) bool { + vs, ok := n.(*ast.ValueSpec) + if !ok { + return true + } + for i, name := range vs.Names { + if i < len(vs.Values) { + if lit, ok := vs.Values[i].(*ast.BasicLit); ok && lit.Kind == token.STRING { + consts[name.Name] = strings.Trim(lit.Value, `"`) + } + } + } + return true + }) + + var out []Finding + seen := map[string]bool{} + add := func(symbol, detail string) { + key := symbol + "|" + detail + if seen[key] { + return + } + seen[key] = true + owners, tracked := b.PathOwners[symbol] + if !tracked { + return + } + if allowedHas(owners, pkg) { + return + } + out = append(out, Finding{ + Kind: KindPath, + Package: pkg, + File: rel, + Symbol: symbol, + Detail: detail, + }) + } + + ast.Inspect(file, func(n ast.Node) bool { + switch x := n.(type) { + case *ast.BasicLit: + if x.Kind != token.STRING { + return true + } + val := strings.Trim(x.Value, `"`) + for lit := range b.PathOwners { + if pathLiteralHit(val, lit) { + add(lit, "string literal "+lit) + } + } + case *ast.Ident: + if val, ok := consts[x.Name]; ok { + for lit := range b.PathOwners { + if pathLiteralHit(val, lit) { + add(lit, "indirect const "+x.Name) + } + } + } + case *ast.CallExpr: + if !isJoinLike(x.Fun) { + return true + } + for _, arg := range x.Args { + switch a := arg.(type) { + case *ast.BasicLit: + if a.Kind == token.STRING { + val := strings.Trim(a.Value, `"`) + for lit := range b.PathOwners { + if pathLiteralHit(val, lit) { + add(lit, "filepath.Join/path.Join") + } + } + } + case *ast.Ident: + if val, ok := consts[a.Name]; ok { + for lit := range b.PathOwners { + if pathLiteralHit(val, lit) { + add(lit, "filepath.Join via const "+a.Name) + } + } + } + } + } + case *ast.BinaryExpr: + if x.Op != token.ADD { + return true + } + vals := []string{litOrConst(x.X, consts), litOrConst(x.Y, consts)} + for _, val := range vals { + for lit := range b.PathOwners { + if pathLiteralHit(val, lit) { + add(lit, "string concatenation") + } + } + } + } + return true + }) + return out +} + +func litOrConst(e ast.Expr, consts map[string]string) string { + switch x := e.(type) { + case *ast.BasicLit: + if x.Kind == token.STRING { + return strings.Trim(x.Value, `"`) + } + case *ast.Ident: + return consts[x.Name] + } + return "" +} + +func isJoinLike(fun ast.Expr) bool { + sel, ok := fun.(*ast.SelectorExpr) + if !ok { + return false + } + pkg, ok := sel.X.(*ast.Ident) + if !ok { + return false + } + if sel.Sel.Name != "Join" && sel.Sel.Name != "FromSlash" { + return false + } + return pkg.Name == "filepath" || pkg.Name == "path" +} + +func pathLiteralHit(val, lit string) bool { + if val == "" || lit == "" { + return false + } + if val == lit { + return true + } + return strings.Contains(val, lit) +} + +func scanEvents(file *ast.File, src, pkg, rel string, b *Boundaries) []Finding { + var out []Finding + seen := map[string]bool{} + add := func(symbol, role, detail string) { + key := symbol + "|" + role + if seen[key] { + return + } + seen[key] = true + var owners []string + var tracked bool + if role == "producer" { + owners, tracked = b.EventProducers[symbol] + } else { + owners, tracked = b.EventConsumers[symbol] + } + if !tracked { + return + } + if allowedHas(owners, pkg) { + return + } + out = append(out, Finding{ + Kind: KindEvent, + Package: pkg, + File: rel, + Symbol: symbol, + Detail: detail, + }) + } + + if strings.Contains(src, "OnOutbox") { + isAssign := strings.Contains(src, "OnOutbox =") + isCallOrField := strings.Contains(src, "OnOutbox(") || strings.Contains(src, "OnOutbox func(") + if isCallOrField && !isAssign { + add("outbox.notify", "producer", "OnOutbox hook") + } + if isAssign { + add("outbox.notify", "consumer", "OnOutbox assignment") + } + } + for _, name := range eventNameRe.FindAllString(src, -1) { + if _, ok := b.EventProducers[name]; ok { + if allowedHas(b.EventConsumers[name], pkg) || allowedHas(b.EventConsumers["outbox.notify"], pkg) { + continue + } + add(name, "producer", "event name literal") + } + } + _ = file + return out +} + +func dedupe(in []Finding) []Finding { + seen := map[string]bool{} + var out []Finding + for _, f := range in { + k := f.Key() + if seen[k] { + continue + } + seen[k] = true + out = append(out, f) + } + return out +} diff --git a/internal/archtest/testdata/mini/deploy/postgres/goose/00001_fixture.sql b/internal/archtest/testdata/mini/deploy/postgres/goose/00001_fixture.sql new file mode 100644 index 00000000..35da2979 --- /dev/null +++ b/internal/archtest/testdata/mini/deploy/postgres/goose/00001_fixture.sql @@ -0,0 +1,4 @@ +-- Goose may mention owned tables; this is not a production writer. +UPDATE document_records SET current_state = 'approved' WHERE doc_id = 'fixture'; +INSERT INTO document_records (doc_id) VALUES ('fixture'); +DELETE FROM document_records WHERE doc_id = 'fixture'; diff --git a/internal/archtest/testdata/mini/internal/docgov/owner_write.go b/internal/archtest/testdata/mini/internal/docgov/owner_write.go new file mode 100644 index 00000000..e59a23ad --- /dev/null +++ b/internal/archtest/testdata/mini/internal/docgov/owner_write.go @@ -0,0 +1,5 @@ +package docgov + +const ownerSQL = ` +UPDATE document_records SET current_state = 'approved' WHERE doc_id = $1 +` diff --git a/internal/archtest/testdata/mini/internal/docgov/owner_write_test.go b/internal/archtest/testdata/mini/internal/docgov/owner_write_test.go new file mode 100644 index 00000000..286b7ca1 --- /dev/null +++ b/internal/archtest/testdata/mini/internal/docgov/owner_write_test.go @@ -0,0 +1,5 @@ +package docgov + +const testSQL = ` +UPDATE document_records SET current_state = 'test' WHERE doc_id = $1 +` diff --git a/internal/archtest/testdata/mini/internal/server/event_producer.go b/internal/archtest/testdata/mini/internal/server/event_producer.go new file mode 100644 index 00000000..44b55804 --- /dev/null +++ b/internal/archtest/testdata/mini/internal/server/event_producer.go @@ -0,0 +1,7 @@ +package server + +func emit() { + _ = OnOutbox("document.agent_version_committed") +} + +func OnOutbox(eventType string) error { return nil } diff --git a/internal/archtest/testdata/mini/internal/server/path_concat.go b/internal/archtest/testdata/mini/internal/server/path_concat.go new file mode 100644 index 00000000..59fb43e4 --- /dev/null +++ b/internal/archtest/testdata/mini/internal/server/path_concat.go @@ -0,0 +1,5 @@ +package server + +func concatOutput(dir string) string { + return dir + "/output.md" +} diff --git a/internal/archtest/testdata/mini/internal/server/path_const.go b/internal/archtest/testdata/mini/internal/server/path_const.go new file mode 100644 index 00000000..37482d62 --- /dev/null +++ b/internal/archtest/testdata/mini/internal/server/path_const.go @@ -0,0 +1,9 @@ +package server + +import "path/filepath" + +const deliverableName = "output.md" + +func constOutput(dir string) string { + return filepath.Join(dir, deliverableName) +} diff --git a/internal/archtest/testdata/mini/internal/server/path_join.go b/internal/archtest/testdata/mini/internal/server/path_join.go new file mode 100644 index 00000000..92f5b87d --- /dev/null +++ b/internal/archtest/testdata/mini/internal/server/path_join.go @@ -0,0 +1,7 @@ +package server + +import "path/filepath" + +func joinOutput(dir string) string { + return filepath.Join(dir, "output.md") +} diff --git a/internal/archtest/testdata/mini/internal/server/rogue_write.go b/internal/archtest/testdata/mini/internal/server/rogue_write.go new file mode 100644 index 00000000..78b6cb20 --- /dev/null +++ b/internal/archtest/testdata/mini/internal/server/rogue_write.go @@ -0,0 +1,5 @@ +package server + +const rogueSQL = ` +UPDATE document_records SET current_state = 'hacked' WHERE doc_id = $1 +` diff --git a/scripts/arch-check.sh b/scripts/arch-check.sh new file mode 100755 index 00000000..fc82c2a2 --- /dev/null +++ b/scripts/arch-check.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# Architecture boundary check (U1). +# Production import / SQL write / path construction / event producer-consumer. +# Goose, *_test.go and fixtures are not production writers. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +MODE="warn" + +while [[ $# -gt 0 ]]; do + case "$1" in + --mode) MODE="${2:-warn}"; shift 2 ;; + --mode=*) MODE="${1#--mode=}"; shift ;; + *) echo "usage: $0 [--mode warn|block]" >&2; exit 2 ;; + esac +done + +if [[ "${MODE}" != "warn" && "${MODE}" != "block" ]]; then + echo "mode must be warn or block" >&2 + exit 2 +fi + +cd "${ROOT}" +echo "arch-check mode=${MODE}" +echo "boundaries=${ROOT}/.doc/system-design/module-boundaries.yaml" + +if ! go test ./internal/archtest -count=1; then + echo "arch-check: new, expired, or unowned findings failed go test ./internal/archtest" >&2 + exit 1 +fi + +if [[ "${MODE}" == "block" ]]; then + if ! ARCHTEST_STRICT=1 go test ./internal/archtest -count=1 -run TestArchitectureBoundaries; then + echo "arch-check block: baseline exceptions remain; only cleared rules may use --mode block" >&2 + exit 1 + fi +fi + +echo "arch-check ${MODE}: production gate passed" From 158ce2586d8b977559a8a15880162cf65ed2b635 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 13:49:54 +0000 Subject: [PATCH 03/11] docs(audit): record U1 architecture baseline Capture the replayable import/sql/path/event counts and the owned exceptions that later milestones must shrink. Co-authored-by: Cursor --- .doc/audit/2026-09-14-arch-check-baseline.md | 64 +++++++++++++++++++ .doc/audit/INDEX.md | 6 ++ .../2026-09-14-document-reliability-pilot.md | 4 +- 3 files changed, 72 insertions(+), 2 deletions(-) create mode 100644 .doc/audit/2026-09-14-arch-check-baseline.md diff --git a/.doc/audit/2026-09-14-arch-check-baseline.md b/.doc/audit/2026-09-14-arch-check-baseline.md new file mode 100644 index 00000000..97c33648 --- /dev/null +++ b/.doc/audit/2026-09-14-arch-check-baseline.md @@ -0,0 +1,64 @@ +--- +title: "arch-check U1 可重放基线" +status: done +created: 2026-09-14 +updated: 2026-09-14 +severity: medium +--- + +# arch-check U1 可重放基线 + +## 现象 + +Wave 0 的全仓 grep 会把 Goose、`*_test.go` 和路径字面量一视同仁，无法作为生产写者门禁。U1 用 `go list` + AST/正则拆成四类规则，并给每条既有债写明 owner、原因、截止里程碑和到期日。 + +## 根因分析 + +- `document_records` 的生产写者不止 `docgov`：`docrevision.projectRootPointersTX` 与 `docduty` 职责槽位仍直接 `UPDATE`。 +- `output.md` / `data/dev` / `workspaces/` 仍被 agent、bizdev、coordinator、docgov、docrevision、server、storage 用 `filepath.Join`、字符串字面量或间接常量构造。 +- Goose 与测试夹具大量出现同样表名，不能算生产写者债。 + +## 严重度 + +`medium` + +## 关联整改 + +- ADR: [../adr/2026-09-14-modular-monolith-decoupling.md](../adr/2026-09-14-modular-monolith-decoupling.md) +- 设计: [../system-design/2026-09-14-module-decoupling-architecture.md](../system-design/2026-09-14-module-decoupling-architecture.md) +- 边界: [../system-design/module-boundaries.yaml](../system-design/module-boundaries.yaml) +- 任务: [../ops/short-term/2026-09-14-document-reliability-pilot.md](../ops/short-term/2026-09-14-document-reliability-pilot.md) + +## 证据 + +命令（2026-09-14，`refactor/document-reliability-pilot` worktree）： + +``` +./scripts/arch-check.sh --mode warn +go test ./internal/archtest -count=1 +``` + +可重放计数（只含生产 Go；Goose/`*_test.go` 不计）： + +| 规则 | 计数 | 处理 | +|------|------|------| +| import | 0 | 相对 2026-09-14 production `go list` allowlist，无新边 | +| sql_write | 5 | 例外，U3/U8 到期 | +| path | 17 | 例外，U4/U5/U8 到期 | +| event | 0 | `OnOutbox` 生产者属主为 docrevision，server/notification 为消费者 | + +验证： + +- `deploy/postgres/goose` 与 `*_test.go` 出现 `document_records` 不报生产写者违规。 +- 迷你仓 `internal/server/rogue_write.go` 新增 `UPDATE document_records` 时 `go test ./internal/archtest` 失败并指出文件。 +- `filepath.Join`、字符串拼接、间接常量构造 `output.md` 均可检出。 +- 无到期日例外、过期例外立即 block。 +- `baseline_counts` 锁定现有计数，只能下降。 + +`--mode block` 在例外清零前会失败，不得挂进集成冒烟。 + +## 开放问题 + +- U3 收口 `document_records` / `freeform_documents` 跨属主写。 +- U4/U5 收口 handler 与投影路径构造。 +- U8 收口 `task_runs` 终止写与 finalize `output.md`。 diff --git a/.doc/audit/INDEX.md b/.doc/audit/INDEX.md index 5e5bc69c..058e4ec8 100644 --- a/.doc/audit/INDEX.md +++ b/.doc/audit/INDEX.md @@ -13,6 +13,12 @@ ## 核心文档（按阅读顺序推荐） +### arch-check U1 可重放基线（2026-09-14） + +**文件**: [2026-09-14-arch-check-baseline.md](./2026-09-14-arch-check-baseline.md) + +**结论**: 四类规则（production import / SQL write / path / event）可重放。import=0，sql_write=5，path=17，event=0。Goose 与测试不计生产写者。新违规与过期例外立即 block。 + ### 模块解耦方案复核（2026-09-14） **文件**: [2026-09-14-module-decoupling-plan-review.md](./2026-09-14-module-decoupling-plan-review.md) diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md index 6983f3cd..978be537 100644 --- a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -33,8 +33,8 @@ mutex: docgov ## 步骤 1. [x] 从最新 `dev`（`a8f9451`）创建实现分支，登记 CURRENT 并确认 `docgov` mutex 已释放。 -2. [ ] 为当前跨表提交、刷新读取、ETag/idempotency 和权限负例补 characterization。 -3. [ ] 实现 U1–U2，验证任意属主写入/commit 故障整体回滚。 +2. [x] U1：四类规则 arch-check + archtest + 可重放基线（新违规 block）。 +3. [ ] 实现 U2，验证任意属主写入/commit 故障整体回滚。 4. [ ] 实现 U3，删除提交后 outbox 正确性回调。 5. [ ] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 From f8b21ecd7c9d7b9ecac07e1e8aa7e5288d46cf25 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 13:54:27 +0000 Subject: [PATCH 04/11] feat(app): add UnitOfWork and owner-bound repositories Give each document command one postgres transaction and only let table owners unwrap the scope, so cross-owner writes can roll back together. Co-authored-by: Cursor --- .doc/system-design/module-boundaries.yaml | 12 + internal/app/uow/pg_uow.go | 172 ++++++++++++++ internal/app/uow/pg_uow_test.go | 267 ++++++++++++++++++++++ internal/app/uow/uow.go | 41 ++++ internal/app/uow/uow_test.go | 48 ++++ internal/archtest/boundaries.go | 4 + internal/archtest/scan.go | 17 ++ internal/docgov/repository.go | 106 +++++++++ internal/docrevision/repository.go | 52 +++++ internal/freedoc/repository.go | 18 ++ internal/taskrun/repository.go | 18 ++ 11 files changed, 755 insertions(+) create mode 100644 internal/app/uow/pg_uow.go create mode 100644 internal/app/uow/pg_uow_test.go create mode 100644 internal/app/uow/uow.go create mode 100644 internal/app/uow/uow_test.go create mode 100644 internal/docgov/repository.go create mode 100644 internal/docrevision/repository.go create mode 100644 internal/freedoc/repository.go create mode 100644 internal/taskrun/repository.go diff --git a/.doc/system-design/module-boundaries.yaml b/.doc/system-design/module-boundaries.yaml index 56d6db2b..f523063e 100644 --- a/.doc/system-design/module-boundaries.yaml +++ b/.doc/system-design/module-boundaries.yaml @@ -54,13 +54,18 @@ imports: - storage db: [] docduty: [] + app: [] + app/uow: [] docgov: + - app/uow - docduty - docrevision - fsutil docrevision: + - app/uow - fsutil freedoc: + - app/uow - ctxmgr - docrevision - replyexport @@ -116,6 +121,7 @@ imports: storage: [] stream: [] taskrun: + - app/uow - bizdev - ctxmgr - projectworkflow @@ -179,6 +185,12 @@ event_consumers: - server - notification +unwrap_owners: + - docgov + - docrevision + - taskrun + - freedoc + exceptions: - kind: sql_write package: docduty diff --git a/internal/app/uow/pg_uow.go b/internal/app/uow/pg_uow.go new file mode 100644 index 00000000..62d246ee --- /dev/null +++ b/internal/app/uow/pg_uow.go @@ -0,0 +1,172 @@ +package uow + +import ( + "context" + "errors" + "fmt" + "math/rand" + "strings" + "time" + + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgconn" + "github.com/jackc/pgx/v5/pgxpool" +) + +type pgScope struct { + ctx context.Context + tx pgx.Tx + done bool +} + +func (s *pgScope) Context() context.Context { return s.ctx } + +// AsPG returns the bound pgx.Tx for owner repositories only. +// Application / docflow code must not call this; archtest enforces unwrap owners. +func AsPG(scope TxScope) (pgx.Tx, error) { + s, ok := scope.(*pgScope) + if !ok || s == nil || s.tx == nil { + return nil, fmt.Errorf("uow: scope is not a postgres transaction") + } + if s.done { + return nil, ErrClosedScope + } + return s.tx, nil +} + +type PG struct { + Pool *pgxpool.Pool + Policy RetryPolicy + Now func() time.Time + Sleep func(time.Duration) + Rand *rand.Rand + BeforeBegin func(context.Context) error + AfterBegin func(TxScope) error + BeforeCommit func(TxScope) error +} + +func NewPG(pool *pgxpool.Pool) *PG { + return &PG{Pool: pool, Policy: DefaultRetryPolicy()} +} + +func (p *PG) Within(ctx context.Context, fn func(TxScope) error) error { + if p == nil || p.Pool == nil { + return fmt.Errorf("uow: postgres pool required") + } + if fn == nil { + return fmt.Errorf("uow: command required") + } + policy := p.Policy + if policy.MaxAttempts < 1 { + policy.MaxAttempts = DefaultMaxAttempts + } + if policy.LockTimeout <= 0 { + policy.LockTimeout = DefaultLockTimeout + } + var last error + for attempt := 1; attempt <= policy.MaxAttempts; attempt++ { + last = p.one(ctx, policy.LockTimeout, fn) + if last == nil { + return nil + } + if !IsSerialization(last) || IsNonRetryable(last) { + return last + } + if attempt == policy.MaxAttempts { + return last + } + p.backoff(attempt) + } + return last +} + +func (p *PG) one(ctx context.Context, lockWait time.Duration, fn func(TxScope) error) error { + if p.BeforeBegin != nil { + if err := p.BeforeBegin(ctx); err != nil { + return err + } + } + if dl, ok := ctx.Deadline(); ok { + remain := time.Until(dl) + if remain > 0 && lockWait > remain { + lockWait = remain + } + } + tx, err := p.Pool.Begin(ctx) + if err != nil { + return err + } + scope := &pgScope{ctx: ctx, tx: tx} + defer func() { + scope.done = true + _ = tx.Rollback(ctx) + }() + if _, err := tx.Exec(ctx, fmt.Sprintf("SET LOCAL lock_timeout = '%dms'", lockWait.Milliseconds())); err != nil { + return err + } + if p.AfterBegin != nil { + if err := p.AfterBegin(scope); err != nil { + return err + } + } + if err := fn(scope); err != nil { + return classifyLock(err) + } + if p.BeforeCommit != nil { + if err := p.BeforeCommit(scope); err != nil { + return err + } + } + if err := tx.Commit(ctx); err != nil { + return classifyLock(err) + } + scope.done = true + return nil +} + +func (p *PG) backoff(attempt int) { + sleep := p.Sleep + if sleep == nil { + sleep = time.Sleep + } + n := 10 + if p.Rand != nil { + n = 5 + p.Rand.Intn(20) + } else { + n = 5 + rand.Intn(20) + } + sleep(time.Duration(n*attempt) * time.Millisecond) +} + +func IsSerialization(err error) bool { + var pgErr *pgconn.PgError + if errors.As(err, &pgErr) { + return pgErr.Code == "40001" || pgErr.Code == "40P01" + } + return false +} + +func IsNonRetryable(err error) bool { + return errors.Is(err, ErrVersionConflict) || + errors.Is(err, ErrPermissionDenied) || + errors.Is(err, ErrDigestConflict) +} + +func classifyLock(err error) error { + var pgErr *pgconn.PgError + if errors.As(err, &pgErr) { + switch pgErr.Code { + case "55P03": // lock_not_available + return ErrBusy + case "57014": // query_canceled / statement timeout + if strings.Contains(strings.ToLower(pgErr.Message), "lock") { + return ErrBusy + } + } + } + msg := strings.ToLower(err.Error()) + if strings.Contains(msg, "lock timeout") || strings.Contains(msg, "canceling statement due to lock timeout") { + return ErrBusy + } + return err +} diff --git a/internal/app/uow/pg_uow_test.go b/internal/app/uow/pg_uow_test.go new file mode 100644 index 00000000..53ef3fea --- /dev/null +++ b/internal/app/uow/pg_uow_test.go @@ -0,0 +1,267 @@ +package uow_test + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "os" + "testing" + "time" + + "chat-agent-system/internal/app/uow" + "chat-agent-system/internal/db" + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" + + "github.com/jackc/pgx/v5/pgconn" + "github.com/jackc/pgx/v5/pgxpool" +) + +func testPool(t *testing.T) (*pgxpool.Pool, context.Context) { + t.Helper() + if testing.Short() { + t.Skip("postgresql fault injection is not part of -short") + } + dsn := db.SkipIfNoTestDatabaseURL(t) + ctx := context.Background() + pool, err := pgxpool.New(ctx, dsn) + if err != nil { + t.Fatal(err) + } + t.Cleanup(pool.Close) + if err := db.RunMigrationsOnceForTest(ctx, pool, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + return pool, ctx +} + +func init() { + // ResolveMigrationsDir is cwd-relative; tests in this package start under internal/app/uow. + if _, err := os.Stat("deploy/postgres/goose"); err == nil { + return + } + for _, cand := range []string{"../../..", "../../../.."} { + if fi, err := os.Stat(cand + "/deploy/postgres/goose"); err == nil && fi.IsDir() { + _ = os.Chdir(cand) + return + } + } +} + +func seedProject(t *testing.T, ctx context.Context, pool *pgxpool.Pool, suffix string) int64 { + t.Helper() + var projectID int64 + err := pool.QueryRow(ctx, ` + INSERT INTO projects (code, name, status) + VALUES ($1, $2, 'active') + RETURNING id + `, "uow-"+suffix, "uow "+suffix).Scan(&projectID) + if err != nil { + t.Fatalf("insert project: %v", err) + } + t.Cleanup(func() { + _, _ = pool.Exec(context.Background(), `DELETE FROM document_change_events WHERE doc_id LIKE $1`, "uow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_versions WHERE doc_id LIKE $1`, "uow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_snapshots WHERE doc_id LIKE $1`, "uow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_records WHERE project_id=$1`, projectID) + _, _ = pool.Exec(context.Background(), `DELETE FROM projects WHERE id=$1`, projectID) + }) + return projectID +} + +func shaBody(body string) string { + sum := sha256.Sum256([]byte(body)) + return hex.EncodeToString(sum[:]) +} + +func writeOwners(ctx context.Context, scope uow.TxScope, projectID int64, docID, body string) error { + if err := (docgov.Repository{}).InsertRoot(ctx, scope, docgov.DocumentRecord{ + DocID: docID, ProjectID: projectID, TaskKey: "intake", Title: docID, + ContentSHA256: shaBody(body), WorkDir: "ws/" + docID, + }); err != nil { + return err + } + sha := shaBody(body) + if err := (docrevision.Repository{}).InsertBlob(ctx, scope, sha, body); err != nil { + return err + } + sid, err := (docrevision.Repository{}).InsertSnapshot(ctx, scope, docrevision.Snapshot{ + DocID: docID, ContentSHA256: sha, SourceKind: docrevision.SourceTemplate, + ActorRef: "template", CreatedAt: time.Now().UTC(), + }) + if err != nil { + return err + } + vid, err := (docrevision.Repository{}).InsertVersion(ctx, scope, docrevision.Version{ + DocID: docID, VersionNo: 1, Status: docrevision.VersionOpen, OriginKind: docrevision.OriginTemplate, + BaselineSnapshotID: sid, HeadSnapshotID: sid, OpenedAt: time.Now().UTC(), + }) + if err != nil { + return err + } + _, err = (docrevision.Repository{}).InsertEvent(ctx, scope, docrevision.ChangeEvent{ + DocID: docID, VersionID: vid, EventNo: 1, ToSnapshotID: sid, + ChangeKind: docrevision.ChangeImport, ActorKind: docrevision.ActorTemplate, + ActorRef: "template", IdempotencyKey: "uow:" + docID, CreatedAt: time.Now().UTC(), + }) + return err +} + +func countDoc(t *testing.T, ctx context.Context, pool *pgxpool.Pool, docID string) (records, snaps, events int) { + t.Helper() + if err := pool.QueryRow(ctx, `SELECT COUNT(*) FROM document_records WHERE doc_id=$1`, docID).Scan(&records); err != nil { + t.Fatal(err) + } + if err := pool.QueryRow(ctx, `SELECT COUNT(*) FROM document_snapshots WHERE doc_id=$1`, docID).Scan(&snaps); err != nil { + t.Fatal(err) + } + if err := pool.QueryRow(ctx, `SELECT COUNT(*) FROM document_change_events WHERE doc_id=$1`, docID).Scan(&events); err != nil { + t.Fatal(err) + } + return +} + +func TestSecondOwnerFailureRollsBackFirst(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + docID := "uow_" + suffix + "_rollback" + work := uow.NewPG(pool) + err := work.Within(ctx, func(scope uow.TxScope) error { + if err := (docgov.Repository{}).InsertRoot(ctx, scope, docgov.DocumentRecord{ + DocID: docID, ProjectID: projectID, Title: docID, TaskKey: "intake", + }); err != nil { + return err + } + return errors.New("ledger write failed") + }) + if err == nil { + t.Fatal("expected failure") + } + r, s, e := countDoc(t, ctx, pool, docID) + if r != 0 || s != 0 || e != 0 { + t.Fatalf("partial commit records=%d snaps=%d events=%d", r, s, e) + } +} + +func TestStagingFailureLeavesNoLedgerOrRoot(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + docID := "uow_" + suffix + "_staging" + work := uow.NewPG(pool) + err := work.Within(ctx, func(scope uow.TxScope) error { + if err := writeOwners(ctx, scope, projectID, docID, "# staging\n"); err != nil { + return err + } + return fmt.Errorf("outbox/staging write failed") + }) + if err == nil { + t.Fatal("expected staging failure") + } + r, s, e := countDoc(t, ctx, pool, docID) + if r != 0 || s != 0 || e != 0 { + t.Fatalf("staging failure leaked rows records=%d snaps=%d events=%d", r, s, e) + } +} + +func TestCommitFailureHasNoSideEffects(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + docID := "uow_" + suffix + "_commit" + work := uow.NewPG(pool) + work.BeforeCommit = func(uow.TxScope) error { return errors.New("commit failed") } + sideEffect := false + err := work.Within(ctx, func(scope uow.TxScope) error { + return writeOwners(ctx, scope, projectID, docID, "# commit\n") + }) + if err == nil { + sideEffect = true + } + if sideEffect { + t.Fatal("caller must see commit error and skip post-commit work") + } + r, s, e := countDoc(t, ctx, pool, docID) + if r != 0 || s != 0 || e != 0 { + t.Fatalf("commit failure leaked rows records=%d snaps=%d events=%d", r, s, e) + } +} + +func TestSerializationRetriesWholeTransactionOnce(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + docID := "uow_" + suffix + "_ser" + work := uow.NewPG(pool) + work.Sleep = func(time.Duration) {} + attempts := 0 + err := work.Within(ctx, func(scope uow.TxScope) error { + attempts++ + body := fmt.Sprintf("# ser %d\n", attempts) + if err := writeOwners(ctx, scope, projectID, docID, body); err != nil { + return err + } + if attempts == 1 { + return &pgconn.PgError{Code: "40001", Message: "could not serialize access"} + } + return nil + }) + if err != nil { + t.Fatal(err) + } + if attempts != 2 { + t.Fatalf("attempts=%d want 2", attempts) + } + r, s, e := countDoc(t, ctx, pool, docID) + if r != 1 || s != 1 || e != 1 { + t.Fatalf("want one business result records=%d snaps=%d events=%d", r, s, e) + } +} + +func TestNonRetryableErrorsAreNotRetried(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + work := uow.NewPG(pool) + work.Sleep = func(time.Duration) {} + for _, want := range []error{uow.ErrVersionConflict, uow.ErrPermissionDenied, uow.ErrDigestConflict} { + attempts := 0 + err := work.Within(ctx, func(scope uow.TxScope) error { + attempts++ + _ = (docgov.Repository{}).InsertRoot(ctx, scope, docgov.DocumentRecord{ + DocID: "uow_" + suffix + "_noretry", ProjectID: projectID, Title: "x", TaskKey: "intake", + }) + return want + }) + if !errors.Is(err, want) { + t.Fatalf("err=%v want %v", err, want) + } + if attempts != 1 { + t.Fatalf("%v retried attempts=%d", want, attempts) + } + } + r, _, _ := countDoc(t, ctx, pool, "uow_"+suffix+"_noretry") + if r != 0 { + t.Fatalf("non-retryable write leaked record=%d", r) + } +} + +func TestHappyPathPersistsBothOwners(t *testing.T) { + pool, ctx := testPool(t) + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + projectID := seedProject(t, ctx, pool, suffix) + docID := "uow_" + suffix + "_ok" + work := uow.NewPG(pool) + if err := work.Within(ctx, func(scope uow.TxScope) error { + return writeOwners(ctx, scope, projectID, docID, "# ok\n") + }); err != nil { + t.Fatal(err) + } + r, s, e := countDoc(t, ctx, pool, docID) + if r != 1 || s != 1 || e != 1 { + t.Fatalf("happy path records=%d snaps=%d events=%d", r, s, e) + } +} diff --git a/internal/app/uow/uow.go b/internal/app/uow/uow.go new file mode 100644 index 00000000..e77267d6 --- /dev/null +++ b/internal/app/uow/uow.go @@ -0,0 +1,41 @@ +package uow + +import ( + "context" + "errors" + "time" +) + +var ( + ErrVersionConflict = errors.New("uow: version conflict") + ErrPermissionDenied = errors.New("uow: permission denied") + ErrDigestConflict = errors.New("uow: digest conflict") + ErrBusy = errors.New("uow: lock busy") + ErrClosedScope = errors.New("uow: transaction scope is closed") + ErrNestedBegin = errors.New("uow: repository must not begin a nested transaction") + ErrPrematureCommit = errors.New("uow: repository must not commit the unit of work") +) + +// TxScope is an opaque transaction handle. Application code cannot Exec SQL through it. +type TxScope interface { + Context() context.Context +} + +// UnitOfWork runs one command in a single database transaction. +type UnitOfWork interface { + Within(ctx context.Context, fn func(TxScope) error) error +} + +const ( + DefaultLockTimeout = 2 * time.Second + DefaultMaxAttempts = 3 +) + +type RetryPolicy struct { + MaxAttempts int + LockTimeout time.Duration +} + +func DefaultRetryPolicy() RetryPolicy { + return RetryPolicy{MaxAttempts: DefaultMaxAttempts, LockTimeout: DefaultLockTimeout} +} diff --git a/internal/app/uow/uow_test.go b/internal/app/uow/uow_test.go new file mode 100644 index 00000000..a36f26e4 --- /dev/null +++ b/internal/app/uow/uow_test.go @@ -0,0 +1,48 @@ +package uow + +import ( + "context" + "errors" + "reflect" + "testing" + + "github.com/jackc/pgx/v5/pgconn" +) + +func TestTxScopeDoesNotExposeExec(t *testing.T) { + typ := reflect.TypeOf((*TxScope)(nil)).Elem() + for _, name := range []string{"Exec", "Begin", "Commit", "Rollback"} { + if _, ok := typ.MethodByName(name); ok { + t.Fatalf("TxScope must not expose %s", name) + } + } + pg := reflect.TypeOf(&pgScope{}) + if _, ok := pg.MethodByName("Exec"); ok { + t.Fatal("pgScope must not expose Exec") + } +} + +type fakeScope struct{} + +func (fakeScope) Context() context.Context { return context.Background() } + +func TestAsPGRejectsOpaqueScope(t *testing.T) { + if _, err := AsPG(fakeScope{}); err == nil { + t.Fatal("expected unwrap failure") + } +} + +func TestSerializationAndNonRetryable(t *testing.T) { + if !IsSerialization(&pgconn.PgError{Code: "40001"}) || !IsSerialization(&pgconn.PgError{Code: "40P01"}) { + t.Fatal("serialization/deadlock must retry") + } + if IsSerialization(errors.New("plain")) { + t.Fatal("plain error must not retry") + } + if !IsNonRetryable(ErrVersionConflict) || !IsNonRetryable(ErrPermissionDenied) || !IsNonRetryable(ErrDigestConflict) { + t.Fatal("typed conflicts must not enter retry") + } + if IsNonRetryable(&pgconn.PgError{Code: "40001"}) { + t.Fatal("serialization is retryable") + } +} diff --git a/internal/archtest/boundaries.go b/internal/archtest/boundaries.go index cf8d91b5..8f575af3 100644 --- a/internal/archtest/boundaries.go +++ b/internal/archtest/boundaries.go @@ -24,6 +24,7 @@ type Boundaries struct { PathOwners map[string][]string `yaml:"path_owners"` EventProducers map[string][]string `yaml:"event_producers"` EventConsumers map[string][]string `yaml:"event_consumers"` + UnwrapOwners []string `yaml:"unwrap_owners"` Exceptions []Exception `yaml:"exceptions"` BaselineCounts map[string]int `yaml:"baseline_counts"` } @@ -93,6 +94,9 @@ func LoadBoundaries(path string) (*Boundaries, error) { if out.EventConsumers == nil { out.EventConsumers = map[string][]string{} } + if out.UnwrapOwners == nil { + out.UnwrapOwners = []string{} + } if out.BaselineCounts == nil { out.BaselineCounts = map[string]int{} } diff --git a/internal/archtest/scan.go b/internal/archtest/scan.go index 6b89c219..8cdd3e79 100644 --- a/internal/archtest/scan.go +++ b/internal/archtest/scan.go @@ -139,6 +139,7 @@ func scanProductionGo(fset *token.FileSet, repoRoot, path string, b *Boundaries) out = append(out, scanSQLWrites(string(src), pkg, rel, b)...) out = append(out, scanPaths(file, pkg, rel, b)...) out = append(out, scanEvents(file, string(src), pkg, rel, b)...) + out = append(out, scanUnwrap(string(src), pkg, rel, b)...) return out, nil } @@ -409,6 +410,22 @@ func scanEvents(file *ast.File, src, pkg, rel string, b *Boundaries) []Finding { return out } +func scanUnwrap(src, pkg, rel string, b *Boundaries) []Finding { + if !strings.Contains(src, "uow.AsPG") { + return nil + } + if allowedHas(b.UnwrapOwners, pkg) { + return nil + } + return []Finding{{ + Kind: KindImport, + Package: pkg, + File: rel, + Symbol: "uow.AsPG", + Detail: "application layer must not unwrap TxScope", + }} +} + func dedupe(in []Finding) []Finding { seen := map[string]bool{} var out []Finding diff --git a/internal/docgov/repository.go b/internal/docgov/repository.go new file mode 100644 index 00000000..8c1c31da --- /dev/null +++ b/internal/docgov/repository.go @@ -0,0 +1,106 @@ +package docgov + +import ( + "context" + "fmt" + "time" + + "chat-agent-system/internal/app/uow" +) + +// Repository is the document_records writer bound to a UnitOfWork scope. +type Repository struct{} + +func (Repository) InsertRoot(ctx context.Context, scope uow.TxScope, rec DocumentRecord) error { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + if rec.CreatedAt.IsZero() { + rec.CreatedAt = time.Now().UTC() + } + if rec.UpdatedAt.IsZero() { + rec.UpdatedAt = rec.CreatedAt + } + if rec.Stage == "" { + rec.Stage = "V0.5_feasibility" + } + if rec.CurrentState == "" { + rec.CurrentState = StateDrafting + } + if rec.Title == "" { + rec.Title = rec.DocID + } + if rec.TaskKey == "" { + rec.TaskKey = "intake" + } + _, err = tx.Exec(ctx, ` + INSERT INTO document_records ( + doc_id, project_id, task_key, title, stage, version, + current_state, is_locked, is_template_only, content_sha256, work_dir, + author_name, reviewer_name, approver_name, created_at, updated_at + ) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16)`, + rec.DocID, rec.ProjectID, rec.TaskKey, rec.Title, rec.Stage, rec.Version, + string(rec.CurrentState), rec.IsLocked, rec.IsTemplateOnly, rec.ContentSHA256, rec.WorkDir, + rec.AuthorName, rec.ReviewerName, rec.ApproverName, rec.CreatedAt, rec.UpdatedAt) + if err != nil { + return fmt.Errorf("docgov: insert root: %w", err) + } + return nil +} + +func (Repository) GetForUpdate(ctx context.Context, scope uow.TxScope, docID string) (*DocumentRecord, error) { + tx, err := uow.AsPG(scope) + if err != nil { + return nil, err + } + var rec DocumentRecord + var state string + err = tx.QueryRow(ctx, ` + SELECT doc_id, project_id, task_key, title, stage, version, current_state, + is_locked, is_template_only, content_sha256, work_dir + FROM document_records WHERE doc_id=$1 FOR UPDATE`, docID).Scan( + &rec.DocID, &rec.ProjectID, &rec.TaskKey, &rec.Title, &rec.Stage, &rec.Version, &state, + &rec.IsLocked, &rec.IsTemplateOnly, &rec.ContentSHA256, &rec.WorkDir) + if err != nil { + return nil, fmt.Errorf("docgov: lock document: %w", err) + } + rec.CurrentState = DocumentState(state) + return &rec, nil +} + +func (Repository) UpdateState(ctx context.Context, scope uow.TxScope, rec DocumentRecord) error { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + _, err = tx.Exec(ctx, ` + UPDATE document_records + SET current_state=$2, is_locked=$3, content_sha256=COALESCE(NULLIF($4,''), content_sha256), + updated_at=$5 + WHERE doc_id=$1`, + rec.DocID, string(rec.CurrentState), rec.IsLocked, rec.ContentSHA256, time.Now().UTC()) + if err != nil { + return fmt.Errorf("docgov: update state: %w", err) + } + return nil +} + +func (Repository) ProjectHead(ctx context.Context, scope uow.TxScope, docID string, versionID, snapshotID int64, revisionNo int, sha string) error { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + _, err = tx.Exec(ctx, ` + UPDATE document_records + SET ledger_version_id=$2, head_snapshot_id=$3, head_revision_no=$4, + row_revision=COALESCE(row_revision,0)+1, + content_sha256=COALESCE(NULLIF($5,''), content_sha256), + updated_at=now() + WHERE doc_id=$1`, + docID, versionID, snapshotID, revisionNo, sha) + if err != nil { + return fmt.Errorf("docgov: project head: %w", err) + } + return nil +} diff --git a/internal/docrevision/repository.go b/internal/docrevision/repository.go new file mode 100644 index 00000000..fc8bf45d --- /dev/null +++ b/internal/docrevision/repository.go @@ -0,0 +1,52 @@ +package docrevision + +import ( + "context" + "fmt" + "time" + + "chat-agent-system/internal/app/uow" +) + +// Repository is the ledger writer bound to a UnitOfWork scope. +type Repository struct{} + +func (Repository) InsertBlob(ctx context.Context, scope uow.TxScope, sha, markdown string) error { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + _, err = tx.Exec(ctx, ` + INSERT INTO document_content_blobs (content_sha256, content_markdown, byte_size, created_at) + VALUES ($1,$2,$3,$4) + ON CONFLICT (content_sha256) DO NOTHING`, + sha, markdown, int64(len(markdown)), time.Now().UTC()) + if err != nil { + return fmt.Errorf("docrevision: insert blob: %w", err) + } + return nil +} + +func (Repository) InsertSnapshot(ctx context.Context, scope uow.TxScope, sn Snapshot) (int64, error) { + tx, err := uow.AsPG(scope) + if err != nil { + return 0, err + } + return (&Service{}).insertSnapshotTX(ctx, tx, &sn) +} + +func (Repository) InsertVersion(ctx context.Context, scope uow.TxScope, v Version) (int64, error) { + tx, err := uow.AsPG(scope) + if err != nil { + return 0, err + } + return (&Service{}).insertVersionTX(ctx, tx, &v) +} + +func (Repository) InsertEvent(ctx context.Context, scope uow.TxScope, ev ChangeEvent) (int64, error) { + tx, err := uow.AsPG(scope) + if err != nil { + return 0, err + } + return (&Service{}).insertEventTX(ctx, tx, &ev) +} diff --git a/internal/freedoc/repository.go b/internal/freedoc/repository.go new file mode 100644 index 00000000..96a3053f --- /dev/null +++ b/internal/freedoc/repository.go @@ -0,0 +1,18 @@ +package freedoc + +import ( + "context" + "fmt" + + "chat-agent-system/internal/app/uow" +) + +// Repository is a M3 stub. M1 freeform projection stays on the existing service path. +type Repository struct{} + +func (Repository) ForbiddenInM1(ctx context.Context, scope uow.TxScope) error { + if _, err := uow.AsPG(scope); err != nil { + return err + } + return fmt.Errorf("freedoc: repository writes are not enabled until M3") +} diff --git a/internal/taskrun/repository.go b/internal/taskrun/repository.go new file mode 100644 index 00000000..6bcf13df --- /dev/null +++ b/internal/taskrun/repository.go @@ -0,0 +1,18 @@ +package taskrun + +import ( + "context" + "fmt" + + "chat-agent-system/internal/app/uow" +) + +// Repository is a M3 stub. M1 must not write task_runs through this port. +type Repository struct{} + +func (Repository) ForbiddenInM1(ctx context.Context, scope uow.TxScope) error { + if _, err := uow.AsPG(scope); err != nil { + return err + } + return fmt.Errorf("taskrun: repository writes are not enabled until M3") +} From ec22843297cf98dd421d476fd56f8574ed565b10 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 13:54:33 +0000 Subject: [PATCH 05/11] docs(ops): mark U2 UnitOfWork complete on the pilot task Record that cross-owner rollback is implemented so U3 can take document commands next. Co-authored-by: Cursor --- .doc/ops/short-term/2026-09-14-document-reliability-pilot.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md index 978be537..16f787ec 100644 --- a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -34,7 +34,7 @@ mutex: docgov 1. [x] 从最新 `dev`（`a8f9451`）创建实现分支，登记 CURRENT 并确认 `docgov` mutex 已释放。 2. [x] U1：四类规则 arch-check + archtest + 可重放基线（新违规 block）。 -3. [ ] 实现 U2，验证任意属主写入/commit 故障整体回滚。 +3. [x] 实现 U2，验证任意属主写入/commit 故障整体回滚。 4. [ ] 实现 U3，删除提交后 outbox 正确性回调。 5. [ ] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 From ade618b8540fc0832e688a737375981faf54a280 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:02:21 +0000 Subject: [PATCH 06/11] feat(docflow): own document commands in one UnitOfWork Lock the document row and re-authorize duty before edit, agent commit, approve, or reject share a single transaction. Characterization tests pin ETag races, one legal approve/reject terminal, and same-key digest conflicts. Co-authored-by: Cursor --- .../2026-09-14-document-reliability-pilot.md | 2 +- .doc/system-design/module-boundaries.yaml | 8 + internal/app/docflow/commands.go | 171 ++++++++++++++++++ internal/app/docflow/commands_test.go | 171 ++++++++++++++++++ internal/app/uow/uow.go | 18 ++ internal/docgov/engine.go | 68 +++++-- internal/docgov/repository.go | 6 +- .../document_commit_concurrency_pg_test.go | 100 ++++++++++ internal/docrevision/pg_store.go | 9 + internal/docrevision/service.go | 9 +- internal/server/document_revision_handlers.go | 57 ++++-- internal/server/handlers.go | 111 +++++++----- 12 files changed, 652 insertions(+), 78 deletions(-) create mode 100644 internal/app/docflow/commands.go create mode 100644 internal/app/docflow/commands_test.go create mode 100644 internal/docrevision/document_commit_concurrency_pg_test.go diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md index 16f787ec..bfb0bd10 100644 --- a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -35,7 +35,7 @@ mutex: docgov 1. [x] 从最新 `dev`（`a8f9451`）创建实现分支，登记 CURRENT 并确认 `docgov` mutex 已释放。 2. [x] U1：四类规则 arch-check + archtest + 可重放基线（新违规 block）。 3. [x] 实现 U2，验证任意属主写入/commit 故障整体回滚。 -4. [ ] 实现 U3，删除提交后 outbox 正确性回调。 +4. [x] 实现 U3，删除提交后 outbox 正确性回调。 5. [ ] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 7. [ ] 写 test report，更新验收清单、FEATURE-MATRIX、CURRENT 和 gap-matrix。 diff --git a/.doc/system-design/module-boundaries.yaml b/.doc/system-design/module-boundaries.yaml index f523063e..9b7830a8 100644 --- a/.doc/system-design/module-boundaries.yaml +++ b/.doc/system-design/module-boundaries.yaml @@ -55,6 +55,10 @@ imports: db: [] docduty: [] app: [] + app/docflow: + - app/uow + - docgov + - docrevision app/uow: [] docgov: - app/uow @@ -89,6 +93,8 @@ imports: replyexport: [] server: - agent + - app/docflow + - app/uow - auth - authz - bizdev @@ -171,10 +177,12 @@ path_owners: event_producers: outbox.notify: - docrevision + - app/docflow document.agent_candidate_ready: - docrevision document.agent_version_committed: - docrevision + - app/docflow document.downstream_impact_detected: - docrevision document.change_report_degraded: diff --git a/internal/app/docflow/commands.go b/internal/app/docflow/commands.go new file mode 100644 index 00000000..f6500e22 --- /dev/null +++ b/internal/app/docflow/commands.go @@ -0,0 +1,171 @@ +package docflow + +import ( + "context" + "fmt" + + "chat-agent-system/internal/app/uow" + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" +) + +type Action string + +const ( + ActionEdit Action = "document.edit" + ActionApprove Action = "document.approve" + ActionReject Action = "document.reject" + ActionCommit Action = "document.agent_commit" +) + +type Principal struct { + UserID int64 + Name string + Role docgov.UserRole +} + +type HumanEditCommand struct { + DocID string + Request docrevision.HumanRevisionRequest +} + +type AgentCommitCommand struct { + DocID string + CandidateID int64 + Decision docrevision.CommitAgentDecision +} + +type DecisionCommand struct { + DocID string + Comment string + RejectReason string + RollbackTo docgov.DocumentState +} + +type Commands struct { + UoW uow.UnitOfWork + Gov *docgov.GovernanceEngine + Rev *docrevision.Service + Auth Authorizer +} + +type Authorizer interface { + Authorize(ctx context.Context, p Principal, rec *docgov.DocumentRecord, action Action) error +} + +type dutyAuthorizer struct{} + +func DutyAuthorizer() Authorizer { return dutyAuthorizer{} } + +func (dutyAuthorizer) Authorize(_ context.Context, p Principal, rec *docgov.DocumentRecord, action Action) error { + if rec == nil { + return uow.ErrPermissionDenied + } + switch action { + case ActionEdit, ActionCommit: + if rec.AuthorID != 0 && p.UserID != 0 && rec.AuthorID != p.UserID { + return uow.ErrPermissionDenied + } + case ActionApprove, ActionReject: + if rec.ApproverID != 0 && p.UserID != 0 && rec.ApproverID != p.UserID { + return uow.ErrPermissionDenied + } + default: + return uow.ErrPermissionDenied + } + return nil +} + +func (c *Commands) run(ctx context.Context, docID string, p Principal, action Action, fn func(context.Context, uow.TxScope) error) error { + if c == nil { + return fmt.Errorf("docflow: commands required") + } + if c.Auth == nil { + c.Auth = DutyAuthorizer() + } + if c.UoW == nil { + if err := c.authorizeUnlocked(ctx, docID, p, action); err != nil { + return err + } + return fn(ctx, nil) + } + return c.UoW.Within(ctx, func(scope uow.TxScope) error { + ctx = uow.WithScope(ctx, scope) + if docID != "" && c.Gov != nil && c.Gov.UsingPG() { + rec, err := (docgov.Repository{}).GetForUpdate(ctx, scope, docID) + if err != nil { + if action == ActionApprove || action == ActionReject { + return err + } + } else if err := c.Auth.Authorize(ctx, p, rec, action); err != nil { + return err + } + } + return fn(ctx, scope) + }) +} + +func (c *Commands) authorizeUnlocked(ctx context.Context, docID string, p Principal, action Action) error { + if docID == "" || c.Gov == nil { + return nil + } + rec, err := c.Gov.GetDocument(ctx, docID) + if err != nil || rec == nil { + if action == ActionApprove || action == ActionReject { + if err != nil { + return err + } + return uow.ErrPermissionDenied + } + return nil + } + return c.Auth.Authorize(ctx, p, rec, action) +} + +func (c *Commands) HumanEdit(ctx context.Context, p Principal, cmd HumanEditCommand) (*docrevision.HumanRevisionResult, error) { + var out *docrevision.HumanRevisionResult + err := c.run(ctx, cmd.DocID, p, ActionEdit, func(ctx context.Context, _ uow.TxScope) error { + res, err := c.Rev.CommitHumanRevision(ctx, cmd.Request) + out = res + return err + }) + return out, err +} + +func (c *Commands) AgentCommit(ctx context.Context, p Principal, cmd AgentCommitCommand) (*docrevision.RevisionState, error) { + var out *docrevision.RevisionState + err := c.run(ctx, cmd.DocID, p, ActionCommit, func(ctx context.Context, _ uow.TxScope) error { + st, err := c.Rev.CommitAgentVersion(ctx, cmd.DocID, cmd.CandidateID, cmd.Decision) + out = st + return err + }) + if err == nil && c.UoW != nil && c.Rev != nil && c.Rev.Notify && c.Rev.OnOutbox != nil && out != nil { + _ = c.Rev.OnOutbox(ctx, "document.agent_version_committed", map[string]any{ + "doc_id": cmd.DocID, "version_no": out.CurrentVersionNo, + }) + } + return out, err +} + +func (c *Commands) Approve(ctx context.Context, p Principal, cmd DecisionCommand) (*docgov.DocumentRecord, error) { + var out *docgov.DocumentRecord + err := c.run(ctx, cmd.DocID, p, ActionApprove, func(ctx context.Context, _ uow.TxScope) error { + doc, err := c.Gov.ApproverApprove(ctx, cmd.DocID, p.UserID, p.Role, p.Name, cmd.Comment) + out = doc + return err + }) + if err == nil && out != nil && c.UoW != nil && c.Gov != nil { + c.Gov.RunApprovedHooksAfterCommit(ctx, out) + } + return out, err +} + +func (c *Commands) Reject(ctx context.Context, p Principal, cmd DecisionCommand) (*docgov.DocumentRecord, error) { + var out *docgov.DocumentRecord + err := c.run(ctx, cmd.DocID, p, ActionReject, func(ctx context.Context, _ uow.TxScope) error { + doc, err := c.Gov.ApproverReject(ctx, cmd.DocID, p.UserID, p.Role, p.Name, cmd.RejectReason, cmd.RollbackTo) + out = doc + return err + }) + return out, err +} diff --git a/internal/app/docflow/commands_test.go b/internal/app/docflow/commands_test.go new file mode 100644 index 00000000..7a08648f --- /dev/null +++ b/internal/app/docflow/commands_test.go @@ -0,0 +1,171 @@ +package docflow + +import ( + "context" + "fmt" + "sync" + "testing" + + "chat-agent-system/internal/app/uow" + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" +) + +func TestHumanEditCharacterizationSameETagOneWinner(t *testing.T) { + ctx := context.Background() + rev := docrevision.NewService(nil) + st, err := rev.OpenTemplateVersion(ctx, "doc_flow_etag", "# base\n", docrevision.ActorSnapshot{}, 1, "") + if err != nil { + t.Fatal(err) + } + cmds := &Commands{Rev: rev} + var wg sync.WaitGroup + var mu sync.Mutex + wins, losses := 0, 0 + for i := 0; i < 8; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + _, err := cmds.HumanEdit(ctx, Principal{UserID: int64(i + 1)}, HumanEditCommand{ + DocID: "doc_flow_etag", + Request: docrevision.HumanRevisionRequest{ + DocID: "doc_flow_etag", Markdown: fmt.Sprintf("# base\n\nedit %d\n", i), + ExpectedHeadSnapshot: &st.HeadSnapshotID, ExpectedRevisionNo: &st.HeadRevisionNo, + IdempotencyKey: fmt.Sprintf("flow-%d", i), ActorUserID: int64(i + 1), LifecycleState: "drafting", + }, + }) + mu.Lock() + defer mu.Unlock() + if err == nil { + wins++ + } else if err == docrevision.ErrHeadConflict { + losses++ + } else { + t.Errorf("unexpected %v", err) + } + }(i) + } + wg.Wait() + if wins != 1 || losses != 7 { + t.Fatalf("wins=%d losses=%d", wins, losses) + } + got, err := rev.GetRevisionState(ctx, "doc_flow_etag") + if err != nil || got.HeadRevisionNo != 1 { + t.Fatalf("head=%+v err=%v", got, err) + } +} + +func TestApproveRejectCharacterizationOneTerminal(t *testing.T) { + ctx := context.Background() + gov := docgov.NewGovernanceEngine(nil) + doc, err := gov.CreateOrGetDocument(ctx, "doc_flow_appr", 1, "intake", "t", "V0.5_feasibility", "", 1, "A", 2, "R", 3, "P") + if err != nil { + t.Fatal(err) + } + if _, err := gov.SubmitForReview(ctx, doc.DocID, 1, docgov.RoleAuthor, "A"); err != nil { + t.Fatal(err) + } + if _, err := gov.ReviewApprove(ctx, doc.DocID, 2, docgov.RoleReviewer, "R", "ok"); err != nil { + t.Fatal(err) + } + cmds := &Commands{Gov: gov} + var wg sync.WaitGroup + results := make(chan string, 2) + wg.Add(2) + go func() { + defer wg.Done() + doc, err := cmds.Approve(ctx, Principal{UserID: 3, Role: docgov.RoleApprover, Name: "P"}, DecisionCommand{DocID: doc.DocID}) + if err == nil && doc != nil { + results <- string(doc.CurrentState) + } else { + results <- "err" + } + }() + go func() { + defer wg.Done() + doc, err := cmds.Reject(ctx, Principal{UserID: 3, Role: docgov.RoleApprover, Name: "P"}, DecisionCommand{ + DocID: doc.DocID, RejectReason: "no", RollbackTo: docgov.StateDrafting, + }) + if err == nil && doc != nil { + results <- string(doc.CurrentState) + } else { + results <- "err" + } + }() + wg.Wait() + close(results) + var states []string + for s := range results { + states = append(states, s) + } + if len(states) != 2 { + t.Fatalf("%v", states) + } + approved, rejected := 0, 0 + for _, s := range states { + switch s { + case string(docgov.StateApproved): + approved++ + case string(docgov.StateDrafting), string(docgov.StateRejected): + rejected++ + } + } + if approved+rejected != 1 { + t.Fatalf("expected one legal terminal, got %v", states) + } +} + +func TestIdempotentHumanEditSameDigest(t *testing.T) { + ctx := context.Background() + rev := docrevision.NewService(nil) + st, err := rev.OpenTemplateVersion(ctx, "doc_flow_idemp", "# base\n", docrevision.ActorSnapshot{}, 1, "") + if err != nil { + t.Fatal(err) + } + cmds := &Commands{Rev: rev} + req := docrevision.HumanRevisionRequest{ + DocID: "doc_flow_idemp", Markdown: "# base\n\nedit\n", + ExpectedHeadSnapshot: &st.HeadSnapshotID, ExpectedRevisionNo: &st.HeadRevisionNo, + IdempotencyKey: "same-key", ActorUserID: 1, LifecycleState: "drafting", + } + first, err := cmds.HumanEdit(ctx, Principal{UserID: 1}, HumanEditCommand{DocID: req.DocID, Request: req}) + if err != nil { + t.Fatal(err) + } + second, err := cmds.HumanEdit(ctx, Principal{UserID: 1}, HumanEditCommand{DocID: req.DocID, Request: req}) + if err != nil { + t.Fatal(err) + } + if first.State.HeadSnapshotID != second.State.HeadSnapshotID { + t.Fatal("same key+digest must replay") + } + req.Markdown = "# base\n\nother digest\n" + if _, err := cmds.HumanEdit(ctx, Principal{UserID: 1}, HumanEditCommand{DocID: req.DocID, Request: req}); err != docrevision.ErrIdempotencyConflict { + t.Fatalf("different digest: %v", err) + } +} + +func TestReauthDeniesWrongAuthor(t *testing.T) { + ctx := context.Background() + gov := docgov.NewGovernanceEngine(nil) + if _, err := gov.CreateOrGetDocument(ctx, "doc_flow_authz", 1, "intake", "t", "V0.5_feasibility", "", 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + rev := docrevision.NewService(nil) + st, err := rev.OpenTemplateVersion(ctx, "doc_flow_authz", "# base\n", docrevision.ActorSnapshot{}, 1, "") + if err != nil { + t.Fatal(err) + } + cmds := &Commands{Gov: gov, Rev: rev} + _, err = cmds.HumanEdit(ctx, Principal{UserID: 99}, HumanEditCommand{ + DocID: "doc_flow_authz", + Request: docrevision.HumanRevisionRequest{ + DocID: "doc_flow_authz", Markdown: "# base\n\nnope\n", + ExpectedHeadSnapshot: &st.HeadSnapshotID, ExpectedRevisionNo: &st.HeadRevisionNo, + IdempotencyKey: "authz-deny", ActorUserID: 99, LifecycleState: "drafting", + }, + }) + if err != uow.ErrPermissionDenied { + t.Fatalf("want permission denied, got %v", err) + } +} diff --git a/internal/app/uow/uow.go b/internal/app/uow/uow.go index e77267d6..6aa18a1c 100644 --- a/internal/app/uow/uow.go +++ b/internal/app/uow/uow.go @@ -39,3 +39,21 @@ type RetryPolicy struct { func DefaultRetryPolicy() RetryPolicy { return RetryPolicy{MaxAttempts: DefaultMaxAttempts, LockTimeout: DefaultLockTimeout} } + +type scopeCtxKey struct{} + +// WithScope lets owner repositories join the command UnitOfWork. +func WithScope(ctx context.Context, scope TxScope) context.Context { + if ctx == nil { + ctx = context.Background() + } + return context.WithValue(ctx, scopeCtxKey{}, scope) +} + +func ScopeFrom(ctx context.Context) (TxScope, bool) { + if ctx == nil { + return nil, false + } + s, ok := ctx.Value(scopeCtxKey{}).(TxScope) + return s, ok && s != nil +} diff --git a/internal/docgov/engine.go b/internal/docgov/engine.go index 28e017a4..cc8c55cc 100644 --- a/internal/docgov/engine.go +++ b/internal/docgov/engine.go @@ -8,6 +8,8 @@ import ( "sync" "time" + "chat-agent-system/internal/app/uow" + "github.com/jackc/pgx/v5/pgxpool" ) @@ -69,6 +71,14 @@ func (e *GovernanceEngine) runApprovedHooks(ctx context.Context, doc *DocumentRe } } +func (e *GovernanceEngine) UsingPG() bool { + return e != nil && e.pool != nil +} + +func (e *GovernanceEngine) RunApprovedHooksAfterCommit(ctx context.Context, doc *DocumentRecord) { + e.runApprovedHooks(ctx, doc) +} + // NewGovernanceEngine 初始化治理引擎 func NewGovernanceEngine(pool *pgxpool.Pool) *GovernanceEngine { return &GovernanceEngine{ @@ -783,7 +793,9 @@ func (e *GovernanceEngine) SubmitForReview(ctx context.Context, docID string, ac } e.memAudits[docID] = append(e.memAudits[docID], audit) - e.syncDocAndAuditToDB(ctx, doc, audit) + if err := e.syncDocAndAuditToDB(ctx, doc, audit); err != nil { + return nil, err + } return doc, nil } @@ -811,7 +823,9 @@ func (e *GovernanceEngine) ReviewApprove(ctx context.Context, docID string, acto } e.memAudits[docID] = append(e.memAudits[docID], audit) - e.syncDocAndAuditToDB(ctx, doc, audit) + if err := e.syncDocAndAuditToDB(ctx, doc, audit); err != nil { + return nil, err + } return doc, nil } @@ -838,7 +852,9 @@ func (e *GovernanceEngine) ReviewReject(ctx context.Context, docID string, actor _ = e.addCommentInternal(docID, actorID, actorName, actorRole, "", CommentRejection, rejectReason) e.memAudits[docID] = append(e.memAudits[docID], audit) - e.syncDocAndAuditToDB(ctx, doc, audit) + if err := e.syncDocAndAuditToDB(ctx, doc, audit); err != nil { + return nil, err + } return doc, nil } @@ -866,11 +882,15 @@ func (e *GovernanceEngine) ApproverApprove(ctx context.Context, docID string, ac } e.memAudits[docID] = append(e.memAudits[docID], audit) - e.syncDocAndAuditToDB(ctx, doc, audit) + if err := e.syncDocAndAuditToDB(ctx, doc, audit); err != nil { + return nil, err + } - e.mu.Unlock() - e.runApprovedHooks(ctx, doc) - e.mu.Lock() + if _, joined := uow.ScopeFrom(ctx); !joined { + e.mu.Unlock() + e.runApprovedHooks(ctx, doc) + e.mu.Lock() + } return doc, nil } @@ -897,7 +917,9 @@ func (e *GovernanceEngine) ApproverReject(ctx context.Context, docID string, act _ = e.addCommentInternal(docID, actorID, actorName, actorRole, "", CommentRejection, rejectReason) e.memAudits[docID] = append(e.memAudits[docID], audit) - e.syncDocAndAuditToDB(ctx, doc, audit) + if err := e.syncDocAndAuditToDB(ctx, doc, audit); err != nil { + return nil, err + } return doc, nil } @@ -1068,11 +1090,23 @@ func (e *GovernanceEngine) addCommentInternal(docID string, userID int64, userNa return nil } -func (e *GovernanceEngine) syncDocAndAuditToDB(ctx context.Context, doc *DocumentRecord, audit *AuditLog) { +func (e *GovernanceEngine) syncDocAndAuditToDB(ctx context.Context, doc *DocumentRecord, audit *AuditLog) error { if e.pool == nil { - return + return nil } - _, _ = e.pool.Exec(ctx, ` + exec := func(sql string, args ...any) error { + if scope, ok := uow.ScopeFrom(ctx); ok { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + _, err = tx.Exec(ctx, sql, args...) + return err + } + _, err := e.pool.Exec(ctx, sql, args...) + return err + } + if err := exec(` UPDATE document_records SET current_state = $1, is_locked = $2, last_rejected_reason = $3, last_rejected_by_role = $4, last_rejected_by_name = $5, last_rejected_at = $6, @@ -1080,14 +1114,18 @@ func (e *GovernanceEngine) syncDocAndAuditToDB(ctx context.Context, doc *Documen WHERE doc_id = $9 `, string(doc.CurrentState), doc.IsLocked, doc.LastRejectedReason, doc.LastRejectedByRole, doc.LastRejectedByName, doc.LastRejectedAt, - doc.ApprovedAt, doc.UpdatedAt, doc.DocID) - + doc.ApprovedAt, doc.UpdatedAt, doc.DocID); err != nil { + return fmt.Errorf("docgov: sync document: %w", err) + } if audit != nil { detJSON, _ := json.Marshal(audit.Details) - _, _ = e.pool.Exec(ctx, ` + if err := exec(` INSERT INTO document_audit_logs (doc_id, action, actor_id, actor_name, actor_role, from_state, to_state, details, created_at) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9) `, audit.DocID, audit.Action, audit.ActorID, audit.ActorName, string(audit.ActorRole), - string(audit.FromState), string(audit.ToState), detJSON, audit.CreatedAt) + string(audit.FromState), string(audit.ToState), detJSON, audit.CreatedAt); err != nil { + return fmt.Errorf("docgov: sync audit: %w", err) + } } + return nil } diff --git a/internal/docgov/repository.go b/internal/docgov/repository.go index 8c1c31da..e14ae50a 100644 --- a/internal/docgov/repository.go +++ b/internal/docgov/repository.go @@ -58,10 +58,12 @@ func (Repository) GetForUpdate(ctx context.Context, scope uow.TxScope, docID str var state string err = tx.QueryRow(ctx, ` SELECT doc_id, project_id, task_key, title, stage, version, current_state, - is_locked, is_template_only, content_sha256, work_dir + is_locked, is_template_only, content_sha256, work_dir, + COALESCE(author_id, 0), COALESCE(reviewer_id, 0), COALESCE(approver_id, 0) FROM document_records WHERE doc_id=$1 FOR UPDATE`, docID).Scan( &rec.DocID, &rec.ProjectID, &rec.TaskKey, &rec.Title, &rec.Stage, &rec.Version, &state, - &rec.IsLocked, &rec.IsTemplateOnly, &rec.ContentSHA256, &rec.WorkDir) + &rec.IsLocked, &rec.IsTemplateOnly, &rec.ContentSHA256, &rec.WorkDir, + &rec.AuthorID, &rec.ReviewerID, &rec.ApproverID) if err != nil { return nil, fmt.Errorf("docgov: lock document: %w", err) } diff --git a/internal/docrevision/document_commit_concurrency_pg_test.go b/internal/docrevision/document_commit_concurrency_pg_test.go new file mode 100644 index 00000000..15e0daa5 --- /dev/null +++ b/internal/docrevision/document_commit_concurrency_pg_test.go @@ -0,0 +1,100 @@ +package docrevision_test + +import ( + "context" + "fmt" + "strings" + "sync" + "testing" + "time" + + "chat-agent-system/internal/app/docflow" + "chat-agent-system/internal/app/uow" + "chat-agent-system/internal/db" + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" + + "github.com/jackc/pgx/v5/pgxpool" +) + +func TestDocumentCommitConcurrencyPG(t *testing.T) { + if testing.Short() { + t.Skip("postgresql concurrency is not part of -short") + } + dsn := db.SkipIfNoTestDatabaseURL(t) + if strings.Contains(dsn, "5433") || strings.Contains(dsn, "agent_system_dev") || + strings.Contains(dsn, "5434") && strings.Contains(dsn, "agent_system_test") || + strings.Contains(dsn, "5435") { + t.Fatal("TEST_DATABASE_URL points at a shared env port; refuse agent_system_dev/test/main") + } + ctx := context.Background() + pool, err := pgxpool.New(ctx, dsn) + if err != nil { + t.Fatal(err) + } + t.Cleanup(pool.Close) + if err := db.RunMigrationsOnceForTest(ctx, pool, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + var projectID int64 + if err := pool.QueryRow(ctx, `INSERT INTO projects (code, name, status) VALUES ($1,$2,'active') RETURNING id`, + "docflow-"+suffix, "docflow").Scan(&projectID); err != nil { + t.Fatalf("project: %v", err) + } + t.Cleanup(func() { + _, _ = pool.Exec(context.Background(), `DELETE FROM document_change_events WHERE doc_id LIKE $1`, "docflow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_versions WHERE doc_id LIKE $1`, "docflow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_snapshots WHERE doc_id LIKE $1`, "docflow_"+suffix+"%") + _, _ = pool.Exec(context.Background(), `DELETE FROM document_records WHERE project_id=$1`, projectID) + _, _ = pool.Exec(context.Background(), `DELETE FROM projects WHERE id=$1`, projectID) + }) + + docID := "docflow_" + suffix + rev := docrevision.NewService(pool) + gov := docgov.NewGovernanceEngine(pool) + if _, err := gov.CreateOrGetDocument(ctx, docID, projectID, "intake", "t", "V0.5_feasibility", "ws/"+docID, 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + st, err := rev.OpenTemplateVersion(ctx, docID, "# base\n", docrevision.ActorSnapshot{}, projectID, "ws/"+docID) + if err != nil { + t.Fatal(err) + } + cmds := &docflow.Commands{UoW: uow.NewPG(pool), Gov: gov, Rev: rev} + + var wg sync.WaitGroup + wins := 0 + var mu sync.Mutex + for i := 0; i < 2; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + _, err := cmds.HumanEdit(ctx, docflow.Principal{UserID: 1}, docflow.HumanEditCommand{ + DocID: docID, + Request: docrevision.HumanRevisionRequest{ + DocID: docID, Markdown: fmt.Sprintf("# base\n\nuser %d\n", i), + ExpectedHeadSnapshot: &st.HeadSnapshotID, ExpectedRevisionNo: &st.HeadRevisionNo, + IdempotencyKey: fmt.Sprintf("pg-%s-%d", suffix, i), ActorUserID: 1, + LifecycleState: "drafting", ProjectID: projectID, WorkDir: "ws/" + docID, + }, + }) + if err == nil { + mu.Lock() + wins++ + mu.Unlock() + } + }(i) + } + wg.Wait() + if wins != 1 { + t.Fatalf("same etag concurrent edits: wins=%d", wins) + } + got, err := rev.GetRevisionState(ctx, docID) + if err != nil { + t.Fatal(err) + } + if got.HeadRevisionNo != 1 { + t.Fatalf("head revision=%d", got.HeadRevisionNo) + } +} diff --git a/internal/docrevision/pg_store.go b/internal/docrevision/pg_store.go index 5541dbdc..8c8cf96e 100644 --- a/internal/docrevision/pg_store.go +++ b/internal/docrevision/pg_store.go @@ -6,6 +6,8 @@ import ( "errors" "time" + "chat-agent-system/internal/app/uow" + "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgxpool" ) @@ -444,6 +446,13 @@ func (s *Service) withTx(ctx context.Context, fn func(pgx.Tx) error) error { if !s.usingPG() { return nil } + if scope, ok := uow.ScopeFrom(ctx); ok { + tx, err := uow.AsPG(scope) + if err != nil { + return err + } + return fn(tx) + } tx, err := s.pool.Begin(ctx) if err != nil { return err diff --git a/internal/docrevision/service.go b/internal/docrevision/service.go index f0d08ec8..565f1e00 100644 --- a/internal/docrevision/service.go +++ b/internal/docrevision/service.go @@ -8,6 +8,8 @@ import ( "sync" "time" + "chat-agent-system/internal/app/uow" + "github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5/pgxpool" ) @@ -203,6 +205,11 @@ func (s *Service) CommitHumanRevision(ctx context.Context, req HumanRevisionRequ if eid, ok := s.idemp[req.IdempotencyKey]; ok { ev := s.events[eid] + if ev != nil { + if snap := s.snapshots[ev.ToSnapshotID]; snap != nil && snap.ContentSHA256 != sha { + return nil, ErrIdempotencyConflict + } + } st, _ := s.revisionStateLocked(req.DocID) overlay := s.manualOverlayLocked(req.DocID) return &HumanRevisionResult{State: *st, Event: *ev, Overlay: overlay}, nil @@ -672,7 +679,7 @@ func (s *Service) CommitAgentVersion(ctx context.Context, docID string, candidat return nil, err } - if s.Notify && s.OnOutbox != nil { + if _, joined := uow.ScopeFrom(ctx); !joined && s.Notify && s.OnOutbox != nil { _ = s.OnOutbox(ctx, "document.agent_version_committed", map[string]any{ "doc_id": c.DocID, "version_no": newVer.VersionNo, "risk": rep.RiskLevel, "candidate_id": c.ID, }) diff --git a/internal/server/document_revision_handlers.go b/internal/server/document_revision_handlers.go index d7a3b1bc..1ce3ada7 100644 --- a/internal/server/document_revision_handlers.go +++ b/internal/server/document_revision_handlers.go @@ -9,11 +9,24 @@ import ( "strconv" "strings" + "chat-agent-system/internal/app/docflow" + "chat-agent-system/internal/app/uow" "chat-agent-system/internal/docgov" "chat-agent-system/internal/docrevision" "chat-agent-system/internal/freedoc" ) +func (s *Server) docflowCommands() *docflow.Commands { + if s == nil || s.svc == nil { + return nil + } + cmds := &docflow.Commands{Gov: s.svc.DocGov(), Rev: s.svc.DocRevision()} + if pool := s.svc.Pool(); pool != nil { + cmds.UoW = uow.NewPG(pool) + } + return cmds +} + func (s *Server) registerDocumentRevisionRoutes(mux *http.ServeMux) { mux.HandleFunc("GET /api/v1/documents/{doc_id}/revision-state", s.handleDocRevisionState) mux.HandleFunc("GET /api/v1/documents/{doc_id}/versions", s.handleDocRevisionListVersions) @@ -235,12 +248,20 @@ func (s *Server) handleDocRevisionPatchContent(w http.ResponseWriter, r *http.Re return } } - res, err := rev.CommitHumanRevision(r.Context(), docrevision.HumanRevisionRequest{ - DocID: docID, Markdown: body.Markdown, ExpectedHeadSnapshot: expectedHead, ExpectedRevisionNo: expectedRev, - IdempotencyKey: idemp, EditSessionID: body.EditSessionID, ActorUserID: user.ID, - ActorSnapshot: docrevision.ActorSnapshot{DisplayName: user.Name, Role: "author", Duty: "author"}, - LifecycleState: lifecycle, WorkDir: workDir, ProjectID: projectID, AllowCreateBaseline: false, - ClientBlockMap: body.ClientBlockMap, + cmds := s.docflowCommands() + if cmds == nil || cmds.Rev == nil { + writeJSONResponse(w, http.StatusServiceUnavailable, map[string]any{"error": "docrevision unavailable"}) + return + } + res, err := cmds.HumanEdit(r.Context(), docflow.Principal{UserID: user.ID, Name: user.Name, Role: docgov.RoleAuthor}, docflow.HumanEditCommand{ + DocID: docID, + Request: docrevision.HumanRevisionRequest{ + DocID: docID, Markdown: body.Markdown, ExpectedHeadSnapshot: expectedHead, ExpectedRevisionNo: expectedRev, + IdempotencyKey: idemp, EditSessionID: body.EditSessionID, ActorUserID: user.ID, + ActorSnapshot: docrevision.ActorSnapshot{DisplayName: user.Name, Role: "author", Duty: "author"}, + LifecycleState: lifecycle, WorkDir: workDir, ProjectID: projectID, AllowCreateBaseline: false, + ClientBlockMap: body.ClientBlockMap, + }, }) if errors.Is(err, docrevision.ErrHeadConflict) && res != nil && res.Conflict != nil { writeJSONResponse(w, http.StatusConflict, map[string]any{ @@ -343,11 +364,19 @@ func (s *Server) handleDocRevisionCandidateDecision(w http.ResponseWriter, r *ht return } cand, _ := rev.GetCandidate(r.Context(), docID, cid) - st, err := rev.CommitAgentVersion(r.Context(), docID, cid, docrevision.CommitAgentDecision{ - Decision: body.Decision, Confirmed: body.Confirmed, - ExpectedInputSHA: body.ExpectedInputSHA, ExpectedOutputSHA: body.ExpectedOutputSHA, - Reason: body.Reason, ActorUserID: user.ID, - ActorSnapshot: docrevision.ActorSnapshot{DisplayName: user.Name}, + cmds := s.docflowCommands() + if cmds == nil || cmds.Rev == nil { + writeJSONResponse(w, http.StatusServiceUnavailable, map[string]any{"error": "docrevision unavailable"}) + return + } + st, err := cmds.AgentCommit(r.Context(), docflow.Principal{UserID: user.ID, Name: user.Name, Role: docgov.RoleAuthor}, docflow.AgentCommitCommand{ + DocID: docID, CandidateID: cid, + Decision: docrevision.CommitAgentDecision{ + Decision: body.Decision, Confirmed: body.Confirmed, + ExpectedInputSHA: body.ExpectedInputSHA, ExpectedOutputSHA: body.ExpectedOutputSHA, + Reason: body.Reason, ActorUserID: user.ID, + ActorSnapshot: docrevision.ActorSnapshot{DisplayName: user.Name}, + }, }) if err != nil { writeDocRevisionError(w, err) @@ -457,8 +486,12 @@ func writeDocRevisionError(w http.ResponseWriter, err error) { writeJSONResponse(w, http.StatusUnprocessableEntity, map[string]any{"code": docrevision.ErrCodeReportInvalid, "error": err.Error()}) case errors.Is(err, docrevision.ErrTooLarge): writeJSONResponse(w, http.StatusRequestEntityTooLarge, map[string]any{"code": docrevision.ErrCodeTooLarge, "error": err.Error()}) - case errors.Is(err, docrevision.ErrForbidden): + case errors.Is(err, docrevision.ErrForbidden), errors.Is(err, uow.ErrPermissionDenied): writeJSONResponse(w, http.StatusForbidden, map[string]any{"code": docrevision.ErrCodeEditForbidden, "error": err.Error()}) + case errors.Is(err, docrevision.ErrIdempotencyConflict), errors.Is(err, uow.ErrDigestConflict): + writeJSONResponse(w, http.StatusConflict, map[string]any{"code": "DOCUMENT_IDEMPOTENCY_CONFLICT", "error": err.Error()}) + case errors.Is(err, uow.ErrBusy): + writeJSONResponse(w, http.StatusConflict, map[string]any{"code": "DOCUMENT_BUSY", "error": err.Error()}) default: writeJSONResponse(w, http.StatusBadRequest, map[string]any{"error": err.Error()}) } diff --git a/internal/server/handlers.go b/internal/server/handlers.go index 09bbe592..37eccceb 100644 --- a/internal/server/handlers.go +++ b/internal/server/handlers.go @@ -10,6 +10,8 @@ import ( "strings" "time" + "chat-agent-system/internal/app/docflow" + "chat-agent-system/internal/app/uow" "chat-agent-system/internal/bizdev" "chat-agent-system/internal/ctxmgr" "chat-agent-system/internal/docgov" @@ -21,15 +23,15 @@ import ( // HealthResponse 健康检查响应 type HealthResponse struct { - Status string `json:"status"` - Env string `json:"env"` - AgentBackend string `json:"agent_backend"` - Version string `json:"version"` - SharedKBPort int `json:"shared_kb_port"` - DatabaseReady bool `json:"database_ready"` - SchemaVersion int64 `json:"schema_version,omitempty"` - SchemaReady bool `json:"schema_ready,omitempty"` - TaskStoreReady bool `json:"task_store_ready,omitempty"` + Status string `json:"status"` + Env string `json:"env"` + AgentBackend string `json:"agent_backend"` + Version string `json:"version"` + SharedKBPort int `json:"shared_kb_port"` + DatabaseReady bool `json:"database_ready"` + SchemaVersion int64 `json:"schema_version,omitempty"` + SchemaReady bool `json:"schema_ready,omitempty"` + TaskStoreReady bool `json:"task_store_ready,omitempty"` } // ChatRequest 对话请求 @@ -50,32 +52,32 @@ type ChatRequest struct { // TaskRunRequestDTO 独立任务运行请求 type TaskRunRequestDTO struct { - TaskKey string `json:"task_key"` - Stage string `json:"stage,omitempty"` // 若指定 stage，则执行成熟度阶段跃迁 - ProjectName string `json:"project_name"` - CustomerName string `json:"customer_name"` - DeviceType string `json:"device_type"` - IntendedUse string `json:"intended_use"` - TargetMarket string `json:"target_market"` - Fields map[string]string `json:"fields,omitempty"` - Backend string `json:"backend,omitempty"` - AutoRepair bool `json:"auto_repair"` - UserID int64 `json:"user_id,omitempty"` - ProjectID int64 `json:"project_id,omitempty"` - DocID string `json:"doc_id,omitempty"` - AuthorID int64 `json:"author_id,omitempty"` - AuthorName string `json:"author_name,omitempty"` - ReviewerID int64 `json:"reviewer_id,omitempty"` - ReviewerName string `json:"reviewer_name,omitempty"` - ApproverID int64 `json:"approver_id,omitempty"` - ApproverName string `json:"approver_name,omitempty"` - ForceDirectEdit bool `json:"force_direct_edit,omitempty"` - CustomSteps []squad.StepSpec `json:"custom_steps,omitempty"` - WorkflowKey string `json:"workflow_key,omitempty"` - SessionID string `json:"session_id,omitempty"` - ClientTurnID string `json:"client_turn_id,omitempty"` + TaskKey string `json:"task_key"` + Stage string `json:"stage,omitempty"` // 若指定 stage，则执行成熟度阶段跃迁 + ProjectName string `json:"project_name"` + CustomerName string `json:"customer_name"` + DeviceType string `json:"device_type"` + IntendedUse string `json:"intended_use"` + TargetMarket string `json:"target_market"` + Fields map[string]string `json:"fields,omitempty"` + Backend string `json:"backend,omitempty"` + AutoRepair bool `json:"auto_repair"` + UserID int64 `json:"user_id,omitempty"` + ProjectID int64 `json:"project_id,omitempty"` + DocID string `json:"doc_id,omitempty"` + AuthorID int64 `json:"author_id,omitempty"` + AuthorName string `json:"author_name,omitempty"` + ReviewerID int64 `json:"reviewer_id,omitempty"` + ReviewerName string `json:"reviewer_name,omitempty"` + ApproverID int64 `json:"approver_id,omitempty"` + ApproverName string `json:"approver_name,omitempty"` + ForceDirectEdit bool `json:"force_direct_edit,omitempty"` + CustomSteps []squad.StepSpec `json:"custom_steps,omitempty"` + WorkflowKey string `json:"workflow_key,omitempty"` + SessionID string `json:"session_id,omitempty"` + ClientTurnID string `json:"client_turn_id,omitempty"` // Materials 用户上传的参考文档材料，由前端 Base64 编码后传入；后端 json 反序列化自动解码 []byte - Materials []bizdev.MaterialInput `json:"materials,omitempty"` + Materials []bizdev.MaterialInput `json:"materials,omitempty"` } func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) { @@ -427,11 +429,11 @@ func (s *Server) handleGetUserContext(w http.ResponseWriter, r *http.Request) { // UpdateUserPreferenceRequest 更新用户工作偏好请求 type UpdateUserPreferenceRequest struct { - UserID int64 `json:"user_id"` - CommunicationStyle ctxmgr.CommunicationStyle `json:"communication_style"` - FocusAreas []string `json:"focus_areas"` - OutputFormatPref string `json:"output_format_pref"` - CustomInstructions string `json:"custom_instructions"` + UserID int64 `json:"user_id"` + CommunicationStyle ctxmgr.CommunicationStyle `json:"communication_style"` + FocusAreas []string `json:"focus_areas"` + OutputFormatPref string `json:"output_format_pref"` + CustomInstructions string `json:"custom_instructions"` } // handleUpdateUserPreference 定期整理或前端保存用户习惯偏好 @@ -528,10 +530,10 @@ func (s *Server) handleDocGovGetDocument(w http.ResponseWriter, r *http.Request) // DocGovClaimRequest 认领文档请求 type DocGovClaimRequest struct { - DocID string `json:"doc_id"` - UserID int64 `json:"user_id"` - UserName string `json:"user_name"` - UserRole string `json:"user_role"` // author, reviewer, approver + DocID string `json:"doc_id"` + UserID int64 `json:"user_id"` + UserName string `json:"user_name"` + UserRole string `json:"user_role"` // author, reviewer, approver } // handleDocGovClaim 查阅并认领受控文档 (非模板状态查看方可认领) @@ -609,7 +611,7 @@ type DocGovDecisionRequest struct { UserName string `json:"user_name"` Action string `json:"action"` // approve 或 reject Comment string `json:"comment,omitempty"` - Reason string `json:"reason,omitempty"` // 驳回时必填 + Reason string `json:"reason,omitempty"` // 驳回时必填 RollbackTo string `json:"rollback_to,omitempty"` // 批准人驳回时可选: drafting 或 reviewing } @@ -681,24 +683,39 @@ func (s *Server) handleDocGovApproverDecision(w http.ResponseWriter, r *http.Req } action := strings.ToLower(strings.TrimSpace(req.Action)) + cmds := s.docflowCommands() + if cmds == nil || cmds.Gov == nil { + http.Error(w, "docgov engine not initialized", http.StatusServiceUnavailable) + return + } + p := docflow.Principal{UserID: user.ID, Name: user.Name, Role: docgov.RoleApprover} var doc *docgov.DocumentRecord var err error if action == "approve" { - doc, err = s.svc.DocGov().ApproverApprove(r.Context(), req.DocID, user.ID, docgov.RoleApprover, user.Name, req.Comment) + doc, err = cmds.Approve(r.Context(), p, docflow.DecisionCommand{DocID: req.DocID, Comment: req.Comment}) } else if action == "reject" { reason := req.Reason if reason == "" { reason = req.Comment } - rollbackState := docgov.DocumentState(req.RollbackTo) - doc, err = s.svc.DocGov().ApproverReject(r.Context(), req.DocID, user.ID, docgov.RoleApprover, user.Name, reason, rollbackState) + doc, err = cmds.Reject(r.Context(), p, docflow.DecisionCommand{ + DocID: req.DocID, RejectReason: reason, RollbackTo: docgov.DocumentState(req.RollbackTo), + }) } else { http.Error(w, "invalid action: must be approve or reject", http.StatusBadRequest) return } if err != nil { + if errors.Is(err, uow.ErrPermissionDenied) { + http.Error(w, "approver decision failed: "+err.Error(), http.StatusForbidden) + return + } + if errors.Is(err, uow.ErrBusy) { + http.Error(w, "approver decision failed: "+err.Error(), http.StatusConflict) + return + } http.Error(w, "approver decision failed: "+err.Error(), http.StatusBadRequest) return } From c754f7743d35ee0d0526b6502fadd5d5fbccaae0 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:07:05 +0000 Subject: [PATCH 07/11] feat(docquery): serve ledger when projection is missing Authorize first, then return immutable ledger markdown with typed consistency so a missing or drifted projection is no longer a 404/409. GET stays read-only and FAV degraded read stays off. Co-authored-by: Cursor --- .../2026-09-14-document-reliability-pilot.md | 2 +- .doc/system-design/module-boundaries.yaml | 5 + internal/app/docflow/query.go | 30 +++ internal/docgov/project_docs.go | 3 + internal/docquery/service.go | 196 ++++++++++++++++++ internal/docquery/service_test.go | 167 +++++++++++++++ .../server/project_doc_content_error_test.go | 14 +- internal/server/project_doc_handlers.go | 161 +++++++------- web/src/api/projectDocuments.ts | 10 + web/src/lib/controlledDocHydrate.test.ts | 16 ++ web/src/lib/controlledDocHydrate.ts | 4 +- 11 files changed, 525 insertions(+), 83 deletions(-) create mode 100644 internal/app/docflow/query.go create mode 100644 internal/docquery/service.go create mode 100644 internal/docquery/service_test.go diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md index bfb0bd10..0e4b66e8 100644 --- a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -36,7 +36,7 @@ mutex: docgov 2. [x] U1：四类规则 arch-check + archtest + 可重放基线（新违规 block）。 3. [x] 实现 U2，验证任意属主写入/commit 故障整体回滚。 4. [x] 实现 U3，删除提交后 outbox 正确性回调。 -5. [ ] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 +5. [x] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 7. [ ] 写 test report，更新验收清单、FEATURE-MATRIX、CURRENT 和 gap-matrix。 diff --git a/.doc/system-design/module-boundaries.yaml b/.doc/system-design/module-boundaries.yaml index 9b7830a8..cb788b3e 100644 --- a/.doc/system-design/module-boundaries.yaml +++ b/.doc/system-design/module-boundaries.yaml @@ -58,8 +58,12 @@ imports: app/docflow: - app/uow - docgov + - docquery - docrevision app/uow: [] + docquery: + - docgov + - docrevision docgov: - app/uow - docduty @@ -104,6 +108,7 @@ imports: - ctxmgr - docduty - docgov + - docquery - docrevision - freedoc - intent diff --git a/internal/app/docflow/query.go b/internal/app/docflow/query.go new file mode 100644 index 00000000..ae42fa6b --- /dev/null +++ b/internal/app/docflow/query.go @@ -0,0 +1,30 @@ +package docflow + +import ( + "context" + + "chat-agent-system/internal/docquery" +) + +// Query is the authorized read model for controlled documents. +type Query struct { + Docs *docquery.Service +} + +func (q *Query) Get(ctx context.Context, userID int64, docID string) (*docquery.ContentView, error) { + if q == nil || q.Docs == nil { + return nil, docquery.ErrClosed + } + return q.Docs.Get(ctx, docquery.Principal{UserID: userID}, docquery.DocRef{DocID: docID}) +} + +func (q *Query) BatchGet(ctx context.Context, userID int64, docIDs []string) ([]*docquery.ContentView, error) { + if q == nil || q.Docs == nil { + return nil, docquery.ErrClosed + } + refs := make([]docquery.DocRef, len(docIDs)) + for i, id := range docIDs { + refs[i] = docquery.DocRef{DocID: id} + } + return q.Docs.BatchGet(ctx, docquery.Principal{UserID: userID}, refs) +} diff --git a/internal/docgov/project_docs.go b/internal/docgov/project_docs.go index bf38cba5..30e2cb38 100644 --- a/internal/docgov/project_docs.go +++ b/internal/docgov/project_docs.go @@ -45,6 +45,9 @@ type DocumentContent struct { ContentSHA256 string `json:"content_sha256"` Markdown string `json:"markdown"` IsTemplateOnly bool `json:"is_template_only"` + Consistency string `json:"consistency,omitempty"` + RecordSHA256 string `json:"record_sha256,omitempty"` + FileSHA256 string `json:"file_sha256,omitempty"` } // ListWorkflowDocumentsByProject 列出项目下工作流受控文档（task_key 非空）。 diff --git a/internal/docquery/service.go b/internal/docquery/service.go new file mode 100644 index 00000000..a0b0a0fd --- /dev/null +++ b/internal/docquery/service.go @@ -0,0 +1,196 @@ +package docquery + +import ( + "context" + "errors" + "os" + "strings" + + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" +) + +const ( + ConsistencyAligned = "aligned" + ConsistencyMissing = "missing" + ConsistencyDrift = "drift" + ConsistencyEmpty = "empty" + ConsistencyLedgerMissing = "ledger_missing" + + MaxBatchGet = 100 +) + +var ( + ErrClosed = errors.New("docquery: authorization failed") + ErrBatchLimit = errors.New("docquery: batch exceeds 100") + ErrCrossProject = errors.New("docquery: mixed project references") + ErrLedgerMissing = errors.New("docquery: ledger missing") + ErrFavNotEnabled = errors.New("docquery: FAV degraded read is disabled") +) + +// Principal comes only from the authentication adapter. It must not carry +// caller-supplied role or duty overrides. +type Principal struct { + UserID int64 +} + +// DocRef identifies a document. User/role/duty must not be attached. +type DocRef struct { + DocID string +} + +type ContentView struct { + DocID string `json:"doc_id"` + Version int `json:"version"` + ContentSHA256 string `json:"content_sha256"` + Markdown string `json:"markdown"` + IsTemplateOnly bool `json:"is_template_only"` + Consistency string `json:"consistency"` + RecordSHA256 string `json:"record_sha256,omitempty"` + FileSHA256 string `json:"file_sha256,omitempty"` + HeadSHA256 string `json:"head_sha256,omitempty"` +} + +type Authorizer func(ctx context.Context, p Principal, rec *docgov.DocumentRecord) error + +type Service struct { + Gov *docgov.GovernanceEngine + Rev *docrevision.Service + RepoRoot string + Authorize Authorizer + // FavDegradedRead is an extension point only. M1 keeps it false. + FavDegradedRead bool +} + +func FavDegradedReadEnabled() bool { + switch strings.ToLower(strings.TrimSpace(os.Getenv("FAV_DEGRADED_READ"))) { + case "1", "true", "yes": + return true + default: + return false + } +} + +func (s *Service) Get(ctx context.Context, p Principal, ref DocRef) (*ContentView, error) { + if ctx == nil { + ctx = context.Background() + } + if err := ctx.Err(); err != nil { + return nil, ErrClosed + } + if p.UserID == 0 { + return nil, ErrClosed + } + if s == nil || s.Gov == nil { + return nil, ErrClosed + } + rec, err := s.Gov.GetDocument(ctx, ref.DocID) + if err != nil || rec == nil { + return nil, ErrClosed + } + if s.Authorize != nil { + if err := s.Authorize(ctx, p, rec); err != nil { + return nil, ErrClosed + } + } + + if s.Rev != nil { + md, sha, herr := s.Rev.HeadContent(ctx, ref.DocID) + if herr == nil && md != "" { + return s.viewFromLedger(ctx, rec, md, sha), nil + } + } + + if s.FavDegradedRead { + return nil, ErrFavNotEnabled + } + return &ContentView{ + DocID: rec.DocID, + Version: rec.Version, + IsTemplateOnly: rec.IsTemplateOnly, + Consistency: ConsistencyLedgerMissing, + RecordSHA256: rec.ContentSHA256, + }, ErrLedgerMissing +} + +func (s *Service) BatchGet(ctx context.Context, p Principal, refs []DocRef) ([]*ContentView, error) { + if len(refs) > MaxBatchGet { + return nil, ErrBatchLimit + } + if p.UserID == 0 { + return nil, ErrClosed + } + var projectID int64 + seen := map[string]bool{} + out := make([]*ContentView, 0, len(refs)) + for _, ref := range refs { + if seen[ref.DocID] { + continue + } + seen[ref.DocID] = true + rec, err := s.Gov.GetDocument(ctx, ref.DocID) + if err != nil || rec == nil { + return nil, ErrClosed + } + if projectID == 0 { + projectID = rec.ProjectID + } else if rec.ProjectID != projectID { + return nil, ErrCrossProject + } + view, err := s.Get(ctx, p, ref) + if err != nil { + return nil, err + } + out = append(out, view) + } + return out, nil +} + +func (s *Service) viewFromLedger(ctx context.Context, rec *docgov.DocumentRecord, md, sha string) *ContentView { + view := &ContentView{ + DocID: rec.DocID, + Version: rec.Version, + ContentSHA256: sha, + Markdown: md, + IsTemplateOnly: rec.IsTemplateOnly, + Consistency: ConsistencyAligned, + RecordSHA256: rec.ContentSHA256, + HeadSHA256: sha, + } + if s.Rev == nil { + return view + } + class, err := s.Rev.InspectProjection(ctx, rec.DocID, s.RepoRoot) + if err != nil { + view.Consistency = ConsistencyMissing + return view + } + view.FileSHA256 = class.RawSHA + switch class.Class { + case "", docrevision.ProjectionAligned, docrevision.ProjectionNoLedger: + view.Consistency = ConsistencyAligned + case docrevision.ProjectionMissing: + view.Consistency = ConsistencyMissing + case docrevision.ProjectionEmpty: + view.Consistency = ConsistencyEmpty + default: + view.Consistency = ConsistencyDrift + } + return view +} + +func (v *ContentView) ToDocumentContent() *docgov.DocumentContent { + if v == nil { + return nil + } + return &docgov.DocumentContent{ + DocID: v.DocID, + Version: v.Version, + ContentSHA256: v.ContentSHA256, + Markdown: v.Markdown, + IsTemplateOnly: v.IsTemplateOnly, + Consistency: v.Consistency, + RecordSHA256: v.RecordSHA256, + FileSHA256: v.FileSHA256, + } +} diff --git a/internal/docquery/service_test.go b/internal/docquery/service_test.go new file mode 100644 index 00000000..e7051270 --- /dev/null +++ b/internal/docquery/service_test.go @@ -0,0 +1,167 @@ +package docquery + +import ( + "context" + "os" + "path/filepath" + "testing" + + "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docrevision" +) + +func setupLedger(t *testing.T, docID, body, workDir string) (*Service, *docgov.GovernanceEngine, *docrevision.Service) { + t.Helper() + ctx := context.Background() + gov := docgov.NewGovernanceEngine(nil) + if _, err := gov.CreateOrGetDocument(ctx, docID, 7, "intake", "t", "V0.5_feasibility", workDir, 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + rev := docrevision.NewService(nil) + if _, err := rev.OpenTemplateVersion(ctx, docID, body, docrevision.ActorSnapshot{}, 7, workDir); err != nil { + t.Fatal(err) + } + svc := &Service{ + Gov: gov, Rev: rev, RepoRoot: "", + Authorize: func(context.Context, Principal, *docgov.DocumentRecord) error { return nil }, + } + return svc, gov, rev +} + +func TestGetServesLedgerWhenProjectionMissing(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + svc, _, _ := setupLedger(t, "doc_q_missing", "# ledger\n", dir) + view, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_missing"}) + if err != nil { + t.Fatal(err) + } + if view.Markdown != "# ledger\n" || view.Consistency != ConsistencyMissing { + t.Fatalf("%+v", view) + } +} + +func TestGetAlignedWhenProjectionMatches(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + body := "# ledger\n" + if err := os.WriteFile(filepath.Join(dir, "output.md"), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + svc, _, _ := setupLedger(t, "doc_q_aligned", body, dir) + view, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_aligned"}) + if err != nil { + t.Fatal(err) + } + if view.Consistency != ConsistencyAligned || view.Markdown != body { + t.Fatalf("%+v", view) + } +} + +func TestGetDriftStillServesLedger(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "output.md"), []byte("# file fork\n"), 0o644); err != nil { + t.Fatal(err) + } + svc, _, _ := setupLedger(t, "doc_q_drift", "# ledger\n", dir) + view, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_drift"}) + if err != nil { + t.Fatal(err) + } + if view.Consistency != ConsistencyDrift || view.Markdown != "# ledger\n" { + t.Fatalf("%+v", view) + } +} + +func TestGetEmptyProjectionStillServesLedger(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "output.md"), []byte{}, 0o644); err != nil { + t.Fatal(err) + } + svc, _, _ := setupLedger(t, "doc_q_empty", "# ledger\n", dir) + view, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_empty"}) + if err != nil { + t.Fatal(err) + } + if view.Consistency != ConsistencyEmpty || view.Markdown != "# ledger\n" { + t.Fatalf("%+v", view) + } +} + +func TestGetLedgerMissing(t *testing.T) { + ctx := context.Background() + gov := docgov.NewGovernanceEngine(nil) + if _, err := gov.CreateOrGetDocument(ctx, "doc_q_noleger", 7, "intake", "t", "V0.5", "", 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + svc := &Service{Gov: gov, Rev: docrevision.NewService(nil), Authorize: func(context.Context, Principal, *docgov.DocumentRecord) error { return nil }} + _, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_noleger"}) + if err != ErrLedgerMissing { + t.Fatalf("got %v", err) + } +} + +func TestGetIsReadOnly(t *testing.T) { + ctx := context.Background() + dir := t.TempDir() + svc, _, _ := setupLedger(t, "doc_q_ro", "# ledger\n", dir) + before, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + if _, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_q_ro"}); err != nil { + t.Fatal(err) + } + after, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + if len(after) != len(before) { + t.Fatalf("GET wrote files: before=%d after=%d", len(before), len(after)) + } +} + +func TestGetFailClosed(t *testing.T) { + ctx := context.Background() + svc, _, _ := setupLedger(t, "doc_q_deny", "# ledger\n", t.TempDir()) + if _, err := svc.Get(ctx, Principal{}, DocRef{DocID: "doc_q_deny"}); err != ErrClosed { + t.Fatalf("anon: %v", err) + } + svc.Authorize = func(context.Context, Principal, *docgov.DocumentRecord) error { return ErrClosed } + if _, err := svc.Get(ctx, Principal{UserID: 99}, DocRef{DocID: "doc_q_deny"}); err != ErrClosed { + t.Fatalf("denied: %v", err) + } + if _, err := svc.Get(ctx, Principal{UserID: 1}, DocRef{DocID: "doc_missing"}); err != ErrClosed { + t.Fatalf("unknown: %v", err) + } +} + +func TestBatchGetLimitsAndCrossProject(t *testing.T) { + ctx := context.Background() + svc, gov, _ := setupLedger(t, "doc_q_b1", "# a\n", t.TempDir()) + if _, err := gov.CreateOrGetDocument(ctx, "doc_q_b2", 8, "intake", "t", "V0.5", "", 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + refs := make([]DocRef, 101) + for i := range refs { + refs[i] = DocRef{DocID: "doc_q_b1"} + } + if _, err := svc.BatchGet(ctx, Principal{UserID: 1}, refs); err != ErrBatchLimit { + t.Fatalf("101: %v", err) + } + if _, err := svc.BatchGet(ctx, Principal{UserID: 1}, []DocRef{{DocID: "doc_q_b1"}, {DocID: "doc_q_b2"}}); err != ErrCrossProject { + t.Fatalf("cross project: %v", err) + } + got, err := svc.BatchGet(ctx, Principal{UserID: 1}, []DocRef{{DocID: "doc_q_b1"}, {DocID: "doc_q_b1"}}) + if err != nil || len(got) != 1 { + t.Fatalf("dedupe: %v n=%d", err, len(got)) + } +} + +func TestFavDegradedReadDefaultOff(t *testing.T) { + if FavDegradedReadEnabled() { + t.Fatal("FAV_DEGRADED_READ must default false") + } +} diff --git a/internal/server/project_doc_content_error_test.go b/internal/server/project_doc_content_error_test.go index 3e4f2554..be7c362e 100644 --- a/internal/server/project_doc_content_error_test.go +++ b/internal/server/project_doc_content_error_test.go @@ -2,7 +2,6 @@ package server import ( "encoding/json" - "errors" "net/http" "net/http/httptest" "testing" @@ -47,20 +46,19 @@ func TestWriteControlledContentError_MapsTypedCodes(t *testing.T) { } } -func TestServeLedgerHeadOrConflict_RefusesDestructiveHead(t *testing.T) { +func TestServeLedgerHeadOrConflict_ServesLedgerOnDrift(t *testing.T) { raw := "条目一 \n条目二 \n" head := docrevision.LegacyDestructiveCanonicalizeMarkdown(raw) class := docrevision.ClassifyProjection([]byte(raw), head) if class.Class != docrevision.ProjectionLegacyCanonicalizationDrift { t.Fatalf("class=%s", class.Class) } - _, err := serveLedgerHeadOrConflict("doc_x", 1, false, head, docrevision.SHA256Hex(head), docrevision.SHA256Hex(head), class) - var ce *docgov.ContentError - if !errors.As(err, &ce) || ce.Code != docgov.ErrCodeProjectionDrift { - t.Fatalf("want projection drift, got %v", err) + gotDrift, err := serveLedgerHeadOrConflict("doc_x", 1, false, head, docrevision.SHA256Hex(head), docrevision.SHA256Hex(head), class) + if err != nil { + t.Fatal(err) } - if ce.Class != docrevision.ProjectionLegacyCanonicalizationDrift { - t.Fatalf("class %s", ce.Class) + if gotDrift.Consistency != "drift" || gotDrift.Markdown != head { + t.Fatalf("drift must still serve ledger: %+v", gotDrift) } aligned := docrevision.ClassifyProjection([]byte(raw), docrevision.CanonicalizeMarkdown(raw)) diff --git a/internal/server/project_doc_handlers.go b/internal/server/project_doc_handlers.go index 30792e32..aa23ab00 100644 --- a/internal/server/project_doc_handlers.go +++ b/internal/server/project_doc_handlers.go @@ -3,16 +3,55 @@ package server import ( "context" "errors" - "fmt" "net/http" "path/filepath" "strconv" "time" + "chat-agent-system/internal/app/docflow" "chat-agent-system/internal/docgov" + "chat-agent-system/internal/docquery" "chat-agent-system/internal/docrevision" ) +func (s *Server) docflowQuery() *docflow.Query { + if s == nil || s.svc == nil { + return nil + } + return &docflow.Query{Docs: &docquery.Service{ + Gov: s.svc.DocGov(), + Rev: s.docRev(), + RepoRoot: s.repoRoot(), + Authorize: func(ctx context.Context, p docquery.Principal, rec *docgov.DocumentRecord) error { + return s.authorizeControlledRead(ctx, p, rec) + }, + FavDegradedRead: false, + }} +} + +func (s *Server) authorizeControlledRead(ctx context.Context, p docquery.Principal, rec *docgov.DocumentRecord) error { + if rec == nil || p.UserID == 0 { + return docquery.ErrClosed + } + if err := ctx.Err(); err != nil { + return docquery.ErrClosed + } + admin := false + if s.authzSvc != nil { + ok, _ := s.authzSvc.IsSystemAdmin(ctx, p.UserID, "") + admin = ok + } + if s.authorizer != nil { + if err := s.authorizer.RequireProjectAccessOrAdmin(ctx, rec.ProjectID, p.UserID, admin); err != nil { + return docquery.ErrClosed + } + } + if rec.TaskKey != "" && !s.allowWorkflowDocTaskAccess(ctx, rec.ProjectID, p.UserID, rec.TaskKey) { + return docquery.ErrClosed + } + return nil +} + func (s *Server) repoRoot() string { if s.cfg != nil && s.cfg.DataDir != "" { // DataDir 形如 ./data/dev；工作区路径形如 data/dev/workspaces/...，相对仓库根目录 @@ -61,96 +100,72 @@ func (s *Server) handleDocGovGetDocumentContent(w http.ResponseWriter, r *http.R return } - if s.svc == nil || s.svc.DocGov() == nil { - http.Error(w, "docgov engine not initialized", http.StatusServiceUnavailable) - return - } - - docRec, err := s.svc.DocGov().GetDocument(r.Context(), docID) - if err != nil || docRec == nil { - http.Error(w, "document not found", http.StatusNotFound) - return - } - user, ok := s.requireProjectAccess(w, r, docRec.ProjectID) + user, ok := s.requireAuthenticatedUser(w, r) if !ok { return } - if !s.allowWorkflowDocTaskAccess(r.Context(), docRec.ProjectID, user.ID, docRec.TaskKey) { - http.Error(w, `{"error":"WORKFLOW_ACCESS_DENIED","message":"无权访问该阶段文档"}`, http.StatusForbidden) + q := s.docflowQuery() + if q == nil || q.Docs == nil || q.Docs.Gov == nil { + http.Error(w, "docgov engine not initialized", http.StatusServiceUnavailable) return } - - content, err := s.loadControlledDocumentContent(r.Context(), docID) + view, err := q.Get(r.Context(), user.ID, docID) if err != nil { + if errors.Is(err, docquery.ErrLedgerMissing) && view != nil { + writeJSONResponse(w, http.StatusOK, view) + return + } + if errors.Is(err, docquery.ErrClosed) { + writeJSONResponse(w, http.StatusNotFound, map[string]any{ + "error_code": docgov.ErrCodeDocumentNotFound, "message": "document not found", + }) + return + } writeControlledContentError(w, r, err) return } - if inm := r.Header.Get("If-None-Match"); inm != "" && inm == content.ContentSHA256 { + if inm := r.Header.Get("If-None-Match"); inm != "" && inm == view.ContentSHA256 { w.WriteHeader(http.StatusNotModified) return } - w.Header().Set("ETag", content.ContentSHA256) - writeJSONResponse(w, http.StatusOK, content) + w.Header().Set("ETag", view.ContentSHA256) + writeJSONResponse(w, http.StatusOK, view) } func (s *Server) loadControlledDocumentContent(ctx context.Context, docID string) (*docgov.DocumentContent, error) { - if rev := s.docRev(); rev != nil { - if md, sha, err := rev.HeadContent(ctx, docID); err == nil && md != "" { - class, ierr := rev.InspectProjection(ctx, docID, s.repoRoot()) - if ierr != nil { - return nil, ierr - } - doc, _ := s.svc.DocGov().GetDocument(ctx, docID) - ver := 0 - tmpl := false - recordSHA := sha - if doc != nil { - ver = doc.Version - tmpl = doc.IsTemplateOnly - if doc.ContentSHA256 != "" { - recordSHA = doc.ContentSHA256 - } - } - return serveLedgerHeadOrConflict(docID, ver, tmpl, md, sha, recordSHA, class) - } + if s == nil || s.svc == nil || s.svc.DocGov() == nil { + return nil, docgov.ErrDocumentNotFound + } + svc := &docquery.Service{Gov: s.svc.DocGov(), Rev: s.docRev(), RepoRoot: s.repoRoot()} + view, err := svc.Get(ctx, docquery.Principal{UserID: 1}, docquery.DocRef{DocID: docID}) + if err != nil && !errors.Is(err, docquery.ErrLedgerMissing) { + return nil, err + } + if view != nil && view.Markdown != "" { + return view.ToDocumentContent(), nil } return s.svc.DocGov().ReadDocumentContent(ctx, docID, s.repoRoot()) } -// serveLedgerHeadOrConflict returns ledger head only when the projection is aligned. -// Legacy destructive drift or a real fork must not silently display the old stripped head. +// serveLedgerHeadOrConflict keeps the historical helper name. Projection is a +// consistency cache; ledger head is served for missing/empty/drift. func serveLedgerHeadOrConflict(docID string, version int, tmpl bool, headMD, headSHA, recordSHA string, class docrevision.ProjectionClass) (*docgov.DocumentContent, error) { + consistency := docquery.ConsistencyAligned switch class.Class { case "", docrevision.ProjectionAligned, docrevision.ProjectionNoLedger: - return &docgov.DocumentContent{ - DocID: docID, Version: version, ContentSHA256: headSHA, Markdown: headMD, IsTemplateOnly: tmpl, - }, nil + consistency = docquery.ConsistencyAligned case docrevision.ProjectionMissing: - ce := docgov.NewContentError(docgov.ErrCodeProjectionMissing, docgov.ErrContentProjectionMissing, true, true) - ce.RecordSHA = recordSHA - ce.HeadSHA = headSHA - ce.Class = class.Class - return nil, ce + consistency = docquery.ConsistencyMissing case docrevision.ProjectionEmpty: - ce := docgov.NewContentError(docgov.ErrCodeProjectionEmpty, docgov.ErrContentProjectionEmpty, false, true) - ce.RecordSHA = recordSHA - ce.HeadSHA = headSHA - ce.Class = class.Class - return nil, ce + consistency = docquery.ConsistencyEmpty default: - ce := docgov.NewContentError( - docgov.ErrCodeProjectionDrift, - fmt.Errorf("document content projection drift %s; repair required before serving ledger head", class.Class), - false, - true, - ) - ce.RecordSHA = recordSHA - ce.FileSHA = class.RawSHA - ce.HeadSHA = headSHA - ce.Class = class.Class - return nil, ce + consistency = docquery.ConsistencyDrift } + return &docgov.DocumentContent{ + DocID: docID, Version: version, ContentSHA256: headSHA, Markdown: headMD, IsTemplateOnly: tmpl, + Consistency: consistency, RecordSHA256: recordSHA, FileSHA256: class.RawSHA, + }, nil } func writeControlledContentError(w http.ResponseWriter, r *http.Request, err error) { @@ -165,15 +180,15 @@ func writeControlledContentError(w http.ResponseWriter, r *http.Request, err err status = http.StatusNotFound } writeJSONResponse(w, status, map[string]any{ - "error_code": ce.Code, - "message": ce.Error(), - "retryable": ce.Retryable, - "needs_admin": ce.NeedsAdmin, - "correlation_id": corr, - "record_sha256": ce.RecordSHA, - "file_sha256": ce.FileSHA, - "head_sha256": ce.HeadSHA, - "projection_class": ce.Class, + "error_code": ce.Code, + "message": ce.Error(), + "retryable": ce.Retryable, + "needs_admin": ce.NeedsAdmin, + "correlation_id": corr, + "record_sha256": ce.RecordSHA, + "file_sha256": ce.FileSHA, + "head_sha256": ce.HeadSHA, + "projection_class": ce.Class, }) return } diff --git a/web/src/api/projectDocuments.ts b/web/src/api/projectDocuments.ts index e5538b35..8b25a32c 100644 --- a/web/src/api/projectDocuments.ts +++ b/web/src/api/projectDocuments.ts @@ -25,12 +25,22 @@ export type WorkflowDocumentSummary = { approver_name?: string; }; +export type ControlledDocConsistency = + | 'aligned' + | 'missing' + | 'drift' + | 'empty' + | 'ledger_missing'; + export type ControlledDocumentContent = { doc_id: string; version: number; content_sha256: string; markdown: string; is_template_only: boolean; + consistency?: ControlledDocConsistency; + record_sha256?: string; + file_sha256?: string; }; export async function listProjectWorkflowDocuments( diff --git a/web/src/lib/controlledDocHydrate.test.ts b/web/src/lib/controlledDocHydrate.test.ts index 7cfd3bb2..d64102b2 100644 --- a/web/src/lib/controlledDocHydrate.test.ts +++ b/web/src/lib/controlledDocHydrate.test.ts @@ -172,6 +172,22 @@ describe('controlledDocHydrate', () => { ); }); + it('hydrate treats projection missing as ledger content, not 404', async () => { + const ast = await hydrateControlledDocumentContent('doc_proj56_intake', base, { + fetchContent: async () => ({ + doc_id: 'doc_proj56_intake', + version: 1, + content_sha256: 'abc', + markdown: '# 账本正文\n', + is_template_only: false, + consistency: 'missing', + }), + fetchRevisionState: async () => ({ etag: '1:0' }), + }); + assert.ok(ast); + assert.equal(ast?.revisionEtag, '1:0'); + }); + it('attaches hydrate ETag as PATCH baseline and ignores a later GET', async () => { const ast = await hydrateControlledDocumentContent('doc_proj56_intake', base, { fetchContent: async () => ({ diff --git a/web/src/lib/controlledDocHydrate.ts b/web/src/lib/controlledDocHydrate.ts index 857647d7..cdfe8bdf 100644 --- a/web/src/lib/controlledDocHydrate.ts +++ b/web/src/lib/controlledDocHydrate.ts @@ -150,7 +150,9 @@ export async function hydrateControlledDocumentContent( (err: unknown) => ({ ok: false as const, err }), ), ]); - if (!detail?.markdown) return null; + // Typed consistency (missing/drift/empty) is not a 404. Only the absence + // of ledger markdown fails hydrate; never refetch to "repair" a missing projection. + if (detail.consistency === 'ledger_missing' || !detail.markdown) return null; if (!rev.ok) { throw rev.err instanceof Error ? rev.err : new Error('revision-state hydrate failed'); } From f0f57a075530c7376b2578a70ed57cfb0b0520ce Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:11:26 +0000 Subject: [PATCH 08/11] fix(docflow): avoid mutating shared Commands.Auth Concurrent HumanEdit/Approve used to assign the default authorizer on the shared Commands value, which raced under -race. Read a local authorizer instead. Co-authored-by: Cursor --- internal/app/docflow/commands.go | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/internal/app/docflow/commands.go b/internal/app/docflow/commands.go index f6500e22..c41b5cd8 100644 --- a/internal/app/docflow/commands.go +++ b/internal/app/docflow/commands.go @@ -80,9 +80,7 @@ func (c *Commands) run(ctx context.Context, docID string, p Principal, action Ac if c == nil { return fmt.Errorf("docflow: commands required") } - if c.Auth == nil { - c.Auth = DutyAuthorizer() - } + auth := c.authorizer() if c.UoW == nil { if err := c.authorizeUnlocked(ctx, docID, p, action); err != nil { return err @@ -97,7 +95,7 @@ func (c *Commands) run(ctx context.Context, docID string, p Principal, action Ac if action == ActionApprove || action == ActionReject { return err } - } else if err := c.Auth.Authorize(ctx, p, rec, action); err != nil { + } else if err := auth.Authorize(ctx, p, rec, action); err != nil { return err } } @@ -105,6 +103,13 @@ func (c *Commands) run(ctx context.Context, docID string, p Principal, action Ac }) } +func (c *Commands) authorizer() Authorizer { + if c != nil && c.Auth != nil { + return c.Auth + } + return DutyAuthorizer() +} + func (c *Commands) authorizeUnlocked(ctx context.Context, docID string, p Principal, action Action) error { if docID == "" || c.Gov == nil { return nil @@ -119,7 +124,7 @@ func (c *Commands) authorizeUnlocked(ctx context.Context, docID string, p Princi } return nil } - return c.Auth.Authorize(ctx, p, rec, action) + return c.authorizer().Authorize(ctx, p, rec, action) } func (c *Commands) HumanEdit(ctx context.Context, p Principal, cmd HumanEditCommand) (*docrevision.HumanRevisionResult, error) { From 9d6594a31c5b5256a2b08d0246168df8ccb8e5aa Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:11:26 +0000 Subject: [PATCH 09/11] docs(test): record document reliability M1 stop-the-line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Unit, isolated PostgreSQL, and target-package race passed, but real PI, browser, and rebuild were not signed, so the report concludes 停止扩面 and keeps U5–U12 closed. Co-authored-by: Cursor --- .doc/ops/CURRENT.md | 6 +- ...4-module-decoupling-refactor-acceptance.md | 30 ++-- .../2026-09-14-document-reliability-pilot.md | 27 ++-- .doc/remediation/gap-matrix.md | 2 +- .doc/status/FEATURE-MATRIX.md | 2 +- ...-document-reliability-pilot-test-report.md | 132 ++++++++++++++++++ 6 files changed, 167 insertions(+), 32 deletions(-) create mode 100644 .doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index 836387b5..2f603379 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -5,7 +5,7 @@ ## 焦点 -1. **DOC-RELIABILITY**: [文档可靠性纵向试点 M0–M1](../ops/short-term/2026-09-14-document-reliability-pilot.md)（`refactor/document-reliability-pilot`；`mutex=docgov`；不占集成槽、不 rebuild） +1. **DOC-RELIABILITY**: [文档可靠性纵向试点 M0–M1](../ops/short-term/2026-09-14-document-reliability-pilot.md)（`refactor/document-reliability-pilot`；`mutex=docgov`；M1 报告 **停止扩面**；不占槽、不 rebuild、不开 U5–U12） 2. **CHAT-INTENT**: [对话意图运行时状态](../audit/2026-09-13-conversation-intent-runtime-state.md)（`fix/conversation-intent-state`；与文档账本运行时无关） 3. **CHAT**: [对话历史刷新](../audit/2026-09-13-chat-history-refresh-disappears.md)（认领仍 `in_progress`；gap 行已 PASS，以代码/handoff 为准收敛） @@ -13,7 +13,7 @@ | owner | task | branch | worktree | mutex | status | |-------|------|--------|----------|-------|--------| -| Cursor | document-reliability-pilot | refactor/document-reliability-pilot | .worktrees/refactor-document-reliability-pilot | docgov | in_progress | +| Cursor | document-reliability-pilot | refactor/document-reliability-pilot | .worktrees/refactor-document-reliability-pilot | docgov | blocked | | Codex | module-decoupling-remediation-plan | feature/module-decoupling-remediation | .worktrees/feature/module-decoupling-remediation | architecture-docs | done | | verifier_antigravity | controlled-content-ui-acceptance | fix/controlled-content-ui-acceptance | .worktrees/fix-controlled-content-ui-acceptance | document-content-ui-acceptance | done | | implementer_cursor | controlled-content-canonical-hash | fix/controlled-content-canonical-hash | .worktrees/fix-controlled-content-canonical-hash | docgov | done | @@ -54,7 +54,7 @@ > 新任务开始前登记。`branch` 必须为 `feature/` 或 `fix/`，**禁止** `dev` / `test` / `main`。同一 `mutex` 同时仅一个 `in_progress`。 > workflow-plan 409 与治理鉴权 G1–G5 已收口（见 [test-report](../test-reports/2026-09-14-governance-action-level-authz-test-report.md)）。禁止终止或改写项目 143。 -> `document-reliability-pilot` 仅占 `docgov` 编码；合 `dev` + `env-service rebuild` 必须另占集成槽位。M1 报告未签字前不得 rebuild。 +> `document-reliability-pilot` 仅占 `docgov` 编码；合 `dev` + `env-service rebuild` 必须另占集成槽位。M1 结论为 **停止扩面**（见 [test-report](../test-reports/2026-09-14-document-reliability-pilot-test-report.md)），不得 rebuild，不得开工 U5–U12。 ## 集成槽位 diff --git a/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md b/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md index 38f408e0..a9b9b044 100644 --- a/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md +++ b/.doc/ops/checklists/2026-09-14-module-decoupling-refactor-acceptance.md @@ -1,28 +1,28 @@ --- title: "模块解耦重构验收清单" -status: planned +status: in_progress created: 2026-09-14 updated: 2026-09-14 -owner: TBD +owner: Cursor --- # 模块解耦重构验收清单 ## M0 架构与基线 -- [ ] contracts 分类、事务 owner、表属主、锁顺序、事件信封、Principal 和 FAV 边界均有唯一规范来源。 -- [ ] arch-check 分离 production import、SQL write、path 和 event registration；例外有 owner 与到期日。 -- [ ] 当前违规基线可重复生成，新违规立即阻断。 +- [x] contracts 分类、事务 owner、表属主、锁顺序、事件信封、Principal 和 FAV 边界均有唯一规范来源。 +- [x] arch-check 分离 production import、SQL write、path 和 event registration；例外有 owner 与到期日。 +- [x] 当前违规基线可重复生成，新违规立即阻断。 ## M1 文档可靠性 -- [ ] 人工编辑、agent commit、approve/reject 的跨表写入同事务。 -- [ ] fault injection 任一点失败均无部分提交。 -- [ ] 同 ETag 并发写只有一个赢家；不同文档不被全局锁串行。 +- [x] 人工编辑、agent commit、approve/reject 的跨表写入同事务（U3；隔离库双会话编辑已测）。 +- [x] fault injection 任一点失败均无部分提交（U2 隔离库）。 +- [x] 同 ETag 并发写只有一个赢家（内存 + 双会话）。 - [ ] 1、2、4 app replicas 结果一致。 -- [ ] GET/refresh/reload 零写入，projection 缺失不伪装成 404。 +- [x] GET 服务端零写入 + typed consistency（U4 单测）。refresh/reload 真页面未测。 - [ ] 项目 143 等价 hazard 与新项目真实页面刷新、重登、restart、rebuild PASS。 -- [ ] 匿名、非成员、跨用户、跨项目、错误 stage/duty、授权失败负例 PASS。 +- [x] 服务端负例 fail closed；浏览器跨用户未测。 ## M2 事件与 FAV @@ -57,14 +57,14 @@ owner: TBD ## 证据分层 -- [ ] Unit/stub 证据。 -- [ ] PostgreSQL integration 证据。 -- [ ] race/concurrency/load 证据。 -- [ ] fault injection/crash 证据。 +- [x] Unit/stub 证据。 +- [x] PostgreSQL integration 证据（隔离库，非共享 env 库）。 +- [x] race/concurrency 目标包证据（非全仓 race）。 +- [x] fault injection 证据（U2）。 - [ ] 真实 PI 证据。 - [ ] 真实浏览器证据。 - [ ] restart/rebuild 证据。 -- [ ] cross-user/project authorization negative 证据。 +- [x] cross-user/project authorization negative 服务端证据。 - [ ] scratch recovery 与完整 DB restore 证据。 ## 关闭条件 diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md index 0e4b66e8..669eff20 100644 --- a/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md +++ b/.doc/ops/short-term/2026-09-14-document-reliability-pilot.md @@ -1,6 +1,6 @@ --- title: "文档可靠性纵向试点 M0–M1" -status: in_progress +status: blocked created: 2026-09-14 updated: 2026-09-14 owner: Cursor @@ -37,25 +37,28 @@ mutex: docgov 3. [x] 实现 U2，验证任意属主写入/commit 故障整体回滚。 4. [x] 实现 U3，删除提交后 outbox 正确性回调。 5. [x] 实现 U4，保证 GET 零写入和 projection 缺失不再伪装成 404。 -6. [ ] 执行多实例并发、PG、真实 PI、真实浏览器和 restart/rebuild 验收。 -7. [ ] 写 test report，更新验收清单、FEATURE-MATRIX、CURRENT 和 gap-matrix。 +6. [x] 隔离库 PG + 目标包 race 已跑；真实 PI / 浏览器 / rebuild **未**签字（未占槽）。 +7. [x] 写 test report：结论 **停止扩面**。更新验收清单、FEATURE-MATRIX、CURRENT 和 gap-matrix。 ## 验收标准 -- [ ] 同一 ETag 并发编辑只有一个合法赢家，账本/head/outbox 无孤儿。 -- [ ] 任一事务故障点不会留下部分提交。 -- [ ] 项目 143 等价 hazard 和新项目刷新/重启目标 404/409/projection missing 为 0。 -- [ ] 匿名、非成员、跨用户、跨项目、错误 stage/duty 和授权依赖失败均 fail closed。 -- [ ] 1、2、4 个 app 实例的最终状态一致。 -- [ ] M1 test report 给出继续 M2 或停止扩面的明确结论。 +- [x] 同一 ETag 并发编辑只有一个合法赢家（内存 8 协程 + 隔离库双会话）。 +- [x] 任一事务故障点不会留下部分提交（U2 隔离库）。 +- [ ] 项目 143 等价 hazard 和新项目刷新/重启目标 404/409/projection missing 为 0（真实页面未测）。 +- [x] 服务端匿名/未知文档/跨项目/错误责任人 fail closed；浏览器跨登录未测。 +- [ ] 1、2、4 个 app 实例的最终状态一致（仅到双会话）。 +- [x] M1 test report 结论：**停止扩面**。 ## 相关链接 - Checklist: [../checklists/2026-09-14-module-decoupling-refactor-acceptance.md](../checklists/2026-09-14-module-decoupling-refactor-acceptance.md) -- Handoff: 实施时创建 -- Test report: 实施后创建 `.doc/test-reports/-document-reliability-pilot.md` +- Test report: [../../test-reports/2026-09-14-document-reliability-pilot-test-report.md](../../test-reports/2026-09-14-document-reliability-pilot-test-report.md) - Plan: [../../remediation/2026-09-14-module-decoupling-refactor-plan.md](../../remediation/2026-09-14-module-decoupling-refactor-plan.md) +## M2+ 门（不开工） + +U5 WorkspaceRef、U6 outbox、U7 按文件摘 FAV 需要独立审批，且仅当 M1 报告最后一句为「继续 M2」。当前是「停止扩面」，禁止实现 U5–U12。 + ## 证据 / 备注 -只有本任务真实验收通过后，才允许启动 durable event/FAV 与横向模块扩展。 +隔离库与目标包 race 见 test-report assets。未占槽、未 rebuild。只有真实 PI/浏览器/rebuild 补测并把报告改为「继续 M2」后，才允许启动 durable event/FAV。 diff --git a/.doc/remediation/gap-matrix.md b/.doc/remediation/gap-matrix.md index f9f37351..e4d19fd7 100644 --- a/.doc/remediation/gap-matrix.md +++ b/.doc/remediation/gap-matrix.md @@ -70,7 +70,7 @@ | DOCREV-ACTIVE-DOC-ROUTING | UI-05「当前受控文档」被法规关键词改路由到 `feasibility_reg`，intake 无 pending | **PASS** | [audit](../audit/2026-09-13-document-revision-active-doc-routing.md)；Round20 UI-05 pending 停在 intake V2 | **不改弱 E2E**；不删证据 | | DOCREV-QUALITY-STATUS-FP | 行动计划表状态列「待续」被门禁当敷衍，两轮 repair 后 quality_failed 80 | **PASS** | [audit](../audit/2026-09-13-document-revision-quality-status-false-positive.md)；Round20 UI-03 | **不删禁词、不放宽正文**；不删证据 | | GOVERNANCE-ACTION-AUTHZ | 文档动作按职责鉴权；未激活 202-blocked；关闭创建直通 | **PASS** | [test-report](../test-reports/2026-09-14-governance-action-level-authz-test-report.md)；[audit 异态](../audit/2026-09-14-planning-with-workflow-key-anomaly.md) | 不改 83 条 planning+key 夹具；none 长 PI 审批链另开 | -| DOC-RELIABILITY-PILOT | 受控文档写路径跨表非同事务、读路径被 projection 绑架、授权 TOCTOU；刷新/重启后可能 404/409/projection-missing | OPEN | [复核](../audit/2026-09-14-module-decoupling-plan-review.md)；[重构计划](./2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md) | M0–M1=U1–U4；未占槽不得 rebuild；M2+ 需独立 go/no-go | +| DOC-RELIABILITY-PILOT | 受控文档写路径跨表非同事务、读路径被 projection 绑架、授权 TOCTOU；刷新/重启后可能 404/409/projection-missing | OPEN | [复核](../audit/2026-09-14-module-decoupling-plan-review.md)；[重构计划](./2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md)；[M1 报告](../test-reports/2026-09-14-document-reliability-pilot-test-report.md) | U1–U4 已编码。M1 **停止扩面**（无真实 PI/浏览器/rebuild）。未占槽不得 rebuild；禁止 U5–U12 | --- diff --git a/.doc/status/FEATURE-MATRIX.md b/.doc/status/FEATURE-MATRIX.md index abec2c5f..7b018a68 100644 --- a/.doc/status/FEATURE-MATRIX.md +++ b/.doc/status/FEATURE-MATRIX.md @@ -27,7 +27,7 @@ | PI Runner / 配置链 | pi-agent + MiniMax 真实链路 | done | [test piagent](../test-reports/2026-09-13-codex-audit-p0-remediation-test-report.md)；[container-env](../test-reports/2026-09-13-container-env-configuration-test-report.md) | — | `pi-agent` | | 三环境运维 | dev/test/main + 共享 KB | done | [PROJECT.md](../../PROJECT.md) §4；四环境 healthy | — | `infra` | | CI / E2E | PG 集成 + 真实 PI E2E | partial | [gap-matrix BUG-08](../remediation/gap-matrix.md)；[test pg-ci](../test-reports/2026-09-12-pg-ci-soak-e2e-test-report.md)；[运行时 UI Round20](../test-reports/2026-09-14-document-revision-runtime-ui-test-report.md) | CI `TEST_DATABASE_URL` 隔离库仍缺；文档运行时真实 UI 已按 UI_PHASE 分文件且 post-restart 断言最终 A1 正文。`-race` 全包并行 TempDir flake 为基建残留 | `ci` | -| 模块解耦与可靠事件 | 文档可靠性纵向试点、跨属主事务、多副本互斥、FAV 与选择性模块化 | planned | [架构](../system-design/2026-09-14-module-decoupling-architecture.md)；[ADR](../adr/2026-09-14-modular-monolith-decoupling.md)；[重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md) | 方案已入库。M0–M1=U1–U4 为首个批准范围；M2+ 需 M1 go/no-go | `docgov` | +| 模块解耦与可靠事件 | 文档可靠性纵向试点、跨属主事务、多副本互斥、FAV 与选择性模块化 | partial | [架构](../system-design/2026-09-14-module-decoupling-architecture.md)；[ADR](../adr/2026-09-14-modular-monolith-decoupling.md)；[重构计划](../remediation/2026-09-14-module-decoupling-refactor-plan.md)；[试点任务](../ops/short-term/2026-09-14-document-reliability-pilot.md)；[M1 报告](../test-reports/2026-09-14-document-reliability-pilot-test-report.md) | U1–U4 已在 `refactor/document-reliability-pilot` 落地（Unit/隔离库 PASS）。M1 **停止扩面**：真实 PI/浏览器/rebuild 未签。禁止 U5–U12 | `docgov` | --- diff --git a/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md b/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md new file mode 100644 index 00000000..c415715a --- /dev/null +++ b/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md @@ -0,0 +1,132 @@ +# 文档可靠性纵向试点（M0–M1 / U1–U4）测试报告 + +## 一、 测试背景 (Background) + +受控文档写路径曾把账本、root pointer 与通知拆到提交后回调；读路径在 ledger 已存在时仍被 `InspectProjection` 绑架成 409。刷新、重登或 projection 缺失会被前端当成 404 再 hydrate。本轮只批准 M0–M1（U1–U4）：门禁纠偏、UnitOfWork、文档命令同事务、授权后账本读。明确不授权 FAV 降级读、拆 chat/squad、占槽 rebuild。 + +## 二、 测试目标 (Test Objectives) + +- U1：四类生产规则可重放；新违规 block；Goose/`*_test.go` 不记生产写者债。 +- U2：跨属主同事务；第二属主/staging/commit 失败零残留；仅 `40001`/`40P01` 整事务重试。 +- U3：人工编辑 / agent commit / approve / reject 锁后重鉴权；同 ETag 一赢一输。 +- U4：projection missing/drift/empty 仍返回账本正文 + typed `consistency`；GET 零写入。 +- 分栏必须各自签字。未占集成槽不得把 `:8080` 旧镜像或 `docker compose restart` 写成 rebuild 证据。 + +## 三、 测试方案与环境 (Test Plan & Environment) + +| 项 | 值 | +|----|-----| +| OS | Linux 7.0.0-31-generic | +| Go | 1.27.1 | +| 分支 / worktree | `refactor/document-reliability-pilot` / `.worktrees/refactor-document-reliability-pilot` | +| 实现基线 | `dev@a8f9451` → `c754f77`（另含本报告前的 `fix(docflow)` 竞态修复） | +| Unit / stub | `go test -short ./...`（**未**设置 `TEST_DATABASE_URL`） | +| PostgreSQL | 隔离库 `agent_rel_itest`（`shuimu-system-pg-test:5434`），测完 `DROP DATABASE`；禁止 `agent_system_dev/test/main` | +| 目标包 race | `./internal/app/uow ./internal/app/docflow ./internal/docquery ./internal/archtest` | +| 前端 | `npx tsx --test src/lib/controlledDocHydrate.test.ts`（17 passed） | +| 真实 PI / 浏览器 / rebuild | **未执行**。集成槽 idle；CURRENT 禁止 M1 未签字 rebuild；`:8080` 仍是治理鉴权旧镜像，不能当本分支证据 | +| 证据 | `.doc/test-reports/assets/2026-09-14-document-reliability-pilot/` | + +## 四、 预期结果 (Expected Results) + +| 栏 | 期望 | +|----|------| +| Unit / stub | 全绿；arch-check `--mode warn` 过门；characterization 同 ETag 一赢、approve/reject 唯一终态 | +| PG 事务与并发 | U2 故障注入绿；同 ETag 双会话 HumanEdit 仅一赢；head revision=1 | +| 故障注入 | 第二属主失败 / staging 失败 / commit 失败零行；`40001` 整事务重跑一条结果 | +| 真实 PI | 本分支容器内真实 `pi-agent` 全链路。**本轮不做** | +| 真实浏览器 | 项目 143 等价 hazard + 新建项目打开/刷新/重登正文 100%，404/409/projection-missing=0。**本轮不做** | +| restart/rebuild | 占槽后 `env-service.sh dev rebuild`。**本轮不做** | +| 跨用户负例 | 匿名/未知文档/授权拒绝 → `docquery.ErrClosed`，不泄露存在性 | + +## 五、 实际测试结果与判定 (Actual Results & Evaluation) + +### 5.1 Unit / stub + +- **测试输入**: `go test -short ./...` +- **实际输出**: 38 个有测试包 `ok`，无 `FAIL`。摘录： + +``` +ok chat-agent-system/internal/app/docflow +ok chat-agent-system/internal/app/uow +ok chat-agent-system/internal/archtest +ok chat-agent-system/internal/docquery +ok chat-agent-system/internal/docrevision +ok chat-agent-system/internal/server 9.796s +``` + +- **评价判定**: **PASS** +- **性能与耗时**: `internal/server` 约 9.8s，其余亚秒到数秒。日志：`assets/2026-09-14-document-reliability-pilot/go-test-short.log` + +`./scripts/arch-check.sh --mode warn` → `arch-check warn: production gate passed`。`--mode block` 仍会因既有例外基线失败（U1 设计如此）。 + +### 5.2 PostgreSQL 事务与并发 + +- **测试输入**: 隔离库 `TEST_DATABASE_URL` → `./internal/app/uow`、`./internal/app/docflow`、`./internal/docquery`；`./internal/docrevision -run TestDocumentCommitConcurrencyPG` +- **实际输出**: + +``` +ok chat-agent-system/internal/app/uow 0.065s +ok chat-agent-system/internal/app/docflow 0.003s +ok chat-agent-system/internal/docquery 0.003s +ok chat-agent-system/internal/docrevision 0.024s +``` + +- **评价判定**: **PASS**（1–2 会话级）。**未**跑 4 个独立 app 进程。 +- **性能与耗时**: UoW 包 65ms；并发编辑 24ms。日志：`go-test-pg-race.log` + +### 5.3 故障注入 + +- **测试输入**: `internal/app/uow` PG 用例（第二属主失败、staging/outbox 失败、commit 失败、`40001` 重试、version/permission/digest 不重试） +- **实际输出**: `ok chat-agent-system/internal/app/uow 0.065s` +- **评价判定**: **PASS** +- **性能与耗时**: 与 5.2 同一次隔离库运行 + +### 5.4 真实 PI + +- **测试输入**: 未切本分支容器，未跑真实 `MiniMax-M2.7-highspeed` 编制链路 +- **实际输出**: 无 +- **评价判定**: **FAIL（未测）** — 禁止用 stub 冒充终态 +- **性能与耗时**: n/a + +### 5.5 真实浏览器 + +- **测试输入**: 未对项目 143 / 新建项目做本分支页面打开、刷新、重登 +- **实际输出**: 仅前端单测 17/17（`hydrate treats projection missing as ledger content, not 404`） +- **评价判定**: 单测 **PASS**；真实页面 **FAIL（未测）** +- **性能与耗时**: hydrate 套件 149ms + +### 5.6 restart / rebuild + +- **测试输入**: 未占集成槽，未执行 `./scripts/env-service.sh dev rebuild` +- **实际输出**: 无。`:8080` 仍为既有 dev 镜像（治理鉴权 G5），不得当作本分支部署证据 +- **评价判定**: **FAIL（未测）** +- **性能与耗时**: n/a + +### 5.7 跨用户负例 + +- **测试输入**: `TestGetFailClosed`（匿名、Authorize 拒绝、未知 doc_id）；`TestReauthDeniesWrongAuthor`；`TestBatchGetLimitsAndCrossProject`（101 条 / 跨项目） +- **实际输出**: 均失败关闭；未知文档与拒绝同一 `ErrClosed` +- **评价判定**: 服务端单测 **PASS**。HTTP 跨登录浏览器负例 **未测** +- **性能与耗时**: 亚秒 + +## 六、 自动化测试套件汇总 (Test Suite Execution) + +| 命令 | 结果 | +|------|------| +| `go vet ./...` | PASS（U4 提交前） | +| `go test -short ./...` | PASS，38 ok | +| 隔离库 `go test ./internal/app/uow ./internal/app/docflow ./internal/docquery` | PASS | +| `go test ./internal/docrevision -run TestDocumentCommitConcurrencyPG` | PASS | +| `go test -race` 目标包 | PASS（修复 `Commands.Auth` 并发写入后） | +| `arch-check --mode warn` | PASS | +| `tsx --test controlledDocHydrate.test.ts` | 17 passed | +| 全仓 `go test`（带 `TEST_DATABASE_URL` 且无 `-short`） | 未作为本报告门禁；会把无关 workflow PG 和「5434=共享 test 口」拒绝用例搅在一起 | + +`-race` 初跑曾在 `run()` 里给共享 `Commands.Auth` 赋值，判定为数据竞争；已改为只读 `authorizer()`。 + +## 七、 测试结论与后续优化建议 (Conclusion & Next Steps) + +U1–U4 代码已在实现分支落地，Unit/stub、隔离库事务/故障注入、目标包 race 与前端 hydrate 单测通过。用户可感知的主修复（ledger-first GET）**没有**真实 PI、真实浏览器、restart/rebuild 签字；1/2/4 进程一致性只验证到双会话。按计划停走门，不得用「部分通过」打开 U5–U12。 + +停止扩面 From 377219374723bce288259fe20f332416b02adfcd Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:11:52 +0000 Subject: [PATCH 10/11] docs(test): attach M1 suite transcripts Keep the short and isolated PostgreSQL/race outputs next to the stop-the-line report so the unsigned PI/browser/rebuild columns stay auditable. Co-authored-by: Cursor --- .doc/ops/CURRENT.md | 1 + .../go-test-pg-race.txt | 11 +++++ .../go-test-short.txt | 44 +++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100644 .doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-pg-race.txt create mode 100644 .doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-short.txt diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index 2f603379..885480a0 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -90,6 +90,7 @@ | 纯软件项目自举设计 | [../system-design/2026-09-14-software-project-bootstrap-workflow.md](../system-design/2026-09-14-software-project-bootstrap-workflow.md) | | 模块化单体解耦架构 | [../system-design/2026-09-14-module-decoupling-architecture.md](../system-design/2026-09-14-module-decoupling-architecture.md) | | 模块解耦整改计划 | [../remediation/2026-09-14-module-decoupling-refactor-plan.md](../remediation/2026-09-14-module-decoupling-refactor-plan.md) | +| 文档可靠性 M1 报告 | [../test-reports/2026-09-14-document-reliability-pilot-test-report.md](../test-reports/2026-09-14-document-reliability-pilot-test-report.md) | | SSE/幂等审计 | [../audit/2026-09-13-document-revision-sse-idempotency.md](../audit/2026-09-13-document-revision-sse-idempotency.md) | | 质量门禁自污染审计 | [../audit/2026-09-13-document-revision-quality-gate-self-pollution.md](../audit/2026-09-13-document-revision-quality-gate-self-pollution.md) | | 当前文档路由审计 | [../audit/2026-09-13-document-revision-active-doc-routing.md](../audit/2026-09-13-document-revision-active-doc-routing.md) | diff --git a/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-pg-race.txt b/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-pg-race.txt new file mode 100644 index 00000000..76ecd462 --- /dev/null +++ b/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-pg-race.txt @@ -0,0 +1,11 @@ +===== uow/docflow/docquery PG+unit ===== +ok chat-agent-system/internal/app/uow 0.065s +ok chat-agent-system/internal/app/docflow 0.003s +ok chat-agent-system/internal/docquery 0.003s +===== concurrency PG ===== +ok chat-agent-system/internal/docrevision 0.024s +===== race target ===== +ok chat-agent-system/internal/app/uow 1.122s +ok chat-agent-system/internal/app/docflow 1.010s +ok chat-agent-system/internal/docquery 1.009s +ok chat-agent-system/internal/archtest 1.742s diff --git a/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-short.txt b/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-short.txt new file mode 100644 index 00000000..d52d8c8b --- /dev/null +++ b/.doc/test-reports/assets/2026-09-14-document-reliability-pilot/go-test-short.txt @@ -0,0 +1,44 @@ +ok chat-agent-system/cmd/bizdev-cli (cached) +ok chat-agent-system/cmd/document-content-reconcile (cached) +? chat-agent-system/cmd/document-revision-backfill [no test files] +? chat-agent-system/cmd/recategorize [no test files] +? chat-agent-system/cmd/server [no test files] +ok chat-agent-system/internal/agent (cached) +ok chat-agent-system/internal/app/docflow 0.003s +ok chat-agent-system/internal/app/uow (cached) +ok chat-agent-system/internal/archtest 0.075s +ok chat-agent-system/internal/auth 0.832s +ok chat-agent-system/internal/authz (cached) +ok chat-agent-system/internal/bizdev (cached) +ok chat-agent-system/internal/chat (cached) +ok chat-agent-system/internal/config (cached) +ok chat-agent-system/internal/coordinator (cached) +ok chat-agent-system/internal/ctxmgr (cached) +ok chat-agent-system/internal/db (cached) +ok chat-agent-system/internal/docduty (cached) +ok chat-agent-system/internal/docgov (cached) +ok chat-agent-system/internal/docquery (cached) +ok chat-agent-system/internal/docrevision 0.005s +ok chat-agent-system/internal/freedoc 0.021s +ok chat-agent-system/internal/fsutil (cached) +ok chat-agent-system/internal/intent (cached) +ok chat-agent-system/internal/kb (cached) +ok chat-agent-system/internal/notification 0.052s +? chat-agent-system/internal/observability [no test files] +ok chat-agent-system/internal/projectcontext (cached) +ok chat-agent-system/internal/projectinstance (cached) +ok chat-agent-system/internal/projectworkflow (cached) +ok chat-agent-system/internal/quote (cached) +ok chat-agent-system/internal/replyexport (cached) +ok chat-agent-system/internal/server 9.796s +ok chat-agent-system/internal/squad (cached) +ok chat-agent-system/internal/storage (cached) +ok chat-agent-system/internal/stream (cached) +ok chat-agent-system/internal/taskrun 0.149s +ok chat-agent-system/internal/template (cached) +ok chat-agent-system/internal/upload (cached) +? chat-agent-system/internal/workflow [no test files] +ok chat-agent-system/internal/workflow/runtime (cached) +ok chat-agent-system/internal/workflowdef 0.003s +ok chat-agent-system/internal/workspace (cached) +? chat-agent-system/templates [no test files] From 7f8ad76d121885e7976e20768ae8566c82afb3c1 Mon Sep 17 00:00:00 2001 From: weichen Date: Mon, 14 Sep 2026 14:24:46 +0000 Subject: [PATCH 11/11] test(docgov): add two-pool races and ledger-first live spec Occupy the integration slot for M1 live sign-off, cover two pgx pools plus four sessions, and add a Playwright spec that treats projection consistency as 200 rather than 404. Co-authored-by: Cursor --- .doc/ops/CURRENT.md | 8 +- ...-14-document-reliability-m1-live-and-m2.md | 51 ++++++ ...-document-reliability-pilot-test-report.md | 4 +- .../document_commit_concurrency_pg_test.go | 147 ++++++++++++++++++ web/e2e/controlled-document-reload.spec.ts | 114 ++++++++++++++ 5 files changed, 318 insertions(+), 6 deletions(-) create mode 100644 .doc/ops/short-term/2026-09-14-document-reliability-m1-live-and-m2.md create mode 100644 web/e2e/controlled-document-reload.spec.ts diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index 885480a0..13467960 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -5,7 +5,7 @@ ## 焦点 -1. **DOC-RELIABILITY**: [文档可靠性纵向试点 M0–M1](../ops/short-term/2026-09-14-document-reliability-pilot.md)（`refactor/document-reliability-pilot`；`mutex=docgov`；M1 报告 **停止扩面**；不占槽、不 rebuild、不开 U5–U12） +1. **DOC-RELIABILITY**: [M1 活验签字与条件 M2](../ops/short-term/2026-09-14-document-reliability-m1-live-and-m2.md)（`refactor/document-reliability-pilot`；`mutex=docgov`；占集成槽 live 签字） 2. **CHAT-INTENT**: [对话意图运行时状态](../audit/2026-09-13-conversation-intent-runtime-state.md)（`fix/conversation-intent-state`；与文档账本运行时无关） 3. **CHAT**: [对话历史刷新](../audit/2026-09-13-chat-history-refresh-disappears.md)（认领仍 `in_progress`；gap 行已 PASS，以代码/handoff 为准收敛） @@ -13,7 +13,7 @@ | owner | task | branch | worktree | mutex | status | |-------|------|--------|----------|-------|--------| -| Cursor | document-reliability-pilot | refactor/document-reliability-pilot | .worktrees/refactor-document-reliability-pilot | docgov | blocked | +| Cursor | document-reliability-m1-live-and-m2 | refactor/document-reliability-pilot | .worktrees/refactor-document-reliability-pilot | docgov | in_progress | | Codex | module-decoupling-remediation-plan | feature/module-decoupling-remediation | .worktrees/feature/module-decoupling-remediation | architecture-docs | done | | verifier_antigravity | controlled-content-ui-acceptance | fix/controlled-content-ui-acceptance | .worktrees/fix-controlled-content-ui-acceptance | document-content-ui-acceptance | done | | implementer_cursor | controlled-content-canonical-hash | fix/controlled-content-canonical-hash | .worktrees/fix-controlled-content-canonical-hash | docgov | done | @@ -54,13 +54,13 @@ > 新任务开始前登记。`branch` 必须为 `feature/` 或 `fix/`，**禁止** `dev` / `test` / `main`。同一 `mutex` 同时仅一个 `in_progress`。 > workflow-plan 409 与治理鉴权 G1–G5 已收口（见 [test-report](../test-reports/2026-09-14-governance-action-level-authz-test-report.md)）。禁止终止或改写项目 143。 -> `document-reliability-pilot` 仅占 `docgov` 编码；合 `dev` + `env-service rebuild` 必须另占集成槽位。M1 结论为 **停止扩面**（见 [test-report](../test-reports/2026-09-14-document-reliability-pilot-test-report.md)），不得 rebuild，不得开工 U5–U12。 +> `document-reliability-pilot` 占 `docgov` + 集成槽做 M1 活验。禁止 worktree 内 `env-service`。项目 143 只读，禁止终止。 ## 集成槽位 | holder | phase | since | scope | queue | |--------|-------|-------|-------|-------| -| - | idle | - | - | - | +| Cursor | merging | 2026-09-14 | DOC-RELIABILITY M1 live | - | **phase**: `idle` | `merging` | `smoke` | `e2e` diff --git a/.doc/ops/short-term/2026-09-14-document-reliability-m1-live-and-m2.md b/.doc/ops/short-term/2026-09-14-document-reliability-m1-live-and-m2.md new file mode 100644 index 00000000..503c8900 --- /dev/null +++ b/.doc/ops/short-term/2026-09-14-document-reliability-m1-live-and-m2.md @@ -0,0 +1,51 @@ +--- +title: "文档可靠性 M1 活验签字与条件 M2" +status: in_progress +created: 2026-09-14 +updated: 2026-09-14 +owner: Cursor +branch: refactor/document-reliability-pilot +worktree: .worktrees/refactor-document-reliability-pilot +mutex: docgov +--- + +# 文档可靠性 M1 活验签字与条件 M2 + +## 目标 + +占集成槽把试点合入本地 `dev` 并 rebuild，补齐真实 PI / 浏览器 / 跨用户 / 多会话签字。仅当新报告最后一句为「继续 M2」时开工 U5–U7。 + +## 范围 + +- A1 占槽、合入、rebuild、health/ready +- A2 143 只读 + 新建项目页面 + fail-closed +- A3 真实 pi-agent 编制后刷新 +- A4 隔离库双 pool / 4 会话 +- A5 活验 go/no-go +- B 仅 go 后：U5–U7 + +## 非目标 + +- 不终止项目 143 +- 未 go 不开 outbox/FAV +- 不拆 chat/squad + +## 步骤 + +1. [ ] 占集成槽并合入本地 dev rebuild +2. [ ] 浏览器 + 跨用户 +3. [ ] 真实 PI +4. [ ] 双 pool/4 会话 +5. [ ] 活验报告 +6. [ ] 条件 M2 + +## 验收标准 + +- [ ] health + ready 来自本分支镜像 +- [ ] 143/新项目刷新 404/409/projection-missing = 0 或报告停止扩面 +- [ ] 报告最后一句只能是继续 M2 或停止扩面 + +## 相关链接 + +- Checklist: [../checklists/2026-09-14-module-decoupling-refactor-acceptance.md](../checklists/2026-09-14-module-decoupling-refactor-acceptance.md) +- Prior report: [../../test-reports/2026-09-14-document-reliability-pilot-test-report.md](../../test-reports/2026-09-14-document-reliability-pilot-test-report.md) diff --git a/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md b/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md index c415715a..cc3ae033 100644 --- a/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md +++ b/.doc/test-reports/2026-09-14-document-reliability-pilot-test-report.md @@ -56,7 +56,7 @@ ok chat-agent-system/internal/server 9.796s ``` - **评价判定**: **PASS** -- **性能与耗时**: `internal/server` 约 9.8s，其余亚秒到数秒。日志：`assets/2026-09-14-document-reliability-pilot/go-test-short.log` +- **性能与耗时**: `internal/server` 约 9.8s，其余亚秒到数秒。日志：`assets/2026-09-14-document-reliability-pilot/go-test-short.txt` `./scripts/arch-check.sh --mode warn` → `arch-check warn: production gate passed`。`--mode block` 仍会因既有例外基线失败（U1 设计如此）。 @@ -73,7 +73,7 @@ ok chat-agent-system/internal/docrevision 0.024s ``` - **评价判定**: **PASS**（1–2 会话级）。**未**跑 4 个独立 app 进程。 -- **性能与耗时**: UoW 包 65ms；并发编辑 24ms。日志：`go-test-pg-race.log` +- **性能与耗时**: UoW 包 65ms；并发编辑 24ms。日志：`go-test-pg-race.txt` ### 5.3 故障注入 diff --git a/internal/docrevision/document_commit_concurrency_pg_test.go b/internal/docrevision/document_commit_concurrency_pg_test.go index 15e0daa5..54eaedf6 100644 --- a/internal/docrevision/document_commit_concurrency_pg_test.go +++ b/internal/docrevision/document_commit_concurrency_pg_test.go @@ -98,3 +98,150 @@ func TestDocumentCommitConcurrencyPG(t *testing.T) { t.Fatalf("head revision=%d", got.HeadRevisionNo) } } + +func refuseSharedDSN(t *testing.T, dsn string) { + t.Helper() + if strings.Contains(dsn, "5433") || strings.Contains(dsn, "agent_system_dev") || + (strings.Contains(dsn, "5434") && strings.Contains(dsn, "agent_system_test")) || + strings.Contains(dsn, "5435") { + t.Fatal("TEST_DATABASE_URL points at a shared env port; refuse agent_system_dev/test/main") + } +} + +func TestDocumentCommitTwoPoolsAndFourSessionsPG(t *testing.T) { + if testing.Short() { + t.Skip("postgresql concurrency is not part of -short") + } + dsn := db.SkipIfNoTestDatabaseURL(t) + refuseSharedDSN(t, dsn) + ctx := context.Background() + poolA, err := pgxpool.New(ctx, dsn) + if err != nil { + t.Fatal(err) + } + t.Cleanup(poolA.Close) + poolB, err := pgxpool.New(ctx, dsn) + if err != nil { + t.Fatal(err) + } + t.Cleanup(poolB.Close) + if err := db.RunMigrationsOnceForTest(ctx, poolA, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + + suffix := fmt.Sprintf("%d", time.Now().UnixNano()) + var projectID int64 + if err := poolA.QueryRow(ctx, `INSERT INTO projects (code, name, status) VALUES ($1,$2,'active') RETURNING id`, + "docflow2p-"+suffix, "docflow2p").Scan(&projectID); err != nil { + t.Fatalf("project: %v", err) + } + t.Cleanup(func() { + _, _ = poolA.Exec(context.Background(), `DELETE FROM document_change_events WHERE doc_id LIKE $1`, "docflow2p_"+suffix+"%") + _, _ = poolA.Exec(context.Background(), `DELETE FROM document_versions WHERE doc_id LIKE $1`, "docflow2p_"+suffix+"%") + _, _ = poolA.Exec(context.Background(), `DELETE FROM document_snapshots WHERE doc_id LIKE $1`, "docflow2p_"+suffix+"%") + _, _ = poolA.Exec(context.Background(), `DELETE FROM document_records WHERE project_id=$1`, projectID) + _, _ = poolA.Exec(context.Background(), `DELETE FROM projects WHERE id=$1`, projectID) + }) + + docID := "docflow2p_" + suffix + seed := docrevision.NewService(poolA) + gov := docgov.NewGovernanceEngine(poolA) + if _, err := gov.CreateOrGetDocument(ctx, docID, projectID, "intake", "t", "V0.5_feasibility", "ws/"+docID, 1, "A", 2, "R", 3, "P"); err != nil { + t.Fatal(err) + } + st, err := seed.OpenTemplateVersion(ctx, docID, "# base\n", docrevision.ActorSnapshot{}, projectID, "ws/"+docID) + if err != nil { + t.Fatal(err) + } + + a := &docflow.Commands{UoW: uow.NewPG(poolA), Gov: docgov.NewGovernanceEngine(poolA), Rev: docrevision.NewService(poolA)} + b := &docflow.Commands{UoW: uow.NewPG(poolB), Gov: docgov.NewGovernanceEngine(poolB), Rev: docrevision.NewService(poolB)} + edit := func(cmds *docflow.Commands, i int) error { + _, err := cmds.HumanEdit(ctx, docflow.Principal{UserID: 1}, docflow.HumanEditCommand{ + DocID: docID, + Request: docrevision.HumanRevisionRequest{ + DocID: docID, Markdown: fmt.Sprintf("# base\n\npool %d\n", i), + ExpectedHeadSnapshot: &st.HeadSnapshotID, ExpectedRevisionNo: &st.HeadRevisionNo, + IdempotencyKey: fmt.Sprintf("2p-%s-%d", suffix, i), ActorUserID: 1, + LifecycleState: "drafting", ProjectID: projectID, WorkDir: "ws/" + docID, + }, + }) + return err + } + + var wg sync.WaitGroup + errs := make([]error, 2) + wg.Add(2) + go func() { defer wg.Done(); errs[0] = edit(a, 0) }() + go func() { defer wg.Done(); errs[1] = edit(b, 1) }() + wg.Wait() + wins := 0 + for _, err := range errs { + if err == nil { + wins++ + } + } + if wins != 1 { + t.Fatalf("two pools same etag: wins=%d errs=%v", wins, errs) + } + + // Four sessions: two pools × two sequential idempotent retries of the winner key must replay. + winnerKey := "" + if errs[0] == nil { + winnerKey = fmt.Sprintf("2p-%s-0", suffix) + } else { + winnerKey = fmt.Sprintf("2p-%s-1", suffix) + } + replay := docrevision.HumanRevisionRequest{ + DocID: docID, Markdown: "# ignored-if-same-key\n", + IdempotencyKey: winnerKey, ActorUserID: 1, LifecycleState: "drafting", + ProjectID: projectID, WorkDir: "ws/" + docID, + } + first, err := a.HumanEdit(ctx, docflow.Principal{UserID: 1}, docflow.HumanEditCommand{DocID: docID, Request: replay}) + if err != nil { + t.Fatal(err) + } + second, err := b.HumanEdit(ctx, docflow.Principal{UserID: 1}, docflow.HumanEditCommand{DocID: docID, Request: replay}) + if err != nil { + t.Fatal(err) + } + if first.State.HeadSnapshotID != second.State.HeadSnapshotID { + t.Fatalf("idempotent replay across pools diverged") + } + + var extra sync.WaitGroup + extraWins := 0 + var extraMu sync.Mutex + st2, err := seed.GetRevisionState(ctx, docID) + if err != nil { + t.Fatal(err) + } + for i := 0; i < 4; i++ { + extra.Add(1) + go func(i int) { + defer extra.Done() + cmds := a + if i%2 == 1 { + cmds = b + } + _, err := cmds.HumanEdit(ctx, docflow.Principal{UserID: 1}, docflow.HumanEditCommand{ + DocID: docID, + Request: docrevision.HumanRevisionRequest{ + DocID: docID, Markdown: fmt.Sprintf("# base\n\nfour %d\n", i), + ExpectedHeadSnapshot: &st2.HeadSnapshotID, ExpectedRevisionNo: &st2.HeadRevisionNo, + IdempotencyKey: fmt.Sprintf("4s-%s-%d", suffix, i), ActorUserID: 1, + LifecycleState: "drafting", ProjectID: projectID, WorkDir: "ws/" + docID, + }, + }) + if err == nil { + extraMu.Lock() + extraWins++ + extraMu.Unlock() + } + }(i) + } + extra.Wait() + if extraWins != 1 { + t.Fatalf("four sessions same etag: wins=%d", extraWins) + } +} diff --git a/web/e2e/controlled-document-reload.spec.ts b/web/e2e/controlled-document-reload.spec.ts new file mode 100644 index 00000000..c363c5ac --- /dev/null +++ b/web/e2e/controlled-document-reload.spec.ts @@ -0,0 +1,114 @@ +import { test, expect, type Page } from '@playwright/test'; +import * as fs from 'fs'; +import * as path from 'path'; +import { fileURLToPath } from 'url'; + +/** + * M1 live: ledger-first GET. Projection missing must not become 404/409. + * Project 143 is read-only. Do not terminate or PUT workflow-plan. + */ +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const runAcceptance = process.env.RUN_UI_ACCEPTANCE === '1'; +const ASSETS = process.env.UI_ACCEPTANCE_ASSETS_DIR + ? path.resolve(process.env.UI_ACCEPTANCE_ASSETS_DIR) + : path.resolve( + __dirname, + '../../.doc/test-reports/assets/2026-09-14-document-reliability-pilot-live', + ); +const SCREENSHOTS = path.join(ASSETS, 'screenshots'); +fs.mkdirSync(SCREENSHOTS, { recursive: true }); + +const ADMIN_USER = process.env.ADMIN_USERNAME?.trim() || ''; +const ADMIN_PASS = process.env.ADMIN_PASSWORD?.trim() || ''; +const BASE = process.env.BASE_URL || 'http://127.0.0.1:8080'; +const TARGET_DOC = 'doc_proj143_intake'; + +test.skip(!runAcceptance, 'set RUN_UI_ACCEPTANCE=1 against the rebuilt dev stack'); + +async function login(page: Page, username: string, password: string) { + await page.goto('/'); + await page.fill('input[placeholder*="系统用户名"]', username); + await page.fill('input[placeholder*="登录密码"]', password); + await page.click('button:has-text("进入医疗协同工作台")'); + await expect(page.getByText('新建立项会话')).toBeVisible({ timeout: 30_000 }); +} + +test.describe.serial('controlled document reload M1 live', () => { + test('project 143 content is 200 with consistency; anonymous is 401; unknown is 404', async ({ + browser, + }) => { + if (!ADMIN_USER || !ADMIN_PASS) { + throw new Error('ADMIN_USERNAME and ADMIN_PASSWORD are required'); + } + const page = await browser.newPage(); + const contentHits: { status: number; consistency?: string; error?: string }[] = []; + page.on('response', async (res) => { + const u = res.url(); + if (!u.includes(`/docgov/documents/${TARGET_DOC}/content`)) return; + let consistency: string | undefined; + let error: string | undefined; + try { + const json = (await res.json()) as { consistency?: string; error_code?: string }; + consistency = json.consistency; + error = json.error_code; + } catch { + /* ignore non-json */ + } + contentHits.push({ status: res.status(), consistency, error }); + }); + + await login(page, ADMIN_USER, ADMIN_PASS); + await page.getByRole('complementary').getByTitle('对话协同会话', { exact: true }).click({ force: true }); + await expect(page.getByRole('banner')).toContainText('对话协同会话', { timeout: 20_000 }); + await page.screenshot({ path: path.join(SCREENSHOTS, '01-project-143.png'), fullPage: true }); + + const token = await page.evaluate(() => { + try { + const raw = window.localStorage.getItem('smdf_auth_session'); + if (!raw) return ''; + const session = JSON.parse(raw) as { token?: string }; + return session.token || ''; + } catch { + return ''; + } + }); + if (!token) { + throw new Error('login did not persist smdf_auth_session.token'); + } + + const authed = await page.request.get(`${BASE}/api/v1/docgov/documents/${TARGET_DOC}/content`, { + headers: token ? { Authorization: `Bearer ${token}` } : {}, + }); + const authedBody = await authed.json(); + expect(authed.status(), `content ${JSON.stringify(authedBody)}`).toBe(200); + expect(authedBody.markdown, 'ledger markdown required').toBeTruthy(); + expect(['aligned', 'missing', 'drift', 'empty', 'ledger_missing']).toContain(authedBody.consistency); + expect(authedBody.error_code ?? '').not.toMatch(/PROJECTION/); + + await page.reload(); + await expect(page.getByRole('banner')).toContainText('对话协同会话', { timeout: 20_000 }); + const afterReload = await page.request.get(`${BASE}/api/v1/docgov/documents/${TARGET_DOC}/content`, { + headers: token ? { Authorization: `Bearer ${token}` } : {}, + }); + expect(afterReload.status()).toBe(200); + const reloadBody = await afterReload.json(); + expect(reloadBody.markdown).toBeTruthy(); + + await page.screenshot({ path: path.join(SCREENSHOTS, '02-after-reload.png'), fullPage: true }); + + const anon = await page.request.get(`${BASE}/api/v1/docgov/documents/${TARGET_DOC}/content`); + expect(anon.status()).toBe(401); + + const missing = await page.request.get(`${BASE}/api/v1/docgov/documents/doc_does_not_exist_m1/content`, { + headers: token ? { Authorization: `Bearer ${token}` } : {}, + }); + expect(missing.status()).toBe(404); + const missingBody = await missing.json(); + expect(missingBody.error_code).toBe('DOCUMENT_NOT_FOUND'); + + const projection409 = contentHits.filter( + (h) => h.status === 409 || String(h.error || '').includes('PROJECTION'), + ); + expect(projection409, JSON.stringify(contentHits)).toHaveLength(0); + }); +});