diff --git a/.task/feature-container-storage-plan-u1.md b/.task/feature-container-storage-plan-u1.md new file mode 100644 index 00000000..6f5a73c1 --- /dev/null +++ b/.task/feature-container-storage-plan-u1.md @@ -0,0 +1,178 @@ +# feature/container-storage-plan-u1 任务卡 + +任务主记录：；状态：进行中；owner：weichen + +- 权威仓：ssh://git@192.168.3.2:2222/weichen/Agent-System-Dev.git；目标分支：main；R0（initial_base_sha）= 955a32de2bb49171348bad5f566cce935df19212 +- 工作区：/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan-u1（worktree）；分支：feature/container-storage-plan-u1 +- 分支类型：child；initiative_id：feature-container-storage-plan；root_target：main；immediate_target：feature/container-storage-plan；accepting_requirements：false；parent_branch：feature/container-storage-plan；parent_head_sha：b360bcb9745e20d9d4f9f06c44a2936f47217ae4；parent_pr：无；parent_source：origin +- 需求分流：免（父 initiative 子任务登记已列出 feature/container-storage-plan-u1；parent_head_sha=b360bcb9745e20d9d4f9f06c44a2936f47217ae4） + +- 交付模式：goal-first（目标、范围和交付终点一次确认；范围内连续推进） +- 实施执行：agent-per-unit（配置：AGENT-EXECUTION.md） +- 合入执行：agent（已授权交付范围内，Review 通过、冲突解决且当前 HEAD 的 CI 通过后合入） +- 流程依据：仓库根目录 policy/delivery.md（归档后仍按仓库根目录定位） + +- 授权与交付终点：在父 initiative 授权范围内实施方案 U1（环境与数据归属合同），交付终点为合入父分支 feature/container-storage-plan。不含：生产数据盘点/访问、部署、生产迁移、容器改造实施（U2 起）。 +- 当前阶段：实施与自检 + +## 目标与非目标 + +**目标（方案 U1）**：扩展现有实例配置与 manifest，绑定基座库、RDM 领域库、知识库三个逻辑数据库、持久文件根和环境身份（prod/test/verify）；产出全部必需持久数据所有者的归属清单（类型、源版本、目录、引用形式、唯一写者；不含秘密与业务内容）。 + +**主要落点**：`scripts/svc_manifest.py`、`scripts/svc.sh`、`docs/svc.md`、`PROJECT.md`；新增环境 schema/示例建议置 `deploy/`。测试扩展 `tests/test_svc_identity.py`、`tests/test_svc_lifecycle_isolation.py`、`tests/test_svc_precheck.py`。 + +**完成条件**：配置、归属清单和迁移范围有唯一解释；明确需迁移的每个数据所有者；旧配置升级保留身份；错库/错根/foreign marker/缺失必需绑定拒绝；同名项目或冲突端口在修改前失败；在另一 checkout 调用仍定位同一资源。 + +**非目标**：不实现容器启停（U2）、迁移执行（U3）、备份恢复包（U4）、克隆隔离（U5）；不改业务权限模型；不引入第二份 DSN/卷权威（KTD3：manifest 唯一权威）。 + +## Agent 实施收据 + +路由与 fail-stop 规则见根目录 AGENT-EXECUTION.md 与 policy/delivery.md；每次尝试一行，原始日志留在源码外收据路径。 + +| 单元 ID | 预期范围 | 配置摘要 | 角色 | 候选顺序 | attempt | CLI/模型 | 终态 | transport | fallback | 源码外收据路径 | +|---|---|---|---|---|---|---|---|---|---|---| +| container-storage-u1-env-data-ownership | manifest/svc 绑定三库+文件根+环境身份；数据归属清单 | ccceb3bf… | implementation | cursor,antigravity,pi-glm,pi,codex | 1 | cursor(auto) | failed-unknown | 无（失败保留） | 否 | /tmp/container-storage-u1-run-receipt.json | +| container-storage-u1-env-data-ownership | 同上（采用 attempt1 补丁并验证） | ccceb3bf… | implementation | cursor,… | 2 | cursor(auto) | failed-unknown | 无（JSON 后带尾文解码失败，现场保留） | 否 | /tmp/container-storage-u1-run2-receipt.json | +| container-storage-u1-env-data-ownership | 同上 | ccceb3bf… | implementation | cursor,… | 3 | cursor(auto) | failed-unknown | 无（CLI 启动即报 Cannot use this model: auto，模型列表为空；单测 CLI 正常，判瞬时故障） | 否 | /tmp/container-storage-u1-run3-receipt.json | +| container-storage-u1-env-data-ownership | 同上 | ccceb3bf… | implementation | cursor,antigravity,… | 4 | cursor(auto) | failed-unknown | 无（工作完成且 80 测试通过，但终态 JSON 后再次附带尾文解码失败；cursor 累计 4 次尝试耗尽，转下一候选） | 否 | /tmp/container-storage-u1-run4-receipt.json | +| container-storage-u1-env-data-ownership | — | ccceb3bf… | implementation | （run5 于尝试启动前被 host 终止并删除其私有运行目录，无工作产物；目的：避免 cursor 超上限重试） | 5a | — | cancelled | 无 | 否 | 无（无收据；host 披露于本行） | +| container-storage-routing-promote-antigravity | AGENT-EXECUTION.md implementation 顺序临时调整 | ccceb3bf… | implementation | cursor,… | 1（第 2 次运行；第 1 次遇 cursor 模型列表瞬时为空 failed-unknown） | cursor(auto) | completed | apply（0c8bea37） | 否 | /tmp/container-storage-routing-receipt2.json | +| container-storage-u1-env-data-ownership | 同上（采用 attempt1 补丁内容，经 antigravity 全新会话独立重做并验证） | ccceb3bf… | implementation | antigravity,pi-glm,pi,cursor,codex | 1 | agy(gemini-3.8-flash-high) | completed | apply（2d75d101） | 否 | /tmp/container-storage-u1-run7-receipt.json | +| 待定义 | — | NOT_RUN | NOT_RUN | NOT_RUN | NOT_RUN | NOT_RUN | NOT_RUN | NOT_RUN | NOT_RUN | — | + +## 同步记录 + +| 节点 | sync_base_sha | tested_head_sha | checked_at | +|---|---|---|---| +| 启动 | 955a32de2bb49171348bad5f566cce935df19212 | b360bcb9745e20d9d4f9f06c44a2936f47217ae4 | 2026-10-22T09:12:44Z | + +## 证据（Evidence） + + +| 时间(UTC) | 命令 | 被测HEAD | exit | 来源 | sha256(16) | +|---|---|---|---|---|---| + +## 结果与验证（完成后填写） + +- 结果：U1 完成。manifest/svc 扩展三库+文件根+环境身份绑定与错误拒绝；deploy/data-ownership.md 归属清单与 env-bindings.example.env 示例；文档同步。经 agent-exec transport ed39db43 进入分支（2d75d101）。 +- 验证：三套件 80 项 PASS（1 skip，SKIP 为环境缺 DSN 的既有用例）；make test-affected 574 项 PASS（18 skip）。证据见上方两个导入收据（SELF）。容器真实运行验证属 U2，NOT_RUN。 +- 影响与剩余项：无阻塞。生产数据盘点需授权（方案 U1 生产部分推迟到 O1 前核实）。 + +## 合入与清理 + +- Review 记录：REVIEW PASS（无阻塞项；N1—N8 非阻塞，报告 docs/superpowers/reviews/2026-10-22-container-storage-u1-review.md，transport 7f0f... 见 /tmp/container-storage-u1-review-receipt2.json）。审阅者：pi-glm 全新会话（fresh_session，非实施者）；base 955a32de、head e1f08650；重放 80 项单测 OK +- 冲突记录：NOT_RUN（最新目标同步、冲突处置、组合兼容核验及复审/补测） +- CI 记录：NOT_RUN（当前 HEAD 完整 SHA、全部必需检查、运行链接、执行身份及 GATE 结果） +- 合入记录：NOT_RUN（交付授权范围、PR、被核验 HEAD、合入 SHA、目标分支及内容核验） +- 清理记录：NOT_RUN（授权依据、写者退出确认、worktree/分支/任务卡处置） + + +```json +{ + "summary_schema": 1, + "source": "SELF", + "timestamp": "2026-10-22T09:11:55.612223+00:00", + "command": "make precheck", + "exit_code": 0, + "cwd": "/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan", + "before_head": "b360bcb9745e20d9d4f9f06c44a2936f47217ae4", + "after_head": "b360bcb9745e20d9d4f9f06c44a2936f47217ae4", + "branch": "feature/container-storage-plan", + "target_branch": "main", + "target_head": "955a32de2bb49171348bad5f566cce935df19212", + "root_target": "main", + "immediate_target": "main", + "before_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "after_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "output_tail": [ + "make[1]: Entering directory '/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan'", + "[repo] branch=feature/container-storage-plan head=b360bcb974", + "[remote] origin/main=955a32de2b ahead=2 behind=0", + "[tree] clean submodules_drift=0 uninit=0", + "[worktrees] linked=6", + "[initiatives] local cards (not authoritative): feature/container-storage-plan", + "[hooks] core.hooksPath=scripts/hooks（pre-commit/post-merge 实际接线）", + "[facts] policy/facts.md 6 条事实均在有效期内", + "[required-reading] core/baseline.md, workflows/delivery.md §1, policy/delivery.md", + "[R0] 955a32de2b（完整：955a32de2bb49171348bad5f566cce935df19212）", + "[worktree] before-sync: recursive component checks passed", + "[verdict] READY — 保留本次 precheck receipt 后建任务工作区：PRECHECK_RECEIPT=/tmp/task-precheck.json make task-start NAME=feature/your-branch", + "make[1]: Leaving directory '/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan'" + ], + "output_sha256": "58ef898d944057a44fb02e1fabb6acc2fd97943beed355977229d13711b2e6b2", + "receipt_sha256": "59e8b2e7b7032c19207a19f9427486439ca15250333e83ddd162db739c5dc395", + "receipt_path": "/home/weichen/.cache/agent-system-dev/evidence/59e8b2e7b7032c19207a19f9427486439ca15250333e83ddd162db739c5dc395.json" +} +``` + + +```json +{ + "summary_schema": 1, + "source": "SELF", + "timestamp": "2026-10-22T10:41:41.286181+00:00", + "command": "PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.test_svc_identity tests.test_svc_lifecycle_isolation tests.test_svc_precheck", + "exit_code": 0, + "cwd": "/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan-u1", + "before_head": "2d75d101a04ce1d1c9fa1d66d9567894222399d5", + "after_head": "2d75d101a04ce1d1c9fa1d66d9567894222399d5", + "branch": "feature/container-storage-plan-u1", + "target_branch": "main", + "target_head": "98ee59c477785b2d5094fcbea3e67a4dc864c388", + "root_target": "main", + "immediate_target": "feature/container-storage-plan", + "before_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "after_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "output_tail": [ + "........................s.......................................................", + "----------------------------------------------------------------------", + "Ran 80 tests in 66.437s", + "", + "OK (skipped=1)" + ], + "output_sha256": "1bc6f83d7e4c64a628d5f29bad07fc4e8cc2468937c66bda4bd9a5e00fc22587", + "receipt_sha256": "bfe74991277ebe308f667532e9b21af8215f453db124afeab1fc3d2115adf67d", + "receipt_path": "/home/weichen/.cache/agent-system-dev/evidence/bfe74991277ebe308f667532e9b21af8215f453db124afeab1fc3d2115adf67d.json" +} +``` + + +```json +{ + "summary_schema": 1, + "source": "SELF", + "timestamp": "2026-10-22T10:42:53.576009+00:00", + "command": "make test-affected", + "exit_code": 0, + "cwd": "/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan-u1", + "before_head": "2d75d101a04ce1d1c9fa1d66d9567894222399d5", + "after_head": "2d75d101a04ce1d1c9fa1d66d9567894222399d5", + "branch": "feature/container-storage-plan-u1", + "target_branch": "main", + "target_head": "98ee59c477785b2d5094fcbea3e67a4dc864c388", + "root_target": "main", + "immediate_target": "feature/container-storage-plan", + "before_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "after_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "output_tail": [ + "test_invalid_json_is_denied (test_write_guard.WriteGuardTests.test_invalid_json_is_denied) ... ok", + "test_missing_parent_on_main_is_denied (test_write_guard.WriteGuardTests.test_missing_parent_on_main_is_denied) ... ok", + "test_missing_unknown_and_malformed_write_input_are_denied (test_write_guard.WriteGuardTests.test_missing_unknown_and_malformed_write_input_are_denied) ... ok", + "test_non_git_destination_is_explicitly_outside_scope (test_write_guard.WriteGuardTests.test_non_git_destination_is_explicitly_outside_scope) ... ok", + "test_nonwrite_tools_are_outside_scope (test_write_guard.WriteGuardTests.test_nonwrite_tools_are_outside_scope) ... ok", + "test_relative_path_uses_hook_cwd_not_process_cwd (test_write_guard.WriteGuardTests.test_relative_path_uses_hook_cwd_not_process_cwd) ... ok", + "test_symlink_into_main_with_missing_parents_is_denied (test_write_guard.WriteGuardTests.test_symlink_into_main_with_missing_parents_is_denied) ... ok", + "test_valid_multi_file_patch_on_feature_is_allowed (test_write_guard.WriteGuardTests.test_valid_multi_file_patch_on_feature_is_allowed) ... ok", + "", + "----------------------------------------------------------------------", + "Ran 574 tests in 400.853s", + "", + "OK (skipped=18)", + "make[2]: Leaving directory '/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan-u1'", + "make[1]: Leaving directory '/home/weichen/projects/Agent-System-Dev-wt-feature-container-storage-plan-u1'" + ], + "output_sha256": "d738cef7d9ea78bcc4624e0fe8eb87b651e88712fcd283bb26c21755168604e9", + "receipt_sha256": "0565d5b327cb8d01e48deda867bd7369dd5728b5ad9024b51bbd6cb1a2fdf246", + "receipt_path": "/home/weichen/.cache/agent-system-dev/evidence/0565d5b327cb8d01e48deda867bd7369dd5728b5ad9024b51bbd6cb1a2fdf246.json" +} +``` diff --git a/AGENT-EXECUTION.md b/AGENT-EXECUTION.md index 502cc4c0..03943637 100644 --- a/AGENT-EXECUTION.md +++ b/AGENT-EXECUTION.md @@ -41,7 +41,9 @@ execution: max_attempts_per_profile: 3 roles: - implementation: [cursor, antigravity, pi-glm, pi, codex] + # cursor 在 U1 单元已耗尽 4 次尝试：1 次传输中断、2 次终态 JSON 后附带尾文、1 次模型列表瞬时为空。 + # 临时调整，交付前回退。 + implementation: [antigravity, pi-glm, pi, cursor, codex] repair: [cursor, antigravity, pi-glm, pi, codex] test: [pi-glm, pi, antigravity, cursor, codex] review: [pi-glm, pi, antigravity, cursor, codex] diff --git a/PROJECT.md b/PROJECT.md index e94b99dd..4c8ba361 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -2,7 +2,10 @@ - 用途：开发集成父仓（gitlink 权威）；非分发单元。 - 组合：`modules/`（产品组件）+ `deps/`（共享依赖）；完整组件清单以 [`.gitmodules`](.gitmodules) 为准。 -- 部署：非容器；Releases 仓从本仓固定提交导出版本。 +- 部署：非容器为现行默认（`storage_mode=legacy`）；容器化存储改造见 + [容器化存储方案](docs/superpowers/plans/2026-10-22-container-storage-plan.md)， + 环境与数据归属合同（U1）见 [deploy/data-ownership.md](deploy/data-ownership.md)。 + Releases 仓从本仓固定提交导出版本。 ## 验证 @@ -56,6 +59,8 @@ [计划](modules/documents/docs/proposals/2026-10-12-work-companion-ui-plan.md) 覆盖任务摘要、对话/输入、窄屏和真实状态投影。 - [服务重启与全量数据恢复](modules/documents/docs/proposals/2026-10-12-service-restart-data-recovery.md)：svc 启停、异常退出后的依赖、配置/数据身份、任务/HITL/副作用、文件与组合备份恢复； U8 规格见 [SQLite 实例守护](docs/recovery/u8-sqlite-instance-guard.md)。 +- [单一应用二进制与容器化数据库、知识库存储](docs/superpowers/plans/2026-10-22-container-storage-plan.md)：两容器三库、外置文件根、环境独立绑定； + U1 归属清单见 [deploy/data-ownership.md](deploy/data-ownership.md)。 - [svc 统一部署与一次性登录 Key](modules/documents/docs/proposals/2026-10-12-svc-unified-deploy-and-login-key.md)： 二进制/源码统一部署、环境准备、无初始密码开户与账户恢复；规格见 `docs/svc-deploy/`， 账户与恢复续审见 [续审记录](docs/svc-deploy/2026-10-12-account-durability-plan-review.md)。 diff --git a/deploy/data-ownership.md b/deploy/data-ownership.md new file mode 100644 index 00000000..74a85245 --- /dev/null +++ b/deploy/data-ownership.md @@ -0,0 +1,40 @@ +# 容器化存储：数据归属清单（U1） + +权威设计：`docs/superpowers/plans/2026-10-22-container-storage-plan.md`（U1、KTD1—KTD4）。 +实例身份与期望态唯一权威：`runtime//manifest.json`（`scripts/svc_manifest.py`）； +本清单只登记**数据所有者、类型、引用形式与唯一写者**，不含秘密值与业务内容。 + +生产体量/源版本盘点需另行授权访问后补全；未核实项不得据此声明迁移完成。 + +## 归属总表 + +| 所有者 | 类型 | 引用形式（登记键） | 唯一写者 | 迁移范围 | +|---|---|---|---|---| +| 基座逻辑库 | PostgreSQL 逻辑库（账号/项目/消息/文档/设置等基座表族） | `base_dbref` 或 `base_dsn_ref`（绝对路径秘密文件）+ `base_role_ref` + `base_port`；身份锚 `identity.base_db_ref` | 本实例应用进程（经基座 DSN）；引导/迁移角色与应用角色分离（U2） | 必需 | +| RDM 领域库 | PostgreSQL 逻辑库（RDM 域表/项目文档树等） | `rdm_dbref` 或 `rdm_dsn_ref` + `rdm_role_ref` + `rdm_port`；锚 `identity.rdm_db_ref` | 本实例应用内 RDM 域（独立 DSN）；非基座连接串 | 必需 | +| 知识库 | PostgreSQL/pgvector 逻辑库（语料、索引投影） | `kb_dbref` 或 `kb_dsn_ref` + `kb_role_ref` + `kb_port`；锚 `identity.kb_db_ref` | 本实例知识库接入（应用内）；索引可重建但语料库为权威 | 必需 | +| 持久文件根 | 目录树（工件、RDM uploads、KB 原件及必要解析产物） | `files_root`（**绝对路径**，不相对 checkout/worktree 解析）；锚 `identity.files_root` | 本实例应用进程；禁止他实例共用同一根 | 必需 | +| 环境身份 | 枚举 | `env_class` ∈ `prod`\|`test`\|`verify`；锚 `identity.env_class` | 运维显式登记；克隆时换新（U5） | 必需（container） | +| 实例身份 | 稳定 ID + 数据 marker | `instance_id`；`/.agentx/data-marker.json` | `svc` manifest 治理；启动只校验不重建 | 必需 | +| Compose 项目 | 容器项目名 | `compose_project`；同 runtime 内唯一 | 本环境容器驱动（U2）；冲突在修改前拒绝 | container 必需 | + +`storage_mode=legacy`（缺省）：沿用既有 SQLite / 宿主 PG（`db`/`dbref`/`dsn`）路径，不要求三库与 `files_root`。 +`storage_mode=container`：上表 container 行全部必需；缺任一绑定、错库、错文件根、foreign marker、同名 Compose 或端口冲突 → **拒绝**，不静默建空库、不回退默认。 + +凭据只允许以受保护文件路径引用（`*_dsn_ref` / `*_role_ref`），明文不得入 manifest、日志或本清单。 + +## 配置形态 + +见同目录 [`env-bindings.example.env`](env-bindings.example.env)。写入实例： + +```bash +# 先写齐绑定键，最后打开 container 合同（缺一即拒） +scripts/svc.sh … # 经 svc_set_key / manifest set；Compose·端口冲突在修改前失败 +``` + +另一 checkout/worktree 只要指向同一绝对 `files_root` 与库引用，仍定位同一资源（路径不相对仓库根解析）。 + +## 与旧恢复清单关系 + +细粒度基座表/插件枚举仍见 [`docs/recovery/u1-data-ownership.md`](../docs/recovery/u1-data-ownership.md)。 +本文件是容器化三库+文件根的**合同级**归属；实施迁移（U3）时两者对照，必需所有者未覆盖则阻塞切换。 diff --git a/deploy/env-bindings.example.env b/deploy/env-bindings.example.env new file mode 100644 index 00000000..103985d6 --- /dev/null +++ b/deploy/env-bindings.example.env @@ -0,0 +1,31 @@ +# 容器化存储实例绑定示例（U1）——写入 runtime/ 经 manifest 权威登记。 +# 秘密只引用受保护文件路径，不在此填写明文密码/DSN 值。 +# 完整合同见 deploy/data-ownership.md；schema 键由 scripts/svc_manifest.py 校验。 + +# legacy（缺省）| container（三库+文件根+环境身份全部必需） +storage_mode=container + +# 环境身份：prod | test | verify（三选一） +env_class=prod + +# 持久文件根：必须是绝对路径（不相对代码 checkout/worktree 解析） +files_root=/var/lib/agent-system/state/prod/files + +# Compose 项目名：同机各环境唯一；冲突在修改前拒绝 +compose_project=agentx-prod + +# 基座逻辑库：dbref=/<库名> 或 dsn_ref=<绝对路径秘密文件>（二选一） +# base_dbref=pg-business/agent_base +base_dsn_ref=/etc/agent-system/prod/base.dsn +base_role_ref=/etc/agent-system/prod/base.role +base_port=15432 + +# RDM 领域库 +rdm_dsn_ref=/etc/agent-system/prod/rdm.dsn +rdm_role_ref=/etc/agent-system/prod/rdm.role +rdm_port=15432 + +# 知识库（pgvector 容器独立端口） +kb_dsn_ref=/etc/agent-system/prod/kb.dsn +kb_role_ref=/etc/agent-system/prod/kb.role +kb_port=15433 diff --git a/docs/superpowers/reviews/2026-10-22-container-storage-u1-review.md b/docs/superpowers/reviews/2026-10-22-container-storage-u1-review.md new file mode 100644 index 00000000..6cfd2eae --- /dev/null +++ b/docs/superpowers/reviews/2026-10-22-container-storage-u1-review.md @@ -0,0 +1,64 @@ +# 独立 Review：容器化存储 U1（环境与数据归属合同） + +- 日期：2026-10-22 +- 审阅对象：`feature/container-storage-plan-u1` 分支 HEAD + - base：`955a32de2bb49171348bad5f566cce935df19212` + - head：`e1f08650a12656611f27438f23fddbfab6a96299`（分支尖端；实施内容经 agent-exec transport 提交 `2d75d101a04ce1d1c9fa1d66d9567894222399d5`，另有路由调整 transport 提交 `0c8bea37` 与任务卡记录提交） + - 送审 diff：`/tmp/container-storage-u1-review.diff`（已核对与 `git diff 955a32de..HEAD` 的代码部分逐字一致；仅 `.task/**` 任务卡不在 diff 内，属 host 直写豁免区） +- 方案依据：`docs/superpowers/plans/2026-10-22-container-storage-plan.md` U1 小节、KTD1—KTD4 +- 审阅方式：全新会话独立审阅（非实施者）；实际读代码核对绑定解析、拒绝路径与身份保留逻辑；未实施任何修复。 + +## 总体判定：REVIEW PASS + +无阻塞项。U1 验收场景（旧配置升级保留身份、错库/错根/foreign marker 拒绝、同名项目或冲突端口修改前失败、另一 checkout 定位同一资源、缺必需绑定不静默新建空库）均有实现且被测试覆盖；KTD3 单一权威合同成立；无秘密入仓；测试 80/80 通过（1 项真实 PG 专项显式 NOT_RUN）。下列非阻塞项建议在 U2 及最终交付前处理。 + +## 阻塞项 + +无。 + +## 非阻塞项 + +| # | 级别 | 位置 | 发现 | 建议 | +|---|---|---|---|---| +| N1 | 建议 | `scripts/svc.sh` `svc_set_key`/`svc_del_key`（4674/4691 行起） | manifest `set` 因绑定校验或冲突被拒（exit 2/3/7）时，回退路径仍把该键写入派生视图 `svc.env`，整体操作"成功"返回。权威 manifest 未被污染（`resolve_storage_bindings` 只读 manifest），且两种漂移方向在启动时均 fail-closed（`svc.env=container`/`manifest=legacy` → `verify --require-storage` 触发 `完整存储合同要求 storage_mode=container` 拒启；反向漂移由 record_state 重建自愈），但"冲突在修改前失败"的语义被弱化为"权威拒绝+视图静默写入"。当前无任何调用方经 `svc_set_key` 写存储键（调用点仅 autostart/exec/dbref/dsn/conffile/addr/tags），风险处于潜伏状态。 | 存储键（STORAGE_KEYS 集合）在 manifest set 失败时硬失败，不走 legacy 单键回退。 | +| N2 | 建议 | `scripts/svc.sh` `svc_manifest_verify`（1302 行） | container 分支门控用 `svc_get storage_mode`（读派生 `svc.env`）而非 manifest identity。manifest=container 而 svc.env 过期（manifest 与 svc.env 两次原子写之间的崩溃窗口，或绕过 svc 手改 manifest）时，该次启动跳过存储校验；下次 record_state 经 bulk-set 重新派生后自愈。窗口窄且可自愈，但与"manifest 为权威"的门控意图不一致。 | 门控改为读 manifest identity 的 `storage_mode`（或在检测到两处不一致时拒绝并提示重建派生视图）。 | +| N3 | 建议 | `deploy/data-ownership.md`（持久文件根行） | 声明"禁止他实例共用同一根"，但 `check_conflicts` 只扫描 compose 项目名与端口，不检测跨实例 files_root 重叠/嵌套；测试夹具（precheck `test_compose_project_conflict_refused`）中 b 的根位于 a 的根之下，同样未拦截。无 U1 验收场景要求此项，属文档强于实现。 | 收紧措辞为"登记层不拦截，容器挂载层（U2）负责隔离"，或在 U2 增加跨实例根重叠预检。 | +| N4 | 记录 | `scripts/svc_manifest.py` `check_conflicts` | 冲突扫描范围限定同一 runtime 目录（`RUNTIME="$ROOT/runtime"`，`--runtime` 显式传入）。同机跨 checkout 的多个 runtime 目录之间的 compose 项目名/端口冲突在 U1 登记层不可见，实际碰撞推迟到 U2 容器启动时由端口占用/compose 报错暴露。实现注释与文档均按"同一 runtime"表述，范围自洽；U2 场景"prod/test/verify 同时运行互不冲突"已承接此项。 | U2 落地容器驱动时复核跨 runtime 同机冲突的最终防线。 | +| N5 | 建议 | `tests/` | 两个覆盖缺口：(a) 无 container 模式实例完整 start/stop 生命周期测试——`preserve_storage_keys`（record_state → bulk-set 写回时保住存储键）与 svc.sh 传 `--runtime` 的改动未被端到端 exercised，若该逻辑回归，绑定会在一次生命周期写回后被静默清掉；(b) `test_CS_U1_missing_binding_rejected` 只覆盖"全部缺失"，未覆盖"三库中任一缺失"（如 base/rdm 齐备仅 kb 缺失）的逐槽拒绝。 | 各补一条测试：container 实例启停后锚点与 kv 绑定不变；逐槽缺失各断言一次。 | +| N6 | 记录 | `AGENT-EXECUTION.md`（implementation 角色顺序） | 不在 U1 单元 allowed_paths 内，属越界文件变更；但它是独立登记的路由单元（任务卡行 `container-storage-routing-promote-antigravity`，transport `0c8bea37`，收据 `/tmp/container-storage-routing-receipt2.json`），披露完整，且变更注释自述"临时调整，交付前回退"。 | 按其自述在 U1 合入/交付前回退；不阻塞 U1 评审结论。 | +| N7 | 记录 | `scripts/svc_manifest.py` | `slot_port_of` 只校验 `isdigit()`，接受 `0` 与超范围数值，未限 1—65535。 | 顺手项，U2 接真实容器端口发布前收紧即可。 | +| N8 | 记录 | `scripts/svc_manifest.py` `cmd_verify` | container 启动路径把 `resolve_storage_bindings`（源自 manifest）的值回传给 `verify` 比对，本质是 identity↔kv 自一致性检查（可拦手改 manifest 的不一致），并非运行时"连到外宾库"检测；后者按方案属 U2（启动顺序真实认证/身份检查）与 U3（库内身份）范围。U1 的"错库拒绝"在绑定锚点层面成立（`base 库绑定漂移` 路径有测试），范围声明准确。 | 无需改动；U2 接线时保留本检查作为快速失败层。 | + +## 核对点（实际读代码/跑测试核对） + +1. **旧配置升级保留身份**：`build()` 对已带 storage 键的 kv 只做形态校验（`require_complete=(mode==container)`），不重建 instance_id；`test_CS_U1_legacy_upgrade_keeps_identity` 断言升级后 `instance_id`/`data_root` 不变、锚点落 identity。✅ +2. **身份保留/重锚**：`reanchor(write=False)` 预览仅用于冲突扫描、不落 marker；`mutate` 顺序为"校验 → 冲突扫描 → 重锚 → 落盘"，冲突时不落盘；`slot_ref_of` 冲突（dbref 与 dsn_ref 同设）、相对路径 `*_dsn_ref`/`*_role_ref`/`files_root` 均拒绝；legacy→container 升级基座可回落既有 `dbref=` 专键，明文 `dsn=` 不进锚点（I5）。✅ +3. **拒绝路径**：container 缺任一必需绑定在 set（`validate_storage_kv` exit 3）与 verify（`--require-storage` + `require_complete=True`）双端拒绝；错文件根/错库绑定漂移 exit 6（测试断言"文件根"/"base 库绑定漂移"）；foreign marker 沿用 base 逻辑（T5b）；compose 项目名/端口冲突 exit 7 且发生在落盘前（`check_conflicts` 在 `save` 之前）。✅ +4. **不静默新建空库**：U1 层无任何默认绑定生成；`resolve_storage_bindings` 缺绑定 exit 3 → svc.sh 拒绝；manifest 缺失时 verify exit 4、marker 缺失/损坏 exit 5，均阻塞启动。未发现静默回退空库路径。✅ +5. **另一 checkout 定位同一资源**：`files_root` 强制绝对路径并 normpath，锚点/解析不依赖 cwd 与仓库根；`test_CS_U1_other_checkout_keeps_absolute_files_root` 与 T6 从外部 cwd 调用验证。✅ +6. **调用方环境污染**：`env_config` 重置存储全局并 `unset AGENT_SYSTEM_FILES_ROOT/ENV_CLASS/COMPOSE_PROJECT/BASE_DB_REF/RDM_DB_REF/KB_DB_REF`；`test_caller_storage_env_does_not_reach_serve` 检查 serve 进程 environ 与 serve.env；三条棘轮测试守卫 env_config。✅ +7. **KTD3 单一权威**：`resolve_storage_bindings` 内嵌 python 直读 `manifest.json`（不读 svc.env、不读调用方 env）；U1 未引入 compose 文件/第二份卷或 DSN 权威；凭据仅存受保护文件路径引用。`svc.env` 仍是派生只读视图（record_state 不产出存储键，`preserve_storage_keys` 自 manifest 旧 kv 保留）。✅ +8. **归属清单与契约文档**：`deploy/data-ownership.md` 覆盖基座/RDM/KB/文件根/环境身份/实例身份/Compose 项目七类所有者，含引用形式与唯一写者；体量/源版本/表清单明示"需另行授权访问后补全"（与方案"生产盘点需先获得对应访问授权"及诚实出口一致）；无秘密值。`docs/svc.md` §10.1 键表与 `PROJECT.md` 指针已同步；链接目标 `docs/recovery/u1-data-ownership.md`、`docs/recovery/u1-identity-manifest.md` 均存在。✅ +9. **测试充分性**：三测试文件合计覆盖全部五条 U1 验收场景 + 调用方隔离 + 原子写崩溃窗口；缺口仅 N5 两项（均非验收场景直接要求）。`PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.test_svc_identity tests.test_svc_lifecycle_isolation tests.test_svc_precheck` → **80 tests, OK (skipped=1)**，唯一 skip 为真实 PG 专项显式 NOT_RUN（诚实标注）。✅ +10. **秘密与越界**：diff 无明文口令/DSN/token（示例 env 仅受保护路径，测试仅 fixture 值 `ref-only`/`x`）；changed files 中除 `AGENT-EXECUTION.md`（见 N6，已另行登记）外全部落在 U1 主要文件/allowed_paths 内；`.task/**` 为 host 直写豁免。✅ + +## 复现命令 + +```bash +# 1) 送审 diff 范围与实际 HEAD diff 一致（代码部分逐字相同） +git diff 955a32de2bb49171348bad5f566cce935df19212 HEAD \ + -- AGENT-EXECUTION.md PROJECT.md deploy docs/superpowers/plans docs/svc.md scripts tests \ + | diff - /tmp/container-storage-u1-review.diff && echo IDENTICAL + +# 2) 单元测试（本机实测：Ran 80 tests, OK (skipped=1)，约 64s） +PYTHONDONTWRITEBYTECODE=1 python3 -m unittest \ + tests.test_svc_identity tests.test_svc_lifecycle_isolation tests.test_svc_precheck + +# 3) 对象 SHA +git rev-parse HEAD # e1f08650a12656611f27438f23fddbfab6a96299 +git rev-parse 955a32de # 955a32de2bb49171348bad5f566cce935df19212 +``` + +## 结论 + +U1 单元按方案完成且质量合格：绑定/身份权威唯一、拒绝路径完整且 fail-closed、验收场景全部落地并有测试背书。上述 N1—N8 均不阻塞本单元交付；N1/N2/N5 建议随 U2 或合入前收口，N6 的路由临时调整须按其自述在交付前回退。 diff --git a/docs/svc.md b/docs/svc.md index 4c878c14..68f96c4b 100644 --- a/docs/svc.md +++ b/docs/svc.md @@ -866,9 +866,21 @@ scripts/svc.sh prod deploy --env .env.prod # 等价 | `dbref` | agent 托管数据库绑定 `/<库名>`（启动时由实例登记重建 DSN；见 §4.5） | | `dsn` | agent 外部直连 DSN（受保护；未命中本机实例登记的直连存放于此） | | `conffile` | agent 配置文件绝对路径（` config` 命令写入，跨重启保留；见 §4.6） | +| `storage_mode` | `legacy`（缺省，SQLite/宿主 PG）/ `container`（三库+文件根合同；见 [deploy/data-ownership.md](../deploy/data-ownership.md)） | +| `env_class` | 环境身份：`prod` / `test` / `verify`（container 必需） | +| `files_root` | 持久文件根**绝对路径**（不相对 checkout 解析；container 必需） | +| `compose_project` | Compose 项目名（同 runtime 唯一；冲突在修改前拒绝） | +| `base_dbref` / `base_dsn_ref` / `base_role_ref` / `base_port` | 基座逻辑库绑定（dsn/role 仅为受保护文件路径引用） | +| `rdm_dbref` / `rdm_dsn_ref` / `rdm_role_ref` / `rdm_port` | RDM 领域库绑定 | +| `kb_dbref` / `kb_dsn_ref` / `kb_role_ref` / `kb_port` | 知识库绑定 | | `user` / `password` / `dbname` / `authmode` / `pgdata` | postgres 专属 | | `htdocs` | static 专属 | +容器存储绑定由 `manifest.json` 冻结为身份锚点（与 `data_root`/`db_*` 同级）；旧配置 +升级保留 `instance_id`。`storage_mode=container` 时缺任一必需绑定、错库、错文件根、 +foreign marker、同名 Compose 或端口冲突均拒绝启动/修改，不静默建空库、不回退默认。 +示例键见 [`deploy/env-bindings.example.env`](../deploy/env-bindings.example.env)。 + ### 10.2 history.jsonl 事件 一行一个 JSON：`ts / event / env / kind / source / result / pid / addr / detail`。 diff --git a/scripts/svc.sh b/scripts/svc.sh index 546c41e5..1a4828cb 100755 --- a/scripts/svc.sh +++ b/scripts/svc.sh @@ -1012,8 +1012,9 @@ record_state() { chmod 600 "$tmp" # U1 身份：manifest 治理下的整份提交（bulk-set 走 temp→fsync→rename 原子链， # revision 递增，svc.env 由 manifest 派生）；失败保留旧配置，不产生半写态。 + # 存储绑定键（STORAGE_KEYS）由 manifest bulk-set 自旧 kv 保留，避免生命周期写回冲掉。 if manifest_ok; then - if python3 "$MANIFEST_PY" "$ENV_DIR" bulk-set < "$tmp"; then + if python3 "$MANIFEST_PY" "$ENV_DIR" bulk-set --runtime "$RUNTIME" < "$tmp"; then rm -f "$tmp" return 0 fi @@ -1217,18 +1218,103 @@ svc_manifest_migrate() { c_err "[$ENV_NAME] manifest 迁移失败（保持旧治理）：$(printf '%s' "$out" | tail -1)" return 0 } +# resolve_storage_bindings：从本实例 manifest 权威解析容器存储绑定（不读调用方环境）。 +# 结果全局：STORAGE_MODE ENV_CLASS FILES_ROOT COMPOSE_PROJECT +# BASE_DB_REF RDM_DB_REF KB_DB_REF BASE_PORT RDM_PORT KB_PORT +# storage_mode=container 且缺任一必需绑定时返回 1（拒绝静默默认/建空库）； +# legacy / 未配置返回 0 且 STORAGE_MODE=legacy。 +resolve_storage_bindings() { + STORAGE_MODE=legacy; ENV_CLASS=""; FILES_ROOT=""; COMPOSE_PROJECT="" + BASE_DB_REF=""; RDM_DB_REF=""; KB_DB_REF="" + BASE_PORT=""; RDM_PORT=""; KB_PORT="" + manifest_ok || return 0 + local out rc=0 + out="$(python3 - "$ENV_DIR" <<'PY' 2>&1 +import json, os, sys +env_dir = sys.argv[1] +path = os.path.join(env_dir, "manifest.json") +with open(path, encoding="utf-8") as f: + m = json.load(f) +ident = m.get("identity") or {} +kv = m.get("kv") or {} +mode = (kv.get("storage_mode") or ident.get("storage_mode") or "legacy").strip() or "legacy" +print("MODE=%s" % mode) +print("ENV_CLASS=%s" % (kv.get("env_class") or ident.get("env_class") or "")) +print("FILES_ROOT=%s" % (kv.get("files_root") or ident.get("files_root") or "")) +print("COMPOSE_PROJECT=%s" % (kv.get("compose_project") or ident.get("compose_project") or "")) +print("BASE_DB_REF=%s" % (ident.get("base_db_ref") or "")) +print("RDM_DB_REF=%s" % (ident.get("rdm_db_ref") or "")) +print("KB_DB_REF=%s" % (ident.get("kb_db_ref") or "")) +print("BASE_PORT=%s" % (kv.get("base_port") or ident.get("base_port") or "")) +print("RDM_PORT=%s" % (kv.get("rdm_port") or ident.get("rdm_port") or "")) +print("KB_PORT=%s" % (kv.get("kb_port") or ident.get("kb_port") or "")) +if mode == "container": + missing = [] + if (kv.get("env_class") or ident.get("env_class") or "") not in ("prod", "test", "verify"): + missing.append("env_class") + fr = kv.get("files_root") or ident.get("files_root") or "" + if not fr or not os.path.isabs(fr): + missing.append("files_root") + if not (kv.get("compose_project") or ident.get("compose_project") or ""): + missing.append("compose_project") + for slot, key in (("base", "base_db_ref"), ("rdm", "rdm_db_ref"), ("kb", "kb_db_ref")): + if not (ident.get(key) or ""): + missing.append(slot + "_db") + if not (kv.get(slot + "_port") or ident.get(slot + "_port") or ""): + missing.append(slot + "_port") + if not (kv.get(slot + "_role_ref") or ""): + missing.append(slot + "_role_ref") + if missing: + print("MISSING=%s" % ",".join(missing), file=sys.stderr) + sys.exit(3) +PY +)" || rc=$? + if [ "$rc" -ne 0 ]; then + c_err "[$ENV_NAME] 存储绑定不完整或无效，拒绝（不静默新建空库/回退默认）：$(printf '%s' "$out" | tail -1)" + return 1 + fi + local line + while IFS= read -r line; do + case "$line" in + MODE=*) STORAGE_MODE="${line#MODE=}" ;; + ENV_CLASS=*) ENV_CLASS="${line#ENV_CLASS=}" ;; + FILES_ROOT=*) FILES_ROOT="${line#FILES_ROOT=}" ;; + COMPOSE_PROJECT=*) COMPOSE_PROJECT="${line#COMPOSE_PROJECT=}" ;; + BASE_DB_REF=*) BASE_DB_REF="${line#BASE_DB_REF=}" ;; + RDM_DB_REF=*) RDM_DB_REF="${line#RDM_DB_REF=}" ;; + KB_DB_REF=*) KB_DB_REF="${line#KB_DB_REF=}" ;; + BASE_PORT=*) BASE_PORT="${line#BASE_PORT=}" ;; + RDM_PORT=*) RDM_PORT="${line#RDM_PORT=}" ;; + KB_PORT=*) KB_PORT="${line#KB_PORT=}" ;; + esac + done <<<"$out" + return 0 +} + # svc_manifest_verify：启动前身份校验——数据 marker 存在且 instance_id 匹配； # 生效 --data / 数据库绑定与冻结锚点一致。失败即拒绝启动（KTD1/KTD2，I4）。 -# 顺序：文件 marker → DB verify（缺/错即拦）。ensure 只在领养（svc_manifest_migrate） -# 与显式 init——普通启动不建身份，缺行即事件（修复路径：svc init，或换具备 -# identity 能力的产物更新时由候选门禁补齐）。 +# 顺序：文件 marker →（container）存储绑定完整/无漂移 → DB verify（缺/错即拦）。 +# ensure 只在领养（svc_manifest_migrate）与显式 init——普通启动不建身份，缺行即事件。 svc_manifest_verify() { manifest_ok || return 0 - local out - if ! out="$(python3 "$MANIFEST_PY" "$ENV_DIR" verify \ - --data "$ENV_DATA" --db-mode "${DB_MODE:-}" --db-ref "${DB_REF:-}" 2>&1)"; then + local out verify_args storage_mode + verify_args=(--data "$ENV_DATA" --db-mode "${DB_MODE:-}" --db-ref "${DB_REF:-}") + storage_mode="$(svc_get storage_mode)" + if [ "$storage_mode" = container ]; then + # 生效值只来自本实例登记（env_config 已剥离调用方污染）；缺绑/错根/错库拒绝。 + resolve_storage_bindings || return 1 + verify_args+=(--require-storage + --files-root "$FILES_ROOT" --env-class "$ENV_CLASS" + --compose-project "$COMPOSE_PROJECT" + --base-db-ref "$BASE_DB_REF" --rdm-db-ref "$RDM_DB_REF" --kb-db-ref "$KB_DB_REF") + python3 "$MANIFEST_PY" "$ENV_DIR" check-conflicts --runtime "$RUNTIME" >/dev/null || { + c_err "[$ENV_NAME] 存储绑定冲突，拒绝启动（Compose 项目名或端口与他实例冲突）" + return 1 + } + fi + if ! out="$(python3 "$MANIFEST_PY" "$ENV_DIR" verify "${verify_args[@]}" 2>&1)"; then c_err "[$ENV_NAME] 身份校验失败，拒绝启动：$(printf '%s' "$out" | tail -1)" - c_err " 数据根/数据库绑定是冻结身份；换根/重绑须显式迁移或登记命令（docs/recovery/u1-identity-manifest.md §5）" + c_err " 数据根/数据库/文件根绑定是冻结身份；换根/重绑须显式迁移或登记命令（docs/recovery/u1-identity-manifest.md §5；容器存储见 deploy/data-ownership.md）" return 1 fi # 文件校验通过后再碰库。fresh 允许库尚未建立；established 只验证； @@ -1704,12 +1790,18 @@ env_config() { # update --all：dev 成功后 genui/rdm1/rdmcand 端口仍被孤儿占用）。 SVC_DB_MODE=""; SVC_DB_SOURCE=""; SVC_DB_REF=""; SVC_DB_DSN_SAVE="" SVC_PIDCHECK=""; SVC_EXEC=""; SVC_VERSION=""; SVC_SOURCE="" + STORAGE_MODE=""; ENV_CLASS=""; FILES_ROOT=""; COMPOSE_PROJECT="" + BASE_DB_REF=""; RDM_DB_REF=""; KB_DB_REF="" + BASE_PORT=""; RDM_PORT=""; KB_PORT="" # U1：身份环境同样按实例重置——上一服务 verify 成功后导出的 # AGENT_SYSTEM_INSTANCE_ID/DBREF 不得串给下一服务（批量 update 单进程逐服务解析； # 2026-10-12 实况：dev 的身份漏给 rdm1/rdmcand，叠加 RDM 产物无 identity 子命令 # 跳过校验，serve 带外宾身份即被基座门禁拒绝）。serve 只允许拿到本服务 verify # 写入的身份。 + # 容器存储 U1：调用方/上一服务的文件根、环境身份、Compose 与三库引用不得污染本实例。 unset AGENT_SYSTEM_INSTANCE_ID AGENT_SYSTEM_DBREF + unset AGENT_SYSTEM_FILES_ROOT AGENT_SYSTEM_ENV_CLASS AGENT_SYSTEM_COMPOSE_PROJECT + unset AGENT_SYSTEM_BASE_DB_REF AGENT_SYSTEM_RDM_DB_REF AGENT_SYSTEM_KB_DB_REF local registered="" [ -f "$ENV_DIR/svc.env" ] && registered="$(svc_kind "$name")" SVC_KIND="${registered:-agent}" @@ -4578,10 +4670,11 @@ check_port_available() { # svc_set_key KEY VALUE：改写当前服务 svc.env 的单键。manifest 治理下经 manifest # 整份提交（原子 + revision）；legacy 路径保持先读后写 + 临时文件原子替换。 +# 存储绑定键变更经 --runtime 做 Compose/端口冲突预检（修改前失败）。 svc_set_key() { local key="$1" val="$2" rest tmp if manifest_ok; then - python3 "$MANIFEST_PY" "$ENV_DIR" set "$key" "$val" && return 0 + python3 "$MANIFEST_PY" "$ENV_DIR" set --runtime "$RUNTIME" "$key" "$val" && return 0 c_warn "[$ENV_NAME] manifest set 失败，回退 legacy 单键写" fi [ -f "$ENV_DIR/svc.env" ] || { c_err "svc.env 不存在：$ENV_DIR"; return 1; } @@ -4598,7 +4691,7 @@ svc_set_key() { svc_del_key() { local key="$1" if manifest_ok; then - python3 "$MANIFEST_PY" "$ENV_DIR" del "$key" && return 0 + python3 "$MANIFEST_PY" "$ENV_DIR" del --runtime "$RUNTIME" "$key" && return 0 c_warn "[$ENV_NAME] manifest del 失败，回退 legacy 单键写" fi [ -f "$ENV_DIR/svc.env" ] || return 0 diff --git a/scripts/svc_manifest.py b/scripts/svc_manifest.py index 8fee1ae1..575026e8 100644 --- a/scripts/svc_manifest.py +++ b/scripts/svc_manifest.py @@ -1,16 +1,19 @@ #!/usr/bin/env python3 """svc 实例身份 manifest（U1）：manifest.json 为启动权威，svc.env 为派生只读视图。 -设计合同：docs/recovery/u1-identity-manifest.md（已批准 2026-10-12）。 +设计合同：docs/recovery/u1-identity-manifest.md（已批准 2026-10-12）； +容器化存储扩展：docs/superpowers/plans/2026-10-22-container-storage-plan.md U1/KTD1—KTD4。 + 落地说明：v1.0 以「identity 锚点（数据根/数据库身份，冻结）+ kv 权威键值映射 + -desired 期望态」承载 KTD1/KTD2 的全部不变量；完整 typed 分区随 U2/U6 演进。 +desired 期望态」承载不变量；本文件在同一权威上扩展三库/文件根/环境身份绑定， +不另造第二份登记权威。完整 typed 分区与容器启停属后续单元。 -用法: svc_manifest.py ENV_DIR [参数] +用法: svc_manifest.py ENV_DIR [参数] -不变量（对应设计 §1）： +不变量（对应设计 §1 + 容器存储 U1）： I3 原子性 —— 整份 temp→fsync→rename→dirsync，任意时刻只有完整旧版或新版。 I4 阻塞 —— manifest 存在时，数据 marker 缺失/身份不符即拒绝，不新建空系统。 - I5 机密 —— 本文件不新增任何机密；kv 沿袭 svc.env 既有内容（DSN 迁出属 U6）。 + I5 机密 —— 凭据只存受保护文件路径引用（*_dsn_ref / *_role_ref），不存明文。 """ import json import os @@ -22,7 +25,27 @@ SCHEMA = 1 MANIFEST = "manifest.json" DERIVED = "svc.env" MARKER_REL = os.path.join(".agentx", "data-marker.json") -PROTECTED = ("db", "dbref", "dsn", "conffile") # 显式持久化写入可重锚；启动期漂移由 verify 拦截 + +# 显式持久化写入可重锚；启动期漂移由 verify 拦截。含容器存储绑定键。 +PROTECTED = ( + "db", "dbref", "dsn", "conffile", + "storage_mode", "env_class", "files_root", "compose_project", + "base_dbref", "base_dsn_ref", "base_role_ref", "base_port", + "rdm_dbref", "rdm_dsn_ref", "rdm_role_ref", "rdm_port", + "kb_dbref", "kb_dsn_ref", "kb_role_ref", "kb_port", +) + +# record_state/bulk-set 重建时须保留的存储绑定键（避免被生命周期写回冲掉）。 +STORAGE_KEYS = ( + "storage_mode", "env_class", "files_root", "compose_project", + "base_dbref", "base_dsn_ref", "base_role_ref", "base_port", + "rdm_dbref", "rdm_dsn_ref", "rdm_role_ref", "rdm_port", + "kb_dbref", "kb_dsn_ref", "kb_role_ref", "kb_port", +) + +ENV_CLASSES = ("prod", "test", "verify") +DB_SLOTS = ("base", "rdm", "kb") +STORAGE_MODES = ("legacy", "container") def die(msg, code=1): @@ -116,6 +139,144 @@ def db_anchor_of(kv): return "", "" +def storage_mode_of(kv): + mode = (kv.get("storage_mode") or "").strip() or "legacy" + if mode not in STORAGE_MODES: + die("storage_mode 仅允许 legacy|container，收到：%s" % mode, 2) + return mode + + +def files_root_of(kv): + """持久文件根：必须已是绝对路径；绝不相对 checkout/worktree 解析。""" + fr = (kv.get("files_root") or "").strip() + if not fr: + return "" + if not os.path.isabs(fr): + die("files_root 必须是绝对路径（不相对 checkout 解析）：%s" % fr, 2) + return os.path.normpath(fr) + + +def slot_ref_of(kv, slot): + """逻辑库引用锚点：优先 dbref，否则 dsn_ref（受保护文件路径）；明文 dsn 禁止。""" + dbref = (kv.get("%s_dbref" % slot) or "").strip() + dsn_ref = (kv.get("%s_dsn_ref" % slot) or "").strip() + if dbref and dsn_ref: + die("%s 绑定冲突：不可同时设置 %s_dbref 与 %s_dsn_ref" % (slot, slot, slot), 2) + if dbref: + return "dbref:%s" % dbref + if dsn_ref: + if not os.path.isabs(dsn_ref): + die("%s_dsn_ref 必须是绝对路径（秘密文件引用）：%s" % (slot, dsn_ref), 2) + return "dsn_ref:%s" % os.path.normpath(dsn_ref) + # 基座可回落既有 dbref/dsn 专键（legacy→container 升级兼容，dsn 明文不进锚点） + if slot == "base": + legacy = (kv.get("dbref") or "").strip() + if legacy: + return "dbref:%s" % legacy + # 明文 dsn= 不是引用；container 模式不得用其充当绑定 + return "" + + +def slot_port_of(kv, slot): + return (kv.get("%s_port" % slot) or "").strip() + + +def slot_role_ref_of(kv, slot): + role = (kv.get("%s_role_ref" % slot) or "").strip() + if not role: + return "" + if not os.path.isabs(role): + die("%s_role_ref 必须是绝对路径（角色秘密文件引用）：%s" % (slot, role), 2) + return os.path.normpath(role) + + +def validate_storage_kv(kv, require_complete=None): + """校验存储绑定形态。container 模式缺任一必需绑定即拒绝（不静默默认/建空库）。""" + mode = storage_mode_of(kv) + if require_complete is None: + require_complete = mode == "container" + env_class = (kv.get("env_class") or "").strip() + if env_class and env_class not in ENV_CLASSES: + die("env_class 仅允许 prod|test|verify，收到：%s" % env_class, 2) + files_root = files_root_of(kv) # 相对路径在此失败 + compose = (kv.get("compose_project") or "").strip() + if compose and (("/" in compose) or (".." in compose) or (" " in compose)): + die("compose_project 非法（禁止路径分隔/空格）：%s" % compose, 2) + for slot in DB_SLOTS: + port = slot_port_of(kv, slot) + if port and not port.isdigit(): + die("%s_port 必须是正整端口号，收到：%s" % (slot, port), 2) + slot_ref_of(kv, slot) # 冲突/相对路径检查 + slot_role_ref_of(kv, slot) + if not require_complete: + return mode + # container 完整合同 + if mode != "container": + die("完整存储合同要求 storage_mode=container", 2) + if env_class not in ENV_CLASSES: + die("container 模式缺少必需绑定 env_class（prod|test|verify）", 3) + if not files_root: + die("container 模式缺少必需绑定 files_root（绝对路径）", 3) + if not compose: + die("container 模式缺少必需绑定 compose_project", 3) + for slot in DB_SLOTS: + ref = slot_ref_of(kv, slot) + port = slot_port_of(kv, slot) + role = slot_role_ref_of(kv, slot) + if not ref: + die("container 模式缺少必需绑定 %s 库（%s_dbref 或 %s_dsn_ref）" + % (slot, slot, slot), 3) + if not port: + die("container 模式缺少必需绑定 %s_port" % slot, 3) + if not role: + die("container 模式缺少必需绑定 %s_role_ref（角色秘密文件路径）" % slot, 3) + return mode + + +def storage_anchor_of(kv): + """从 kv 计算冻结的存储身份锚点（写入 identity；legacy 为空串）。""" + mode = storage_mode_of(kv) + if mode == "legacy" and not any(kv.get(k) for k in STORAGE_KEYS if k != "storage_mode"): + return { + "storage_mode": "legacy", + "env_class": "", + "files_root": "", + "compose_project": "", + "base_db_ref": "", + "base_port": "", + "rdm_db_ref": "", + "rdm_port": "", + "kb_db_ref": "", + "kb_port": "", + } + # 非完整 container 时仍记录已显式写入的锚点（供漂移检测）；完整校验另走 validate + if mode == "container": + validate_storage_kv(kv, require_complete=True) + else: + validate_storage_kv(kv, require_complete=False) + return { + "storage_mode": mode, + "env_class": (kv.get("env_class") or "").strip(), + "files_root": files_root_of(kv), + "compose_project": (kv.get("compose_project") or "").strip(), + "base_db_ref": slot_ref_of(kv, "base"), + "base_port": slot_port_of(kv, "base"), + "rdm_db_ref": slot_ref_of(kv, "rdm"), + "rdm_port": slot_port_of(kv, "rdm"), + "kb_db_ref": slot_ref_of(kv, "kb"), + "kb_port": slot_port_of(kv, "kb"), + } + + +def preserve_storage_keys(new_kv, prev_kv): + """bulk-set/生命周期写回时保留未出现在新写集中的存储绑定键。""" + out = dict(new_kv) + for k in STORAGE_KEYS: + if k not in out and k in prev_kv: + out[k] = prev_kv[k] + return out + + def marker_path(data_root): return os.path.join(data_root, MARKER_REL) @@ -140,13 +301,18 @@ def read_marker(data_root): return {"schema": -1} -def reanchor(env_dir, m): - """按显式持久化写入重算身份锚点；数据根变化时在新根补写 marker（目录存在才写）。""" +def reanchor(env_dir, m, write=True): + """按显式持久化写入重算身份锚点；数据根变化时在新根补写 marker（目录存在才写）。 + + write=False 仅预览身份（冲突扫描用），不落 marker。 + """ ident = dict(m["identity"]) ident["data_root"] = data_root_of(m["kv"], env_dir) ident["db_mode"], ident["db_ref"] = db_anchor_of(m["kv"]) + for k, v in storage_anchor_of(m["kv"]).items(): + ident[k] = v if ident != m["identity"]: - if os.path.isdir(ident["data_root"]): + if write and os.path.isdir(ident["data_root"]): write_marker(ident["data_root"], m["instance_id"]) m["identity"] = ident return m @@ -160,6 +326,9 @@ def build(env_dir, kv, allow_create): if not allow_create: die("数据根不存在且非显式 init，拒绝建立身份（防新建空系统掩盖丢数据）：%s" % data_root, 3) os.makedirs(data_root, exist_ok=True) + # 旧配置升级：若已带 storage 键则校验；纯 legacy 不强制 container 合同 + if storage_mode_of(kv) == "container" or any(kv.get(k) for k in STORAGE_KEYS): + validate_storage_kv(kv, require_complete=(storage_mode_of(kv) == "container")) existing = read_marker(data_root) if existing is not None: if existing.get("instance_id") != kv.get("instance_id"): @@ -170,6 +339,14 @@ def build(env_dir, kv, allow_create): instance_id = kv.get("instance_id") or "svc-%s-%s" % (name, secrets.token_hex(6)) write_marker(data_root, instance_id) db_mode, db_ref = db_anchor_of(kv) + storage = storage_anchor_of(kv) + identity = { + "data_root": data_root, + "db_mode": db_mode, + "db_ref": db_ref, + "db_marker": "fresh", + } + identity.update(storage) m = { "schema": SCHEMA, "revision": 1, @@ -177,18 +354,114 @@ def build(env_dir, kv, allow_create): "service_group": name, "created_at": now_iso(), "desired": "running" if kv.get("autostart") == "1" else "stopped", - "identity": { - "data_root": data_root, - "db_mode": db_mode, - "db_ref": db_ref, - "db_marker": "fresh", - }, + "identity": identity, "kv": {k: v for k, v in kv.items() if k != "autostart"}, } save(env_dir, m) return m +def runtime_dir_of(env_dir): + return os.path.dirname(os.path.abspath(env_dir)) + + +def iter_sibling_manifests(env_dir, runtime_dir=None): + rt = os.path.abspath(runtime_dir) if runtime_dir else runtime_dir_of(env_dir) + self_abs = os.path.abspath(env_dir) + if not os.path.isdir(rt): + return + for name in sorted(os.listdir(rt)): + d = os.path.join(rt, name) + if not os.path.isdir(d) or os.path.abspath(d) == self_abs: + continue + path = os.path.join(d, MANIFEST) + if not os.path.isfile(path): + # 兼容尚未 migrate 的 sibling：读 svc.env 中的存储键 + env_path = os.path.join(d, DERIVED) + if not os.path.isfile(env_path): + continue + kv = {} + with open(env_path, encoding="utf-8") as f: + for line in f: + line = line.rstrip("\n") + if line and "=" in line and not line.startswith("#"): + k, v = line.split("=", 1) + kv[k] = v + yield name, { + "instance_id": kv.get("instance_id", ""), + "identity": storage_anchor_of(kv) if any(kv.get(k) for k in STORAGE_KEYS) else {}, + "kv": kv, + } + continue + try: + with open(path, encoding="utf-8") as f: + yield name, json.load(f) + except Exception: # noqa: BLE001 —— 损坏 sibling 不阻断本实例冲突扫描语义以外的操作 + continue + + +def binding_ports_from(m_or_kv, is_manifest=True): + if is_manifest: + kv = m_or_kv.get("kv") or {} + ident = m_or_kv.get("identity") or {} + else: + kv = m_or_kv + ident = {} + ports = set() + for slot in DB_SLOTS: + p = (kv.get("%s_port" % slot) or "").strip() or (ident.get("%s_port" % slot) or "").strip() + if p: + ports.add(p) + return ports + + +def compose_of(m_or_kv, is_manifest=True): + if is_manifest: + kv = m_or_kv.get("kv") or {} + ident = m_or_kv.get("identity") or {} + return (kv.get("compose_project") or ident.get("compose_project") or "").strip() + return (m_or_kv.get("compose_project") or "").strip() + + +def check_conflicts(env_dir, m, runtime_dir=None): + """同一 runtime 下 Compose 项目名或存储端口冲突 → 修改前失败。""" + kv = m["kv"] + ident = m.get("identity") or {} + compose = (kv.get("compose_project") or ident.get("compose_project") or "").strip() + ports = binding_ports_from(m, is_manifest=True) + if not compose and not ports: + return + self_name = os.path.basename(os.path.abspath(env_dir)) + for name, other in iter_sibling_manifests(env_dir, runtime_dir): + other_compose = compose_of(other, is_manifest=True) + if compose and other_compose and compose == other_compose: + die("Compose 项目名冲突：compose_project=%s 已被实例 %s 占用（修改前拒绝）" + % (compose, name), 7) + other_ports = binding_ports_from(other, is_manifest=True) + clash = ports & other_ports + if clash: + die("存储端口冲突：port=%s 与实例 %s 冲突（修改前拒绝）" + % (",".join(sorted(clash)), name), 7) + # 亦与 sibling 的应用监听 addr 端口冲突（避免与已登记服务抢口） + if ports: + rt = os.path.abspath(runtime_dir) if runtime_dir else runtime_dir_of(env_dir) + for name in sorted(os.listdir(rt)) if os.path.isdir(rt) else []: + if name == self_name: + continue + env_path = os.path.join(rt, name, DERIVED) + if not os.path.isfile(env_path): + continue + with open(env_path, encoding="utf-8") as f: + for line in f: + if line.startswith("addr="): + addr = line[5:].strip() + ap = addr.rsplit(":", 1)[-1] if addr else "" + if ap in ports: + die("存储端口冲突：port=%s 与服务 %s 的 addr=%s 冲突（修改前拒绝）" + % (ap, name, addr), 7) + break + + def cmd_migrate(env_dir): if load(env_dir) is not None: print("manifest 已存在，幂等跳过") @@ -221,32 +494,41 @@ def cmd_status(env_dir): return 0 -def mutate(env_dir, fn): +def mutate(env_dir, fn, runtime_dir=None): m = load(env_dir) if m is None: die("manifest 不存在：先 migrate/init", 3) before = json.dumps(m["identity"], sort_keys=True) fn(m) + # 存储绑定变更：先形态校验，再冲突扫描，最后重锚——冲突时不落盘 + if any(k in m["kv"] for k in STORAGE_KEYS) or storage_mode_of(m["kv"]) == "container": + validate_storage_kv(m["kv"], require_complete=(storage_mode_of(m["kv"]) == "container")) + # 预览重锚后的 identity 以做冲突检测（不落 marker / 不写盘） + preview = json.loads(json.dumps(m)) + preview = reanchor(env_dir, preview, write=False) + check_conflicts(env_dir, preview, runtime_dir=runtime_dir) m["revision"] = int(m["revision"]) + 1 - m = reanchor(env_dir, m) + m = reanchor(env_dir, m, write=True) save(env_dir, m) if before != json.dumps(m["identity"], sort_keys=True): - print("[manifest] 身份锚点已随显式持久化写入更新（data=%s db=%s/%s）" - % (m["identity"]["data_root"], m["identity"]["db_mode"], m["identity"]["db_ref"])) + print("[manifest] 身份锚点已随显式持久化写入更新（data=%s db=%s/%s storage=%s files=%s）" + % (m["identity"]["data_root"], m["identity"]["db_mode"], m["identity"]["db_ref"], + m["identity"].get("storage_mode", "legacy"), + m["identity"].get("files_root") or "-")) return 0 -def cmd_set(env_dir, key, value): - return mutate(env_dir, lambda m: m["kv"].__setitem__(key, value)) +def cmd_set(env_dir, key, value, runtime_dir=None): + return mutate(env_dir, lambda m: m["kv"].__setitem__(key, value), runtime_dir=runtime_dir) -def cmd_del(env_dir, key): +def cmd_del(env_dir, key, runtime_dir=None): def drop(m): m["kv"].pop(key, None) - return mutate(env_dir, drop) + return mutate(env_dir, drop, runtime_dir=runtime_dir) -def cmd_bulk_set(env_dir): +def cmd_bulk_set(env_dir, runtime_dir=None): kv = {} for line in sys.stdin.read().splitlines(): if line and "=" in line and not line.startswith("#"): @@ -255,8 +537,8 @@ def cmd_bulk_set(env_dir): kv.pop("autostart", None) # desired 独占期望态 def replace(m): - m["kv"] = kv - return mutate(env_dir, replace) + m["kv"] = preserve_storage_keys(kv, m["kv"]) + return mutate(env_dir, replace, runtime_dir=runtime_dir) def cmd_set_desired(env_dir, state): @@ -265,7 +547,20 @@ def cmd_set_desired(env_dir, state): return mutate(env_dir, lambda m: m.__setitem__("desired", state)) -def cmd_verify(env_dir, data_root=None, db_mode=None, db_ref=None): +def cmd_check_conflicts(env_dir, runtime_dir=None): + m = load(env_dir) + if m is None: + die("manifest 不存在：先 migrate/init", 3) + check_conflicts(env_dir, m, runtime_dir=runtime_dir) + print("绑定冲突检查通过 instance_id=%s compose=%s" + % (m["instance_id"], m["identity"].get("compose_project") or "-")) + return 0 + + +def cmd_verify(env_dir, data_root=None, db_mode=None, db_ref=None, + files_root=None, env_class=None, compose_project=None, + base_db_ref=None, rdm_db_ref=None, kb_db_ref=None, + require_storage=False): m = load(env_dir) if m is None: die("manifest 缺失：本实例已进入 manifest 治理，缺失即事件（先 init/人工处置，禁止静默重建）", 4) @@ -284,14 +579,51 @@ def cmd_verify(env_dir, data_root=None, db_mode=None, db_ref=None): die("数据库模式漂移：生效 %s ≠ 锚定 %s" % (db_mode, ident["db_mode"]), 6) if db_mode and ident["db_mode"] and db_ref is not None and (db_ref or "") != ident["db_ref"]: die("数据库绑定漂移：生效 %s ≠ 锚定 %s（重绑须经显式登记命令）" % (db_ref or "<空>", ident["db_ref"]), 6) - print("身份校验通过 instance_id=%s data=%s db=%s/%s" - % (m["instance_id"], ident["data_root"], ident["db_mode"] or "-", ident["db_ref"] or "-")) + + storage_mode = ident.get("storage_mode") or "legacy" + if storage_mode == "container" or require_storage: + # container：缺绑定/错根/错库一律拒绝，不回退默认、不建空库 + validate_storage_kv(m["kv"], require_complete=True) + if files_root is not None: + got = os.path.normpath(files_root) if os.path.isabs(files_root) else files_root + if not os.path.isabs(files_root): + die("文件根拒绝相对路径（不相对 checkout 解析）：%s" % files_root, 6) + if got != (ident.get("files_root") or ""): + die("文件根漂移：生效 %s ≠ 锚定 %s" % (got, ident.get("files_root")), 6) + if env_class is not None and env_class != (ident.get("env_class") or ""): + die("环境身份漂移：生效 %s ≠ 锚定 %s" % (env_class, ident.get("env_class")), 6) + if compose_project is not None and compose_project != (ident.get("compose_project") or ""): + die("Compose 项目漂移：生效 %s ≠ 锚定 %s" % (compose_project, ident.get("compose_project")), 6) + for slot, got in (("base", base_db_ref), ("rdm", rdm_db_ref), ("kb", kb_db_ref)): + if got is None: + continue + want = ident.get("%s_db_ref" % slot) or "" + if (got or "") != want: + die("%s 库绑定漂移：生效 %s ≠ 锚定 %s（错库拒绝，不静默新建）" + % (slot, got or "<空>", want or "<空>"), 6) + + print("身份校验通过 instance_id=%s data=%s db=%s/%s storage=%s env=%s files=%s" + % (m["instance_id"], ident["data_root"], ident["db_mode"] or "-", ident["db_ref"] or "-", + storage_mode, ident.get("env_class") or "-", ident.get("files_root") or "-")) return 0 +def _parse_runtime_flag(args): + runtime_dir = None + out = [] + it = iter(args) + for a in it: + if a == "--runtime": + runtime_dir = next(it, None) + else: + out.append(a) + return runtime_dir, out + + def main(argv): if len(argv) < 3: - die("用法: svc_manifest.py ENV_DIR [参数]", 2) + die("用法: svc_manifest.py ENV_DIR [参数]", 2) env_dir, cmd = argv[1], argv[2] args = argv[3:] if cmd == "migrate": @@ -300,16 +632,27 @@ def main(argv): return cmd_init(env_dir) if cmd == "status": return cmd_status(env_dir) - if cmd == "set" and len(args) == 2: - return cmd_set(env_dir, args[0], args[1]) - if cmd == "del" and len(args) == 1: - return cmd_del(env_dir, args[0]) + if cmd == "set": + runtime_dir, args = _parse_runtime_flag(args) + if len(args) == 2: + return cmd_set(env_dir, args[0], args[1], runtime_dir=runtime_dir) + if cmd == "del": + runtime_dir, args = _parse_runtime_flag(args) + if len(args) == 1: + return cmd_del(env_dir, args[0], runtime_dir=runtime_dir) if cmd == "bulk-set": - return cmd_bulk_set(env_dir) + runtime_dir, args = _parse_runtime_flag(args) + return cmd_bulk_set(env_dir, runtime_dir=runtime_dir) if cmd == "set-desired" and len(args) == 1: return cmd_set_desired(env_dir, args[0]) + if cmd == "check-conflicts": + runtime_dir, _ = _parse_runtime_flag(args) + return cmd_check_conflicts(env_dir, runtime_dir=runtime_dir) if cmd == "verify": data_root = db_mode = db_ref = None + files_root = env_class = compose_project = None + base_db_ref = rdm_db_ref = kb_db_ref = None + require_storage = False it = iter(args) for a in it: if a == "--data": @@ -318,7 +661,27 @@ def main(argv): db_mode = next(it, None) elif a == "--db-ref": db_ref = next(it, None) - return cmd_verify(env_dir, data_root, db_mode, db_ref) + elif a == "--files-root": + files_root = next(it, None) + elif a == "--env-class": + env_class = next(it, None) + elif a == "--compose-project": + compose_project = next(it, None) + elif a == "--base-db-ref": + base_db_ref = next(it, None) + elif a == "--rdm-db-ref": + rdm_db_ref = next(it, None) + elif a == "--kb-db-ref": + kb_db_ref = next(it, None) + elif a == "--require-storage": + require_storage = True + return cmd_verify( + env_dir, data_root, db_mode, db_ref, + files_root=files_root, env_class=env_class, + compose_project=compose_project, + base_db_ref=base_db_ref, rdm_db_ref=rdm_db_ref, kb_db_ref=kb_db_ref, + require_storage=require_storage, + ) die("未知命令或参数不全：%s %s" % (cmd, " ".join(args)), 2) diff --git a/tests/test_svc_identity.py b/tests/test_svc_identity.py index d8466bb3..7db8646e 100644 --- a/tests/test_svc_identity.py +++ b/tests/test_svc_identity.py @@ -500,6 +500,199 @@ class IdentityTests(tsr.SvcAutoRecoverTests): self.skipTest("真 PG 专项应由隔离套件执行（本桩文件不连真实 PG）") self.skipTest("真实 PG 身份专项 NOT_RUN：需真实实例（SVC_TEST_PG=1 + AGENT_PG_TEST_DSN）") + # ---- 容器存储 U1：环境与数据归属合同 ---- + + def _storage_secret_dir(self): + d = self.root / "storage-secrets" + d.mkdir(exist_ok=True) + for slot in ("base", "rdm", "kb"): + (d / ("%s.dsn" % slot)).write_text("ref-only\n", encoding="utf-8") + (d / ("%s.role" % slot)).write_text("ref-only\n", encoding="utf-8") + return d + + def _apply_container_bindings(self, name, compose, ports, files_root, env_class="prod"): + """先写齐绑定键，最后打开 storage_mode=container（缺一即拒）。 + 经 svc_manifest.py set（与 svc_set_key 同源权威）。""" + sec = self._storage_secret_dir() + items = [ + ("env_class", env_class), + ("files_root", str(files_root)), + ("compose_project", compose), + ("base_dsn_ref", str(sec / "base.dsn")), + ("base_role_ref", str(sec / "base.role")), + ("base_port", ports[0]), + ("rdm_dsn_ref", str(sec / "rdm.dsn")), + ("rdm_role_ref", str(sec / "rdm.role")), + ("rdm_port", ports[1]), + ("kb_dsn_ref", str(sec / "kb.dsn")), + ("kb_role_ref", str(sec / "kb.role")), + ("kb_port", ports[2]), + ("storage_mode", "container"), + ] + man = self.root / "scripts" / "svc_manifest.py" + env_dir = self.runtime(name) + runtime = env_dir.parent + for k, v in items: + proc = subprocess.run( + ["python3", str(man), str(env_dir), "set", "--runtime", str(runtime), k, str(v)], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertEqual(proc.returncode, 0, "%s=%s → %s" % (k, v, proc.stdout)) + + def test_CS_U1_legacy_upgrade_keeps_identity(self): + """旧配置升级到 container 绑定：instance_id / data_root 保留。""" + self.seed_agent("cs1", 9471) + self.start_ok("cs1") + m0 = self.manifest("cs1") + iid, data = m0["instance_id"], m0["identity"]["data_root"] + self.svc("cs1", "stop") + files = self.root / "files-cs1" + files.mkdir() + self._apply_container_bindings("cs1", "svc-cs1", ("25432", "25433", "25434"), files) + m1 = self.manifest("cs1") + self.assertEqual(m1["instance_id"], iid) + self.assertEqual(m1["identity"]["data_root"], data) + self.assertEqual(m1["identity"]["storage_mode"], "container") + self.assertEqual(m1["identity"]["env_class"], "prod") + self.assertEqual(m1["identity"]["files_root"], str(files)) + self.assertTrue(m1["identity"]["base_db_ref"].startswith("dsn_ref:")) + self.assertTrue(m1["identity"]["rdm_db_ref"].startswith("dsn_ref:")) + self.assertTrue(m1["identity"]["kb_db_ref"].startswith("dsn_ref:")) + + def test_CS_U1_relative_files_root_rejected(self): + self.seed_agent("cs2", 9472) + self.start_ok("cs2") + self.svc("cs2", "stop") + man = self.root / "scripts" / "svc_manifest.py" + env_dir = self.runtime("cs2") + proc = subprocess.run( + ["python3", str(man), str(env_dir), "set", "--runtime", str(env_dir.parent), + "files_root", "relative/not-abs"], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertNotEqual(proc.returncode, 0, proc.stdout) + self.assertIn("绝对路径", proc.stdout) + self.assertNotEqual(self.manifest("cs2")["identity"].get("files_root"), + "relative/not-abs") + + def test_CS_U1_missing_binding_rejected(self): + """container 模式缺任一必需绑定不得开启（不静默默认）。""" + self.seed_agent("cs3", 9473) + self.start_ok("cs3") + self.svc("cs3", "stop") + man = self.root / "scripts" / "svc_manifest.py" + env_dir = self.runtime("cs3") + rt = str(env_dir.parent) + proc = subprocess.run( + ["python3", str(man), str(env_dir), "set", "--runtime", rt, + "storage_mode", "container"], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertNotEqual(proc.returncode, 0, proc.stdout) + self.assertIn("缺少必需绑定", proc.stdout) + self.assertNotEqual(self.manifest("cs3")["kv"].get("storage_mode"), "container") + + def test_CS_U1_wrong_files_root_and_db_verify_rejected(self): + self.seed_agent("cs4", 9474) + self.start_ok("cs4") + self.svc("cs4", "stop") + files = self.root / "files-cs4" + files.mkdir() + self._apply_container_bindings("cs4", "svc-cs4", ("25532", "25533", "25534"), files) + man = self.root / "scripts" / "svc_manifest.py" + env_dir = self.runtime("cs4") + bad_files = subprocess.run( + ["python3", str(man), str(env_dir), "verify", "--require-storage", + "--files-root", str(self.root / "other-files")], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertNotEqual(bad_files.returncode, 0, bad_files.stdout) + self.assertIn("文件根", bad_files.stdout) + bad_db = subprocess.run( + ["python3", str(man), str(env_dir), "verify", "--require-storage", + "--files-root", str(files), + "--base-db-ref", "dbref:foreign/x"], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertNotEqual(bad_db.returncode, 0, bad_db.stdout) + self.assertIn("base 库绑定漂移", bad_db.stdout) + + def test_CS_U1_compose_and_port_conflict_before_modify(self): + self.seed_agent("cs5a", 9475) + self.seed_agent("cs5b", 9476) + self.start_ok("cs5a") + self.start_ok("cs5b") + self.svc("cs5a", "stop") + self.svc("cs5b", "stop") + files_a = self.root / "files-cs5a" + files_b = self.root / "files-cs5b" + files_a.mkdir() + files_b.mkdir() + self._apply_container_bindings("cs5a", "svc-shared", ("25632", "25633", "25634"), files_a) + man = self.root / "scripts" / "svc_manifest.py" + env_b = self.runtime("cs5b") + rt = str(env_b.parent) + # 同 compose 名：修改前失败 + sec = self._storage_secret_dir() + for k, v in (("env_class", "test"), ("files_root", str(files_b)), + ("compose_project", "svc-shared")): + proc = subprocess.run( + ["python3", str(man), str(env_b), "set", "--runtime", rt, k, v], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + if k == "compose_project": + self.assertNotEqual(proc.returncode, 0, proc.stdout) + self.assertIn("Compose 项目名冲突", proc.stdout) + else: + self.assertEqual(proc.returncode, 0, proc.stdout) + # 端口冲突 + self.seed_agent("cs5c", 9477) + self.start_ok("cs5c") + self.svc("cs5c", "stop") + files_c = self.root / "files-cs5c" + files_c.mkdir() + env_c = self.runtime("cs5c") + for k, v in ( + ("env_class", "verify"), + ("files_root", str(files_c)), + ("compose_project", "svc-cs5c"), + ("base_dsn_ref", str(sec / "base.dsn")), + ("base_role_ref", str(sec / "base.role")), + ("base_port", "25632"), + ): + proc = subprocess.run( + ["python3", str(man), str(env_c), "set", "--runtime", rt, k, str(v)], + cwd=str(self.root), env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + if k == "base_port": + self.assertNotEqual(proc.returncode, 0, proc.stdout) + self.assertIn("端口冲突", proc.stdout) + else: + self.assertEqual(proc.returncode, 0, proc.stdout) + + def test_CS_U1_other_checkout_keeps_absolute_files_root(self): + """另一 cwd/checkout 调用仍定位同一绝对 files_root。""" + self.seed_agent("cs6", 9478) + self.start_ok("cs6") + self.svc("cs6", "stop") + files = self.root / "files-cs6" + files.mkdir() + self._apply_container_bindings("cs6", "svc-cs6", ("25732", "25733", "25734"), files) + iid = self.manifest("cs6")["instance_id"] + fr = self.manifest("cs6")["identity"]["files_root"] + elsewhere = tempfile.mkdtemp(prefix="svc-cs-elsewhere-") + self.addCleanup(shutil.rmtree, elsewhere, True) + man = self.root / "scripts" / "svc_manifest.py" + proc = subprocess.run( + ["python3", str(man), str(self.runtime("cs6")), "status"], + cwd=elsewhere, env=self.env, text=True, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=30) + self.assertEqual(proc.returncode, 0, proc.stdout) + payload = json.loads(proc.stdout.strip()) + self.assertEqual(payload["instance_id"], iid) + self.assertEqual(payload["identity"]["files_root"], fr) + self.assertTrue(os.path.isabs(fr), fr) + class AtomicWriteTests(unittest.TestCase): """崩溃窗口单元级：os.replace 失败时原文件必须完好（I3 半写不可能）。""" diff --git a/tests/test_svc_lifecycle_isolation.py b/tests/test_svc_lifecycle_isolation.py index f440c66f..c2110f2a 100644 --- a/tests/test_svc_lifecycle_isolation.py +++ b/tests/test_svc_lifecycle_isolation.py @@ -52,3 +52,41 @@ class LifecycleIsolationTests(tsr.SvcAutoRecoverTests): self.assertEqual(again["identity"]["data_root"], data) self.assertEqual(again["identity"].get("db_marker"), "established") self.svc("iso2", "stop") + + def test_caller_storage_env_does_not_reach_serve(self): + """调用方 AGENT_SYSTEM_FILES_ROOT / ENV_CLASS / COMPOSE 不得进入 serve。""" + self.seed_agent("iso3", 9463) + env = dict(self.env) + env["AGENT_SYSTEM_FILES_ROOT"] = "/tmp/leak-files-root" + env["AGENT_SYSTEM_ENV_CLASS"] = "prod" + env["AGENT_SYSTEM_COMPOSE_PROJECT"] = "leak-compose" + env["AGENT_SYSTEM_BASE_DB_REF"] = "dbref:leak/base" + env["AGENT_SYSTEM_RDM_DB_REF"] = "dbref:leak/rdm" + env["AGENT_SYSTEM_KB_DB_REF"] = "dbref:leak/kb" + self.svc("iso3", "start", env=env) + pid = self.pid_of("iso3") + self.assertIsNotNone(pid) + raw = Path("/proc/%d/environ" % pid).read_bytes() + self.assertNotIn(b"/tmp/leak-files-root", raw) + self.assertNotIn(b"leak-compose", raw) + self.assertNotIn(b"dbref:leak/base", raw) + self.assertNotIn(b"dbref:leak/rdm", raw) + self.assertNotIn(b"dbref:leak/kb", raw) + serve_env = self.runtime("iso3") / "serve.env" + self.assertTrue(serve_env.is_file()) + text = serve_env.read_text() + self.assertNotIn("AGENT_SYSTEM_FILES_ROOT", text) + self.assertNotIn("leak-files-root", text) + self.assertNotIn("leak-compose", text) + self.svc("iso3", "stop") + + def test_env_config_resets_storage_globals(self): + """棘轮：env_config 必须清空存储绑定全局与调用方身份环境。""" + import subprocess + out = subprocess.run( + ["bash", "-c", + 'source "$1"; declare -f env_config | grep -c \'AGENT_SYSTEM_FILES_ROOT\'', + "x", str(SOURCE / "scripts" / "svc.sh")], + capture_output=True, text=True, cwd=str(SOURCE), timeout=30) + self.assertEqual(out.returncode, 0, out.stderr) + self.assertGreaterEqual(int(out.stdout.strip()), 1) diff --git a/tests/test_svc_precheck.py b/tests/test_svc_precheck.py index bdfe4559..91eb9861 100644 --- a/tests/test_svc_precheck.py +++ b/tests/test_svc_precheck.py @@ -246,6 +246,198 @@ class ResolverStateIsolation(unittest.TestCase): ) self.assertEqual(out.strip(), "1") + def test_env_config_resets_storage_identity_env(self): + # 棘轮：env_config 必须剥离调用方 FILES_ROOT / ENV_CLASS / COMPOSE（容器存储 U1）。 + out = _bash( + 'source "$1"; declare -f env_config | ' + 'grep -c \'unset AGENT_SYSTEM_FILES_ROOT AGENT_SYSTEM_ENV_CLASS AGENT_SYSTEM_COMPOSE_PROJECT\'' + ) + self.assertEqual(out.strip(), "1") + + +class StorageBindingsPrecheck(unittest.TestCase): + """容器存储 U1：resolve_storage_bindings 与冲突预检（不启真实服务）。""" + + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.rt = Path(self._tmp.name) / "runtime" + self.rt.mkdir() + self.sec = Path(self._tmp.name) / "secrets" + self.sec.mkdir() + self.files = Path(self._tmp.name) / "files" + self.files.mkdir() + for slot in ("base", "rdm", "kb"): + (self.sec / ("%s.dsn" % slot)).write_text("x\n") + (self.sec / ("%s.role" % slot)).write_text("y\n") + # 复制 manifest 脚本到可读路径（与仓内一致） + self.man = REPO / "scripts" / "svc_manifest.py" + + def tearDown(self): + self._tmp.cleanup() + + def _init_agent(self, name): + env = self.rt / name + data = env / "data" + data.mkdir(parents=True) + (env / "svc.env").write_text( + "kind=agent\nenv=%s\naddr=127.0.0.1:18080\n" + "exec=--addr 127.0.0.1:18080 --data %s\n" % (name, data), + encoding="utf-8", + ) + proc = subprocess.run( + ["python3", str(self.man), str(env), "migrate"], + capture_output=True, text=True, cwd=str(REPO), timeout=30) + self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr) + return env + + def _set(self, env, key, value): + proc = subprocess.run( + ["python3", str(self.man), str(env), "set", + "--runtime", str(self.rt), key, str(value)], + capture_output=True, text=True, cwd=str(REPO), timeout=30) + return proc + + def test_resolve_storage_bindings_legacy_ok(self): + env = self._init_agent("leg") + out = _bash( + 'source "$1"; shift; RUNTIME="$1"; ENV_NAME=leg; ENV_DIR="$2"; ' + 'if resolve_storage_bindings; then echo rc=ok; else echo rc=fail; fi; ' + 'printf "MODE=%s\\n" "$STORAGE_MODE"', + str(self.rt), str(env), + ) + self.assertIn("rc=ok", out) + self.assertIn("MODE=legacy", out) + + def test_resolve_incomplete_container_refuses(self): + env = self._init_agent("inc") + # 只写 storage_mode 会被 manifest 拒绝；直接改 kv 模拟半套后 resolve 拒绝 + proc = self._set(env, "env_class", "prod") + self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr) + # 手工把 storage_mode 写成 container（绕过 mutate 完整校验）以测 resolve + import json + mp = env / "manifest.json" + m = json.loads(mp.read_text(encoding="utf-8")) + m["kv"]["storage_mode"] = "container" + m["identity"]["storage_mode"] = "container" + mp.write_text(json.dumps(m, ensure_ascii=False, indent=2) + "\n", encoding="utf-8") + (env / "svc.env").write_text( + (env / "svc.env").read_text(encoding="utf-8") + "storage_mode=container\n", + encoding="utf-8", + ) + try: + _bash( + 'source "$1"; shift; RUNTIME="$1"; ENV_NAME=inc; ENV_DIR="$2"; ' + 'resolve_storage_bindings', + str(self.rt), str(env), + ) + self.fail("incomplete container must refuse") + except AssertionError as e: + self.assertIn("存储绑定不完整", str(e)) + + def test_relative_files_root_refused_on_set(self): + env = self._init_agent("rel") + proc = self._set(env, "files_root", "not/absolute") + self.assertNotEqual(proc.returncode, 0) + self.assertIn("绝对路径", proc.stdout + proc.stderr) + + def test_compose_project_conflict_refused(self): + a = self._init_agent("ca") + b = self._init_agent("cb") + for k, v in ( + ("env_class", "prod"), + ("files_root", str(self.files)), + ("compose_project", "same-proj"), + ("base_dsn_ref", str(self.sec / "base.dsn")), + ("base_role_ref", str(self.sec / "base.role")), + ("base_port", "26001"), + ("rdm_dsn_ref", str(self.sec / "rdm.dsn")), + ("rdm_role_ref", str(self.sec / "rdm.role")), + ("rdm_port", "26002"), + ("kb_dsn_ref", str(self.sec / "kb.dsn")), + ("kb_role_ref", str(self.sec / "kb.role")), + ("kb_port", "26003"), + ("storage_mode", "container"), + ): + proc = self._set(a, k, v) + self.assertEqual(proc.returncode, 0, "%s: %s" % (k, proc.stdout + proc.stderr)) + (self.files / "b").mkdir(exist_ok=True) + proc = self._set(b, "env_class", "test") + self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr) + proc = self._set(b, "files_root", str(self.files / "b")) + self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr) + proc = self._set(b, "compose_project", "same-proj") + self.assertNotEqual(proc.returncode, 0) + self.assertIn("Compose 项目名冲突", proc.stdout + proc.stderr) + + def test_storage_port_conflict_refused(self): + a = self._init_agent("pa") + b = self._init_agent("pb") + for k, v in ( + ("env_class", "prod"), + ("files_root", str(self.files / "pa")), + ("compose_project", "proj-pa"), + ("base_dsn_ref", str(self.sec / "base.dsn")), + ("base_role_ref", str(self.sec / "base.role")), + ("base_port", "26101"), + ("rdm_dsn_ref", str(self.sec / "rdm.dsn")), + ("rdm_role_ref", str(self.sec / "rdm.role")), + ("rdm_port", "26102"), + ("kb_dsn_ref", str(self.sec / "kb.dsn")), + ("kb_role_ref", str(self.sec / "kb.role")), + ("kb_port", "26103"), + ("storage_mode", "container"), + ): + (self.files / "pa").mkdir(exist_ok=True) + proc = self._set(a, k, v) + self.assertEqual(proc.returncode, 0, "%s: %s" % (k, proc.stdout + proc.stderr)) + (self.files / "pb").mkdir(exist_ok=True) + for k, v in ( + ("env_class", "test"), + ("files_root", str(self.files / "pb")), + ("compose_project", "proj-pb"), + ("base_dsn_ref", str(self.sec / "base.dsn")), + ("base_role_ref", str(self.sec / "base.role")), + ("base_port", "26101"), + ): + proc = self._set(b, k, v) + if k == "base_port": + self.assertNotEqual(proc.returncode, 0, proc.stdout + proc.stderr) + self.assertIn("端口冲突", proc.stdout + proc.stderr) + else: + self.assertEqual(proc.returncode, 0, proc.stdout + proc.stderr) + + def test_resolve_complete_container_ok(self): + env = self._init_agent("okc") + (self.files / "okc").mkdir(exist_ok=True) + for k, v in ( + ("env_class", "verify"), + ("files_root", str(self.files / "okc")), + ("compose_project", "proj-okc"), + ("base_dsn_ref", str(self.sec / "base.dsn")), + ("base_role_ref", str(self.sec / "base.role")), + ("base_port", "26201"), + ("rdm_dsn_ref", str(self.sec / "rdm.dsn")), + ("rdm_role_ref", str(self.sec / "rdm.role")), + ("rdm_port", "26202"), + ("kb_dsn_ref", str(self.sec / "kb.dsn")), + ("kb_role_ref", str(self.sec / "kb.role")), + ("kb_port", "26203"), + ("storage_mode", "container"), + ): + proc = self._set(env, k, v) + self.assertEqual(proc.returncode, 0, "%s: %s" % (k, proc.stdout + proc.stderr)) + out = _bash( + 'source "$1"; shift; RUNTIME="$1"; ENV_NAME=okc; ENV_DIR="$2"; ' + 'if resolve_storage_bindings; then echo rc=ok; else echo rc=fail; fi; ' + 'printf "MODE=%s\\nENV=%s\\nFILES=%s\\nCOMPOSE=%s\\n" ' + '"$STORAGE_MODE" "$ENV_CLASS" "$FILES_ROOT" "$COMPOSE_PROJECT"', + str(self.rt), str(env), + ) + self.assertIn("rc=ok", out) + self.assertIn("MODE=container", out) + self.assertIn("ENV=verify", out) + self.assertIn("COMPOSE=proj-okc", out) + if __name__ == "__main__": unittest.main()