diff --git a/.cursor/rules/agentctl-mandatory.mdc b/.cursor/rules/agentctl-mandatory.mdc index 18124b69..ce9dcea3 100644 --- a/.cursor/rules/agentctl-mandatory.mdc +++ b/.cursor/rules/agentctl-mandatory.mdc @@ -13,6 +13,7 @@ alwaysApply: true 2. **共享文档**：`.doc/ops/CURRENT.md`、`.doc/status/FEATURE-MATRIX.md` 只能 `docs apply`（行级 CAS；MATRIX 键为 `record_id`）。 3. **合入 local dev**：`validate` 后 `queue enqueue`（daemon `--ff-only`）或受控 `integrate`；禁止直接改主目录 `dev` tip。 4. **禁止默认**：未授权 `git push`、晋升 test/main、`--force` 清理、卸载 launchd coordinator、改 gap-matrix / 升 MATRIX 状态值冒充发布。 +5. **需求重叠**：跨对话新设计须 `catalog overlap`；硬重叠拒绝；软重叠须 `catalog ack` 留痕后方可编码（见 `.doc/ops/2026-09-24-governance-catalog-plan.md`）。 ## 会话路径 diff --git a/.cursor/skills/agentctl-session-start/SKILL.md b/.cursor/skills/agentctl-session-start/SKILL.md index be29bc9a..7eb60afc 100644 --- a/.cursor/skills/agentctl-session-start/SKILL.md +++ b/.cursor/skills/agentctl-session-start/SKILL.md @@ -29,11 +29,23 @@ codegraph sync -q 跨文件定位优先用 CodeGraph MCP/`explore`；**grep 仍是完整性唯一权威**（假阴性时以 grep 为准）。 4. 若任务已存在：在注册 worktree 内工作，确认 `tasks.state=active`。 -5. 若新任务：跟 [agentctl-worktree-setup](../agentctl-worktree-setup/SKILL.md)。 -6. 任何仓库写 → [agentctl-lease-write](../agentctl-lease-write/SKILL.md)；合入 → [agentctl-integrate-closeout](../agentctl-integrate-closeout/SKILL.md)。 +5. **开工前需求重叠探测**（治理目录库）：新设计或可能与其他对话重叠时： + +```bash +python3 scripts/agentctl.py catalog search --query '<关键词>' +python3 scripts/agentctl.py catalog overlap --task "$ID" --statement '<设计原文>' [--requirement R-xxx] [--feature ] [--mutex ] +# soft 命中须显式裁决后才可编码： +python3 scripts/agentctl.py catalog overlap ack --task "$ID" --overlap --decision proceed|merge_into|supersede|defer --reason '...' +``` + +硬重叠（同 mutex/feature/scope/R-ID）直接拒绝；向量层失败自动降级到 FTS5，不阻断。 + +6. 若新任务：跟 [agentctl-worktree-setup](../agentctl-worktree-setup/SKILL.md)。 +7. 任何仓库写 → [agentctl-lease-write](../agentctl-lease-write/SKILL.md)；合入 → [agentctl-integrate-closeout](../agentctl-integrate-closeout/SKILL.md)。 ## 禁止 - 主目录直接开发提交；未授权 `git push` / 晋升 test/main - 绕过 `docs apply` 改 CURRENT / FEATURE-MATRIX - 卸载或重启 launchd coordinator（除非用户明示） +- 未执行 `catalog overlap` / 未裁决 soft 命中即编码（有 `--requirement`/`--intent`/`--feature`/`--mutex` 绑定时机器强制） diff --git a/.cursor/skills/agentctl-worktree-setup/SKILL.md b/.cursor/skills/agentctl-worktree-setup/SKILL.md index c33c8c94..622a9c0f 100644 --- a/.cursor/skills/agentctl-worktree-setup/SKILL.md +++ b/.cursor/skills/agentctl-worktree-setup/SKILL.md @@ -23,9 +23,25 @@ python3 ../../scripts/agentctl.py task register \ --scope scripts --scope .doc \ --capability docs:publish # 需要整文/行级 admin 时再加 --capability docs:admin；证据升 done 加 docs:approve +# 若已知需求/功能绑定，追加： +# --requirement R- --intent '<设计原文>' --feature --mutex +# 有 soft 重叠时 register 会 OVERLAP_ACK_REQUIRED：先 catalog overlap + ack 再重试 register codegraph sync -q # 若建场已 init，此处只需 sync；缺失 CLI 可跳过 ``` +## 开工前重叠裁决（catalog） + +```bash +python3 ../../scripts/agentctl.py catalog overlap \ + --task "$ID" --statement '<新设计原文>' \ + [--requirement R-xxx] [--feature ] [--mutex ] +# hard → 换 mutex/合并到持有者任务；soft → ack 留痕 +python3 ../../scripts/agentctl.py catalog ack \ + --task "$ID" --overlap \ + --decision proceed|merge_into|supersede|defer \ + --reason '<理由>' +``` + ## CURRENT 认领（docs apply） ```bash diff --git a/.doc/ops/2026-09-21-policy-registry.md b/.doc/ops/2026-09-21-policy-registry.md index 1b23713a..a4ba1429 100644 --- a/.doc/ops/2026-09-21-policy-registry.md +++ b/.doc/ops/2026-09-21-policy-registry.md @@ -51,3 +51,8 @@ CURRENT 是带时间的活跃协调快照；Git/机器状态用于核验实际 修订 ID：`subagent-workflow-2026-09-24-r2`；owner：Codex；state：`proposed`；能力：`manual`；effective_commit：无。 用户要求研究 Cursor Auto 下的 subagent 分工、阶段流程及报告，授权与本次验收见[任务](short-term/2026-09-24-subagent-workflow-standard.md)。[规范草案](2026-09-24-subagent-workflow-standard.md)为拟议行为的唯一来源，[研究报告](../audit/2026-09-24-subagent-workflow-research.md)给出事实和待修冲突，[模板](templates/subagent-stage-record.md)承载交接。 用户后续明确采用 Cursor Auto 实施 + gmktec 本机 PI 子任务，并批准默认模型分工及可调整文档。[模型配置](2026-09-24-subagent-model-config.md)为模型路由唯一来源，decision_state=approved；流程整体仍 proposed；已批准的模型配置与执行器已合入本地 dev。原 Cursor 原生角色/inherit 方案被替代。执行器已在任务分支实现并验证 Cursor CLI→MCP→PI，代码 69ffaa677e23a1dd9aa0528bb09b3acfcd38b119；[运行说明](2026-09-24-pi-subagent-executor.md)与[验证报告](../test-reports/2026-09-24-pi-executor-test-report.md)限定已验证的机器边界。dev 集成/完整流程审批仍独立记录；不以派发器可用宣称全阶段自动门禁。 + +## 治理目录库与需求重叠检测拟议修订 + +修订 ID：`governance-catalog-2026-09-24-r1`；owner：`Cursor`；state：`proposed`；能力：schema/门禁落地后为 `enforced`，向量层 `fail-open`；effective_commit：无（随受控合入记录）。 +用户授权实施跨对话需求重叠检测与治理目录（扩展既有 `.locks/state.sqlite`，不碰业务 PG）。唯一来源：[方案](./2026-09-24-governance-catalog-plan.md)；任务：[short-term](./short-term/2026-09-24-governance-catalog.md)。策略键 `catalog_gate`/`catalog_vector` 为可选，不得破坏 `shared_docs` 等集合相等校验。 diff --git a/.doc/ops/2026-09-24-governance-catalog-plan.md b/.doc/ops/2026-09-24-governance-catalog-plan.md new file mode 100644 index 00000000..175133fd --- /dev/null +++ b/.doc/ops/2026-09-24-governance-catalog-plan.md @@ -0,0 +1,46 @@ +# 治理目录库与跨对话需求重叠检测（方案） + +**修订 ID**：`governance-catalog-2026-09-24-r1` +**state**：`proposed` → 实施后随合入升 `active` +**owner**：Cursor +**工具能力**：schema/CRUD/门禁 `enforced`（落地后）；向量层 `fail-open` +**任务**：[short-term](./short-term/2026-09-24-governance-catalog.md) + +## 问题 + +跨对话新设计与既有功能/需求重叠时，现有 agentctl 仅做路径级 `overlaps()` 互斥，无需求语义比对。CURRENT/ISSUES/test-reports 为 Markdown 台账，机器无法强制开工前探测。 + +## 方案摘要 + +在既有 `.locks/state.sqlite`（与业务 PG 物理隔离）上 additive 扩展 Catalog 表： + +| 表 | 用途 | +|---|---| +| `catalog_requirements` | R-ID 需求条目 | +| `catalog_features` | 对齐 FEATURE-MATRIX record_id | +| `catalog_req_text` | FTS5 CJK 逐字索引（禁用 trigram） | +| `catalog_overlaps` / `catalog_decisions` | 检测结果与裁决留痕 | +| `catalog_issues` / `catalog_tests` / `catalog_releases` | 缺陷/测试/发布投影 | +| `catalog_embeddings` | 可降级向量增强 | + +### 分级门禁 + +1. **L1 结构化**：`same_mutex` / `same_feature` / `same_scope` / `same_req_id` → hard deny +2. **L2 语义**：FTS5 bm25 软门，须 `catalog overlap ack` 显式裁决后才可编码 +3. **L3 向量**：读 `EMBEDDING_*`；401/超时/缺钥 → `vector_layer=degraded`，不阻断 + +### 策略 + +`catalog_gate` / `catalog_vector` 为**可选键**（不得加入 `shared_docs` 等集合相等校验字段）。 + +### 权威关系 + +`.doc` Markdown 仍为事实来源；Catalog 为可检索投影 + 门禁依据；冲突时 `catalog drift` 告警，不自动改文档。 + +## 边界 + +不新建第二库/容器；不读写业务 PostgreSQL；不改 `overlaps()` 既有语义；不做跨机权威。 + +## CLI / MCP + +`agentctl catalog {add,list,show,search,overlap,ack,status,drift}`；MCP：`catalog_requirement_add` / `catalog_overlap_probe` / `catalog_overlap_ack` / `catalog_search` / `catalog_status`。 diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index 11ba6a99..1f757843 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -18,6 +18,7 @@ | owner | task | branch | worktree | mutex | status | |---|---|---|---|---|---| +| Cursor | governance-catalog-2026-09-24 | feature/governance-catalog-2026-09-24 | .worktrees/governance-catalog-2026-09-24 | agent-workflow-governance+catalog | in_progress | | Codex | subagent-workflow-standard-2026-09-24-land | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | pi-subagent-integration | done | | Cursor/Codex | subagent-workflow-standard-2026-09-24-executor | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | pi-subagent-executor | done | | Codex | subagent-workflow-standard-2026-09-24-config-r2 | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | subagent-workflow-docs | done | diff --git a/.doc/ops/lessons/INDEX.md b/.doc/ops/lessons/INDEX.md index f59e3b52..699ce6c6 100644 --- a/.doc/ops/lessons/INDEX.md +++ b/.doc/ops/lessons/INDEX.md @@ -7,6 +7,6 @@ | LESSON-20260923-nullable-duty | docgov、SQL、nullable、23503、可选职责、创建/heal | 检查所有写入口的缺席值语义；用真实 PG 验证 NULL/FK | validated | [可选职责持久化](2026-09-23-nullable-duty.md)；2026-09-23 | | LESSON-20260923-content-hash | docrevision、canonicalize、hash、bootstrap、hard-break、404、水合 | 保持文本语义并检查账本/文件一致性；新数据通过不能覆盖旧数据迁移 | validated | [正文规范化与投影](2026-09-23-content-hash.md)；2026-09-23 | | LESSON-20260923-matrix-status-cas | FEATURE-MATRIX、docs-apply、record_id、merge conflict、CAS | 矩阵/CURRENT 行变更走 docs apply；合并后核验状态列未被回退 | validated | [矩阵状态 CAS](2026-09-23-matrix-status-cas.md)；2026-09-23 | +| LESSON-20260924-pi-tool-boundaries | PI、MCP、readonly、path、snapshot、process-group | 路径语义与快照边界一致，合法路径和拒绝路径都验收 | validated | [文件工具边界](2026-09-24-pi-tool-boundaries.md)；2026-09-24 | validated 的证据范围见各条目；本日复核为历史材料和源码阅读，未重跑产品回归。后续按机制补充，过时条目标 superseded/retired 并保留替代链接。 -| LESSON-20260924-pi-tool-boundaries | PI、MCP、readonly、path、snapshot、process-group | 路径语义与快照边界一致，合法路径和拒绝路径都验收 | validated | [文件工具边界](2026-09-24-pi-tool-boundaries.md)；2026-09-24 | diff --git a/.doc/ops/short-term/2026-09-24-governance-catalog.md b/.doc/ops/short-term/2026-09-24-governance-catalog.md new file mode 100644 index 00000000..d6b6663c --- /dev/null +++ b/.doc/ops/short-term/2026-09-24-governance-catalog.md @@ -0,0 +1,69 @@ +--- +title: 治理目录库与跨对话需求重叠检测 +task_id: governance-catalog-2026-09-24 +status: in_progress +created: 2026-09-24 +updated: 2026-09-24 +owner: Cursor +task_type: implementation +task_tier: multi_stage +policy_revision: agent-governance-2026-09-21-r1 @ cfc80b34a40e8de78e77a44839571bb684d405d5 +branch: feature/governance-catalog-2026-09-24 +worktree: .worktrees/governance-catalog-2026-09-24 +base_dev_sha: cfc80b34a40e8de78e77a44839571bb684d405d5 +write_scope: + - scripts + - configs + - .doc + - .cursor +allowed_environments: [] +mutex: agent-workflow-governance+catalog +review_state: pending +integration_state: pending +deployment: not_required +cleanup: pending +cleanup_owner: Cursor +cleanup_trigger: validate pass + queue enqueue + closeout +remote_sync: not_requested +--- + +# 治理目录库与跨对话需求重叠检测 + +## 用户目标与授权 + +- **用户目标**：跨对话需求/功能重叠与冲突可机器检测；认领、缺陷、测试、发布版本进入与业务库隔离的治理目录。 +- **授权来源**：用户确认计划「治理目录库与需求重叠检测」并要求按 to-do 全量实施。 +- **交付终点**：Catalog schema + FTS5 中文召回 + 两级门禁 + 回填 + 可降级向量层 + MCP/技能 + 工程回归报告；受控合入 local `dev`。 +- **禁止**：不连业务 PostgreSQL；不新建第二库文件/容器；不自动升 MATRIX；不 push；不强制产品 PI 全链路（工程工具变更，NOT REQUIRED）。 + +## 范围与非目标 + +- **允许写入**：`scripts/agentctl*.py`、`scripts/catalog_backfill.py`、`scripts/mcp/agentctl-mcp.mjs`、`scripts/tests/`、`configs/agent-governance.json`、`.doc/`（方案/任务/报告/模板漂移修复）、`.cursor/skills` 与 mandatory 规则补充。 +- **非目标**：跨机权威认领、产品业务功能、自动删除 worktree。 + +## 相关问题与经验 + +- 检索词：`overlap` / `requirement` / `catalog` / `docs drift` / `FTS5` +- 命中：路径级 `overlaps()` 已 enforced；需求语义重叠无机制；embedding 401 运维债 → 向量层 fail-open。 +- 采纳：可选策略键扩展；CJK 逐字切分禁用 trigram；测试夹具同步复制新模块。 + +## 步骤与验收 + +- [ ] W0 方案主文档 + policy-registry proposed +- [ ] W1–W4 catalog 模块 + task register 联动 +- [ ] W5 回填 + drift +- [ ] W6 向量降级三态 +- [ ] W7 MCP + 技能 +- [ ] W8 unittest 全绿 + 报告 +- [ ] W9 validate → queue → closeout + +## 验证与证据 + +| 范围 / 被测 SHA | 命令与环境 | 预期 | 实际 / 判定 | 证据 | +|---|---|---|---|---| +| 工程工具 | `python3 -B -m unittest discover -s scripts/tests -p 'test_*.py' -v` | 全绿 | （待填） | test-report | + +## 收口与交接 + +- 五态见 frontmatter；合入后更新。 +- 相关：[方案](../2026-09-24-governance-catalog-plan.md) diff --git a/.doc/ops/templates/feature-matrix-row.md b/.doc/ops/templates/feature-matrix-row.md index 23a73a55..d23cc36f 100644 --- a/.doc/ops/templates/feature-matrix-row.md +++ b/.doc/ops/templates/feature-matrix-row.md @@ -1,19 +1,20 @@ # FEATURE-MATRIX 行模板 -复制到 [FEATURE-MATRIX.md](../../status/FEATURE-MATRIX.md) 的完成度表中。能力状态与 `production_ready`（gap-matrix）正交。 +复制到 [FEATURE-MATRIX.md](../../status/FEATURE-MATRIX.md) 的完成度表中。能力状态与 `production_ready`（gap-matrix）正交。`record_id` 必须在**第一列**（docs apply 校验）。 | 字段 | 填写说明 | |------|----------| +| record_id | 稳定 kebab-case slug；不随批次改名 | | 模块 | 短名 | | 声明范围 | 本行冻结边界；`done` 只覆盖此范围 | -| 状态 | `done` \| `partial` \| `planned` \| `deferred` \| `blocked` | +| 状态 | `done` \| `partial` \| `planned` \| `deferred` \| `blocked` \| `removed` | | 发布关系 | `non_blocker` / `补证` / `加固` / `deferred_enhancement` | | 证据 | 设计/代码/test-report 链接；可追 commit/环境/schema | | 剩余工作 / 关闭条件 | partial/planned 必填；stub/SKIP/fallback 不得冒充关闭 | | mutex | 与 CURRENT 并行认领一致 | ```markdown -| <模块名> | <声明范围> | planned | non_blocker | （设计/代码/test-report 链接） | （关闭条件） | `` | +| | <模块名> | <声明范围> | planned | non_blocker | （设计/代码/test-report 链接） | （关闭条件） | `` | ``` 拆行收口时另在「边界 lineage」区记录 before/after、义务去向与证据覆盖结论。 diff --git a/.doc/test-reports/2026-09-24-governance-catalog-test-report.md b/.doc/test-reports/2026-09-24-governance-catalog-test-report.md new file mode 100644 index 00000000..2a152654 --- /dev/null +++ b/.doc/test-reports/2026-09-24-governance-catalog-test-report.md @@ -0,0 +1,77 @@ +# 治理目录库与跨对话需求重叠检测 测试报告 + +## 一、 测试背景 (Background) + +跨对话新设计与既有功能/需求重叠时，agentctl 仅有路径级 `overlaps()` 互斥，无需求语义比对。本轮在 `.locks/state.sqlite` 扩展 Catalog 层（与业务 PostgreSQL 物理隔离），落地 FTS5 中文召回、两级门禁与可降级向量层。 + +## 二、 测试目标 (Test Objectives) + +- Catalog schema / CRUD / FTS5 CJK（双字词召回，禁用 trigram） +- L1 硬拒（same_req_id）与 L2 软门裁决留痕 +- `with` 在未 ack 时拒绝；ack 后放行 +- 向量层缺钥/错 URL 降级不阻断 +- 既有 agentctl 回归无破坏 +- 工程工具变更：**产品 PI 全链路 NOT REQUIRED**（依据：test-reports-standard §范围判定 — 纯工程规范/工具） + +## 三、 测试方案与环境 (Test Plan & Environment) + +- OS：Linux；Python 3.14.7；SQLite 3.53.4（FTS5） +- 入口：`python3 -B -m unittest discover -s scripts/tests -p 'test_*.py'` +- 新增：`scripts/tests/test_agentctl_catalog.py`（6 用例） +- 回填：`scripts/catalog_backfill.py` → features_matrix=29, issues=10, tests=207 + +## 四、 预期结果 (Expected Results) + +| 用例 | 预期 | +|---|---| +| schema/CRUD | add/list/show/status OK | +| CJK digram | trigram `"灰度"`=0；CJK 切分 ≥2 | +| same_req_id | `REQUIREMENT_CONFLICT` | +| soft ack | 未 ack → `OVERLAP_ACK_REQUIRED`；ack 后 `with` 成功 | +| vector degrade | `vector_layer` ∈ degraded/disabled/ok，exit 0 | +| hard proceed ack | 拒绝 | + +## 五、 实际测试结果与判定 (Actual Results & Evaluation) + +### 5.1 Catalog 专项（6/6 PASS） + +```text +test_cjk_fts_digram_recall ... ok +test_hard_proceed_ack_rejected ... ok +test_hard_same_req_id_conflict ... ok +test_schema_and_crud ... ok +test_soft_overlap_requires_ack ... ok +test_vector_layer_degrades_without_key ... ok +``` + +**判定**：PASS + +### 5.2 全量工程回归 + +```text +Ran 179 tests in 123.217s +OK +``` + +证据：`.doc/test-reports/assets/2026-09-24-governance-catalog/unittest-full.log` + +**判定**：PASS + +### 5.3 回填与 drift + +- backfill：`features_matrix=29`, `issues=10`, `tests=207`, `task_features=128`, `releases=0` +- `catalog drift`：`regressions=[]`, `record_count=29` + +**判定**：PASS + +### 性能与耗时 + +全量 unittest ≈ 123 s；catalog 专项 ≈ 3 s。 + +## 六、 自动化测试套件汇总 (Test Suite Execution) + +本任务为工程工具变更，**不要求** `go test ./...` 产品回归；已跑 agentctl/unittest 全量 179 OK。产品 PI NOT REQUIRED。 + +## 七、 测试结论与后续优化建议 (Conclusion & Next Steps) + +达到合入 local `dev` 标准。后续可选：开启 `catalog_vector.enabled`（embedding 401 修复后）、把 CURRENT 认领投影到 requirements、daemon 审计周期附加 `catalog drift`。 diff --git a/.doc/test-reports/INDEX.md b/.doc/test-reports/INDEX.md new file mode 100644 index 00000000..fb606742 --- /dev/null +++ b/.doc/test-reports/INDEX.md @@ -0,0 +1,7 @@ +# 测试报告索引 + +本目录归档按 [测试报告规范](../../.cursor/rules/test-reports-standard.mdc) 命名的验收报告：`YYYY-MM-DD-<模块/功能/版本>-test-report.md`。附件在 `assets//`，不得与报告同层混放。 + +检索：按文件名日期 + slug，或 `rg -n '判定|PASS|FAIL' .doc/test-reports`。结构化投影见 agentctl `catalog_tests`（`agentctl catalog status` / backfill）。 + +历史命名例外（保留不改）：`2026-09-14-controlled-content-ui-acceptance.md`、`2026-09-17-initial-load-performance-baseline.md`。 diff --git a/.doc/test-reports/assets/2026-09-24-governance-catalog/unittest-full.txt b/.doc/test-reports/assets/2026-09-24-governance-catalog/unittest-full.txt new file mode 100644 index 00000000..00843484 --- /dev/null +++ b/.doc/test-reports/assets/2026-09-24-governance-catalog/unittest-full.txt @@ -0,0 +1,70 @@ +........................................................................./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in + def __init__(self, name, parent): +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in + def __init__(self, name, parent): +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in + def __init__(self, name, parent): +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in + def __init__(self, name, parent): +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in + def __init__(self, name, parent): +ResourceWarning: Enable tracemalloc to get the object allocation traceback +{"demoted": [{"from": "running", "run_id": "hb-stale", "to": "suspect"}, {"from": "suspect", "run_id": "sus-stale", "to": "quarantine"}], "heartbeat_ttl_seconds": 300, "status": "ok", "suspect_ttl_seconds": 600} +.{"demoted": [{"from": "running", "run_id": "run-unknown", "to": "suspect"}], "heartbeat_ttl_seconds": 300, "status": "ok", "suspect_ttl_seconds": 600} +...usage: python3.14 -m unittest [-h] [--repo REPO] + {init,status,doctor,task,with,guard,worktree-check,docs,queue,cleanup,recover,operation,freeze,unfreeze,hooks,validate,integrate,deploy,claim,quarantine,daemon,catalog} ... +python3.14 -m unittest: error: unrecognized arguments: --head-sha deadbeef +..................................................................................../home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=10 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=6 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=8 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'> + result = method() +ResourceWarning: Enable tracemalloc to get the object allocation traceback +............... +---------------------------------------------------------------------- +Ran 179 tests in 123.217s + +OK diff --git a/configs/agent-governance.json b/configs/agent-governance.json index 1a3f4431..a1ae14ed 100644 --- a/configs/agent-governance.json +++ b/configs/agent-governance.json @@ -54,5 +54,17 @@ ] }, "heartbeat_ttl_seconds": 300, - "suspect_ttl_seconds": 600 + "suspect_ttl_seconds": 600, + "catalog_gate": { + "hard_kinds": ["same_mutex", "same_feature", "same_scope", "same_req_id"], + "semantic_top_k": 10, + "bm25_threshold": -8.0, + "ack_required": true + }, + "catalog_vector": { + "enabled": false, + "timeout_seconds": 8, + "fail_open": true, + "cosine_threshold": 0.82 + } } diff --git a/scripts/agentctl.py b/scripts/agentctl.py index 87f10ddd..a1585480 100755 --- a/scripts/agentctl.py +++ b/scripts/agentctl.py @@ -26,6 +26,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) sys.dont_write_bytecode = True os.environ.setdefault('PYTHONDONTWRITEBYTECODE', '1') import agentctl_w4 as w4 # noqa: E402 +import agentctl_catalog as catalog # noqa: E402 VERSION = 1 POLICY_REL = 'configs/agent-governance.json' @@ -63,6 +64,19 @@ DEFAULT_POLICY = { }, 'heartbeat_ttl_seconds': 300, 'suspect_ttl_seconds': 600, + # Optional catalog keys (not in required set-equality checks): + 'catalog_gate': { + 'hard_kinds': ['same_mutex', 'same_feature', 'same_scope', 'same_req_id'], + 'semantic_top_k': 10, + 'bm25_threshold': -8.0, + 'ack_required': True, + }, + 'catalog_vector': { + 'enabled': False, + 'timeout_seconds': 8, + 'fail_open': True, + 'cosine_threshold': 0.82, + }, } PROJECTION_STATUS_VOCAB = frozenset({'planned', 'in_progress', 'blocked', 'done', 'cancelled'}) PROJECTION_STATUS_FROM_MACHINE = {'queued': 'in_progress', 'idle': 'planned'} @@ -142,6 +156,9 @@ def deny(code, message, **details): raise Denied(code, message, **details) +catalog.bind_deny(deny) + + def emit(value, stream=sys.stdout): print(json.dumps(value, ensure_ascii=False, sort_keys=True), file=stream, flush=True) @@ -192,6 +209,10 @@ def _validate_policy_shape(data) -> dict: for key in ('heartbeat_ttl_seconds', 'suspect_ttl_seconds'): if key in data and (not isinstance(data[key], int) or data[key] <= 0): deny('POLICY_INVALID', f'{key} must be a positive integer') + # Optional catalog keys: shape checked in catalog.apply_policy; reject non-objects early. + for key in ('catalog_gate', 'catalog_vector'): + if key in data and data[key] is not None and not isinstance(data[key], dict): + deny('POLICY_INVALID', f'{key} must be an object') for filename, rules in data['doc_rules'].items(): if not isinstance(rules, dict): deny('POLICY_INVALID', 'doc_rules entries must be objects', file=filename) @@ -218,6 +239,7 @@ def apply_policy(policy: dict) -> dict: HEARTBEAT_TTL_SECONDS = int(policy.get('heartbeat_ttl_seconds', DEFAULT_POLICY['heartbeat_ttl_seconds'])) SUSPECT_TTL_SECONDS = int(policy.get('suspect_ttl_seconds', DEFAULT_POLICY['suspect_ttl_seconds'])) w4.apply_policy(policy) + catalog.apply_policy(policy) return policy @@ -749,6 +771,7 @@ def ensure_schema(db): if name not in existing: db.execute(f'ALTER TABLE runs ADD COLUMN {name} {decl}') w4.ensure_schema(db) + catalog.ensure_schema(db) def codegraph_is_ignored(path): @@ -1081,6 +1104,10 @@ def execute(repo, args): command = args.command[1:] if args.command[:1] == ['--'] else args.command if not command: deny('INVALID_REQUEST', 'Expected a child command after --') + # Soft-overlap acks must be clear before exclusive writes (shared reads exempt). + if not args.shared: + with repo.db() as db: + catalog.assert_acks_clear(db, args.task) cwd = Path(args.cwd).resolve() if args.cwd else Path(task['worktree']) known = {Path(r['worktree']).resolve() for r in repo.worktrees()} if cwd not in known or cwd not in (Path(task['worktree']), repo.main): @@ -2654,6 +2681,10 @@ def parser(): for name in ('init','status','doctor'):sub.add_parser(name) task=sub.add_parser('task').add_subparsers(dest='task_action',required=True) reg=task.add_parser('register');reg.add_argument('--id',required=True);reg.add_argument('--owner',required=True);reg.add_argument('--scope',action='append',required=True);reg.add_argument('--capability',action='append',default=[]) + reg.add_argument('--requirement',help='Catalog R-ID to bind at register') + reg.add_argument('--intent',help='Design intent text for overlap probe') + reg.add_argument('--feature',help='FEATURE-MATRIX record_id / catalog feature') + reg.add_argument('--mutex',help='Mutex key for hard same_mutex gate') close=task.add_parser('close');close.add_argument('--id',required=True);close.add_argument('--status',choices=['done','cancelled'],default='done') w=sub.add_parser('with');w.add_argument('--task',required=True);w.add_argument('--resource',action='append',required=True);w.add_argument('--shared',action='store_true');w.add_argument('--request-id');w.add_argument('--wait',type=float,default=0);w.add_argument('--cwd');w.add_argument('command',nargs=argparse.REMAINDER) g=sub.add_parser('guard').add_subparsers(dest='kind',required=True) @@ -2718,6 +2749,36 @@ def parser(): drun.add_argument('--audit-every',type=int,default=30,dest='audit_every', help='Every N cycles run cleanup report + docs drift (0=off)') dm.add_parser('status') + cat = sub.add_parser('catalog').add_subparsers(dest='catalog_action', required=True) + cadd = cat.add_parser('add') + cadd.add_argument('--id', required=True) + cadd.add_argument('--title', required=True) + cadd.add_argument('--statement', required=True) + cadd.add_argument('--owner', required=True) + cadd.add_argument('--feature') + cadd.add_argument('--source-task') + cadd.add_argument('--source-doc') + cadd.add_argument('--state', default='proposed') + cadd.add_argument('--conversation-ref') + clist = cat.add_parser('list'); clist.add_argument('--state'); clist.add_argument('--limit', type=int, default=100) + cshow = cat.add_parser('show'); cshow.add_argument('--id', required=True) + csearch = cat.add_parser('search'); csearch.add_argument('--query', required=True); csearch.add_argument('--top', type=int, default=10) + cov = cat.add_parser('overlap') + cov.add_argument('--task', required=True) + cov.add_argument('--statement', required=True) + cov.add_argument('--requirement') + cov.add_argument('--feature') + cov.add_argument('--mutex') + cov.add_argument('--top', type=int, default=10) + cack = cat.add_parser('ack') + cack.add_argument('--task', required=True) + cack.add_argument('--overlap', required=True) + cack.add_argument('--decision', required=True, choices=sorted(catalog.DECISIONS)) + cack.add_argument('--reason', required=True) + cack.add_argument('--decided-by', default='Cursor') + cack.add_argument('--authorization-ref') + cat.add_parser('status') + cat.add_parser('drift', help='Refresh features from MATRIX; report done→non-done regressions') return p @@ -2726,7 +2787,7 @@ def main(): write_actions = { 'init', 'task', 'with', 'docs', 'validate', 'integrate', 'queue', 'deploy', 'recover', 'freeze', 'unfreeze', 'hooks', 'guard', 'cleanup', 'claim', - 'quarantine', 'daemon', + 'quarantine', 'daemon', 'catalog', } # status/doctor remain readable without a policy file and never create one. if args.action in write_actions: @@ -2744,6 +2805,10 @@ def main(): require_policy(repo) # list needs TTL/schema constants elif args.action == 'daemon' and getattr(args, 'daemon_action', None) == 'status': pass # read-only probe; never creates manager.lock or heartbeat + elif args.action == 'catalog' and getattr(args, 'catalog_action', None) in ( + 'list', 'show', 'search', 'status', + ): + require_policy(repo) # needs schema; non-mutating reads elif args.action == 'cleanup': require_policy(repo) elif args.action == 'hooks' and getattr(args, 'hook_action', None) == 'codegraph-sync': @@ -2783,6 +2848,25 @@ def main(): safe_path(repo.current,scope) for cap in args.capability: if cap not in ('docs:admin','docs:approve'):resource(cap) + # Catalog gate BEFORE insert: probe commits in its own txn so soft hits survive deny. + req = getattr(args, 'requirement', None) + intent = getattr(args, 'intent', None) + feature = getattr(args, 'feature', None) + mutex = getattr(args, 'mutex', None) + if req or intent or feature or mutex: + with repo.db(True) as db: + gate_result = catalog.gate_task_register( + db, + task=args.id, + scopes=scopes, + requirement=req, + intent=intent, + feature=feature, + mutex=mutex, + overlaps_fn=overlaps, + statement=intent, + ) + catalog.enforce_gate_result(gate_result) values=(args.id,args.owner,str(repo.current),branch,json.dumps(scopes),json.dumps(sorted(set(args.capability))),'active',git(repo.current,'rev-parse','HEAD')) with repo.os_file('worktree-' + digest(str(repo.current)) + '.lock').open('a+') as lifecycle, repo.db(True) as db: fcntl.flock(lifecycle,fcntl.LOCK_SH|fcntl.LOCK_NB) @@ -2865,9 +2949,101 @@ def main(): elif args.action=='daemon': if args.daemon_action=='run':return daemon_run(repo,args) if args.daemon_action=='status':return daemon_status(repo,args) + elif args.action=='catalog': + return catalog_cmd(repo, args) return 0 +def catalog_cmd(repo, args): + """Dispatch agentctl catalog subcommands.""" + action = args.catalog_action + if action == 'add': + with repo.db(True) as db: + result = catalog.requirement_add( + db, + req_id=args.id, + title=args.title, + statement=args.statement, + owner=args.owner, + feature=args.feature, + source_task=args.source_task, + source_doc=args.source_doc, + state=args.state, + conversation_ref=args.conversation_ref, + ) + emit(result) + return 0 + if action == 'list': + with repo.db() as db: + rows = catalog.requirement_list(db, state=args.state, limit=args.limit) + emit({'status': 'ok', 'requirements': rows}) + return 0 + if action == 'show': + with repo.db() as db: + row = catalog.requirement_show(db, args.id) + emit({'status': 'ok', 'requirement': row}) + return 0 + if action == 'search': + with repo.db() as db: + hits = catalog.search(db, args.query, top=args.top) + emit({'status': 'ok', 'hits': hits}) + return 0 + if action == 'overlap': + with repo.db(True) as db: + scopes = [] + try: + task = task_row(repo, args.task) + scopes = json.loads(task['scopes']) if isinstance(task['scopes'], str) else task['scopes'] + except Denied: + # Allow pre-register probes; path/scope hard checks skipped without a task row. + pass + result = catalog.probe_overlap( + db, + task=args.task, + statement=args.statement, + requirement=args.requirement, + feature=args.feature, + mutex=args.mutex, + scopes=scopes, + top=args.top, + overlaps_fn=overlaps, + ) + emit(result) + return 0 + if action == 'ack': + with repo.db(True) as db: + result = catalog.overlap_ack( + db, + task=args.task, + overlap_id=args.overlap, + decision=args.decision, + decided_by=args.decided_by, + reason=args.reason, + authorization_ref=args.authorization_ref, + ) + emit(result) + return 0 + if action == 'status': + with repo.db() as db: + emit(catalog.catalog_status(db)) + return 0 + if action == 'drift': + matrix_path = repo.main / '.doc/status/FEATURE-MATRIX.md' + # Prefer clean tip from refs/heads/dev when available. + try: + content = git(repo.main, 'show', 'refs/heads/dev:.doc/status/FEATURE-MATRIX.md') + except Exception: + content = matrix_path.read_text(encoding='utf-8') if matrix_path.is_file() else '' + states = matrix_states(content) if content else {} + with repo.db(True) as db: + result = catalog.catalog_drift(db, matrix_states=states) + event(db, 'catalog', {'event': 'drift', **{k: result[k] for k in ('record_count', 'regressions')}}) + emit(result) + return 0 + deny('INVALID_REQUEST', f'Unknown catalog action: {action}') + return 1 + + if __name__=='__main__': try: sys.exit(main() or 0) diff --git a/scripts/agentctl_catalog.py b/scripts/agentctl_catalog.py new file mode 100644 index 00000000..724a0718 --- /dev/null +++ b/scripts/agentctl_catalog.py @@ -0,0 +1,855 @@ +#!/usr/bin/env python3 +"""Catalog governance layer: requirements, FTS5 CJK recall, overlap gates, embeddings. + +Lives beside agentctl.py. Additive schema only; never touches product PostgreSQL. +Vector layer is fail-open: missing/failing embeddings degrade to FTS5 only. +""" +from __future__ import annotations + +import hashlib +import json +import math +import os +import re +import sqlite3 +import struct +import time +import urllib.error +import urllib.request +import uuid +from typing import Any, Callable, Optional + +SCHEMA_SQL = ''' +CREATE TABLE IF NOT EXISTS catalog_requirements( + id TEXT PRIMARY KEY, + title TEXT NOT NULL, + statement TEXT NOT NULL, + feature TEXT, + source_task TEXT, + source_doc TEXT, + owner TEXT NOT NULL, + state TEXT NOT NULL, + supersedes TEXT, + conversation_ref TEXT, + created REAL NOT NULL, + updated REAL NOT NULL +); +CREATE INDEX IF NOT EXISTS catalog_req_state ON catalog_requirements(state, feature); +CREATE INDEX IF NOT EXISTS catalog_req_task ON catalog_requirements(source_task); +CREATE TABLE IF NOT EXISTS catalog_features( + id TEXT PRIMARY KEY, + title TEXT NOT NULL, + scope TEXT, + mutex TEXT, + matrix_status TEXT, + updated REAL NOT NULL +); +CREATE VIRTUAL TABLE IF NOT EXISTS catalog_req_text USING fts5( + req_id UNINDEXED, body, seg, tokenize='unicode61' +); +CREATE TABLE IF NOT EXISTS catalog_overlaps( + id TEXT PRIMARY KEY, + probe_task TEXT NOT NULL, + probe_req TEXT, + target_req TEXT, + target_task TEXT, + target_feature TEXT, + level TEXT NOT NULL, + kind TEXT NOT NULL, + score REAL, + evidence TEXT NOT NULL, + created REAL NOT NULL +); +CREATE INDEX IF NOT EXISTS catalog_overlaps_probe ON catalog_overlaps(probe_task, created); +CREATE TABLE IF NOT EXISTS catalog_decisions( + id TEXT PRIMARY KEY, + overlap_id TEXT NOT NULL, + task TEXT NOT NULL, + decision TEXT NOT NULL, + decided_by TEXT NOT NULL, + reason TEXT NOT NULL, + authorization_ref TEXT, + created REAL NOT NULL +); +CREATE INDEX IF NOT EXISTS catalog_decisions_task ON catalog_decisions(task, overlap_id); +CREATE TABLE IF NOT EXISTS catalog_issues( + issue_id TEXT PRIMARY KEY, + severity TEXT, + issue_state TEXT, + owner TEXT, + module TEXT, + keywords TEXT, + main_record TEXT, + lesson_ids TEXT, + requirements TEXT, + tasks TEXT, + recurrence_count INTEGER DEFAULT 0, + evidence_basis TEXT, + created REAL, + updated REAL +); +CREATE TABLE IF NOT EXISTS catalog_tests( + id TEXT PRIMARY KEY, + report_path TEXT NOT NULL, + task TEXT, + requirements TEXT, + scope TEXT, + verdict TEXT, + level TEXT, + real_agent INTEGER NOT NULL DEFAULT 0, + schema_version TEXT, + env TEXT, + sha TEXT, + created REAL +); +CREATE TABLE IF NOT EXISTS catalog_releases( + id TEXT PRIMARY KEY, + env TEXT NOT NULL, + branch TEXT, + commit_sha TEXT, + tag TEXT, + deploy_seq INTEGER, + schema_version TEXT, + image_digest TEXT, + source_task TEXT, + phase TEXT, + receipt TEXT, + created REAL +); +CREATE TABLE IF NOT EXISTS catalog_embeddings( + req_id TEXT PRIMARY KEY, + model TEXT NOT NULL, + dim INTEGER NOT NULL, + vec BLOB NOT NULL, + source_sha256 TEXT NOT NULL, + status TEXT NOT NULL, + created REAL NOT NULL +); +CREATE TABLE IF NOT EXISTS catalog_task_bindings( + task TEXT PRIMARY KEY, + requirement TEXT, + intent TEXT, + mutex TEXT, + feature TEXT, + updated REAL NOT NULL +); +''' + +REQ_STATES = frozenset({ + 'proposed', 'accepted', 'in_progress', 'delivered', 'superseded', 'rejected', +}) +HARD_KINDS = frozenset({'same_mutex', 'same_feature', 'same_scope', 'same_req_id'}) +SOFT_KINDS = frozenset({'semantic'}) +DECISIONS = frozenset({'proceed', 'merge_into', 'supersede', 'defer', 'reject'}) +REQ_ID_RE = re.compile(r'^R-[A-Za-z0-9][A-Za-z0-9._-]{0,120}$') + +DEFAULT_GATE = { + 'hard_kinds': ['same_mutex', 'same_feature', 'same_scope', 'same_req_id'], + 'semantic_top_k': 10, + 'bm25_threshold': -8.0, + 'ack_required': True, +} +DEFAULT_VECTOR = { + 'enabled': False, + 'timeout_seconds': 8, + 'fail_open': True, + 'cosine_threshold': 0.82, +} + +CATALOG_GATE: dict = dict(DEFAULT_GATE) +CATALOG_VECTOR: dict = dict(DEFAULT_VECTOR) + +# Deny callback injected by agentctl (deny(code, message, **details)). +_deny: Optional[Callable[..., Any]] = None + + +def bind_deny(fn: Callable[..., Any]) -> None: + global _deny + _deny = fn + + +def _fail(code: str, message: str, **details: Any) -> None: + if _deny is None: + raise RuntimeError(f'{code}: {message} {details}') + _deny(code, message, **details) + + +def apply_policy(policy: dict) -> None: + """Optional catalog_gate / catalog_vector keys; missing → defaults.""" + global CATALOG_GATE, CATALOG_VECTOR + gate = policy.get('catalog_gate') + if gate is None: + CATALOG_GATE = dict(DEFAULT_GATE) + elif not isinstance(gate, dict): + _fail('POLICY_INVALID', 'catalog_gate must be an object') + else: + merged = dict(DEFAULT_GATE) + merged.update(gate) + if not isinstance(merged.get('hard_kinds'), list): + _fail('POLICY_INVALID', 'catalog_gate.hard_kinds must be a list') + if not isinstance(merged.get('semantic_top_k'), int) or merged['semantic_top_k'] <= 0: + _fail('POLICY_INVALID', 'catalog_gate.semantic_top_k must be a positive int') + if not isinstance(merged.get('bm25_threshold'), (int, float)): + _fail('POLICY_INVALID', 'catalog_gate.bm25_threshold must be a number') + if not isinstance(merged.get('ack_required'), bool): + _fail('POLICY_INVALID', 'catalog_gate.ack_required must be a bool') + CATALOG_GATE = merged + vec = policy.get('catalog_vector') + if vec is None: + CATALOG_VECTOR = dict(DEFAULT_VECTOR) + elif not isinstance(vec, dict): + _fail('POLICY_INVALID', 'catalog_vector must be an object') + else: + merged = dict(DEFAULT_VECTOR) + merged.update(vec) + if not isinstance(merged.get('enabled'), bool): + _fail('POLICY_INVALID', 'catalog_vector.enabled must be a bool') + if not isinstance(merged.get('timeout_seconds'), int) or merged['timeout_seconds'] <= 0: + _fail('POLICY_INVALID', 'catalog_vector.timeout_seconds must be a positive int') + if not isinstance(merged.get('fail_open'), bool): + _fail('POLICY_INVALID', 'catalog_vector.fail_open must be a bool') + CATALOG_VECTOR = merged + + +def ensure_schema(db: sqlite3.Connection) -> None: + db.executescript(SCHEMA_SQL) + + +def _now() -> float: + return time.time() + + +def split_cjk(text: str) -> str: + """Insert spaces around CJK chars so unicode61 FTS can phrase-match digrams.""" + if not text: + return '' + spaced = re.sub(r'([\u3400-\u9fff\uf900-\ufaff])', r' \1 ', text) + return ' '.join(spaced.split()) + + +def fts_phrase(text: str) -> str: + """Build a double-quoted FTS5 phrase from CJK-split tokens.""" + tokens = split_cjk(text).split() + if not tokens: + return '""' + # Escape embedded double quotes for FTS5. + safe = [t.replace('"', '""') for t in tokens] + return '"' + ' '.join(safe) + '"' + + +def validate_req_id(req_id: str) -> str: + if not REQ_ID_RE.fullmatch(req_id): + _fail('INVALID_REQUEST', 'Requirement id must match R-', id=req_id) + return req_id + + +def requirement_add( + db: sqlite3.Connection, + *, + req_id: str, + title: str, + statement: str, + owner: str, + feature: Optional[str] = None, + source_task: Optional[str] = None, + source_doc: Optional[str] = None, + state: str = 'proposed', + supersedes: Optional[str] = None, + conversation_ref: Optional[str] = None, +) -> dict: + validate_req_id(req_id) + if state not in REQ_STATES: + _fail('INVALID_STATE', 'Unknown requirement state', state=state) + if not title.strip() or not statement.strip() or not owner.strip(): + _fail('INVALID_REQUEST', 'title, statement, and owner are required') + existing = db.execute('SELECT id, source_task FROM catalog_requirements WHERE id=?', (req_id,)).fetchone() + if existing: + if source_task and existing['source_task'] and existing['source_task'] != source_task: + _fail( + 'REQUIREMENT_EXISTS', + 'Requirement id already owned by another task', + id=req_id, + holder=existing['source_task'], + ) + _fail('REQUIREMENT_EXISTS', 'Requirement id already exists', id=req_id) + now = _now() + db.execute( + 'INSERT INTO catalog_requirements VALUES(?,?,?,?,?,?,?,?,?,?,?,?)', + ( + req_id, title.strip(), statement.strip(), feature, source_task, source_doc, + owner.strip(), state, supersedes, conversation_ref, now, now, + ), + ) + _index_requirement(db, req_id, title.strip() + '\n' + statement.strip()) + return {'status': 'added', 'id': req_id} + + +def _index_requirement(db: sqlite3.Connection, req_id: str, body: str) -> None: + db.execute('DELETE FROM catalog_req_text WHERE req_id=?', (req_id,)) + db.execute( + 'INSERT INTO catalog_req_text(req_id, body, seg) VALUES(?,?,?)', + (req_id, body, split_cjk(body)), + ) + + +def requirement_list(db: sqlite3.Connection, *, state: Optional[str] = None, limit: int = 100) -> list[dict]: + if state: + rows = db.execute( + 'SELECT * FROM catalog_requirements WHERE state=? ORDER BY updated DESC LIMIT ?', + (state, limit), + ).fetchall() + else: + rows = db.execute( + 'SELECT * FROM catalog_requirements ORDER BY updated DESC LIMIT ?', (limit,) + ).fetchall() + return [dict(r) for r in rows] + + +def requirement_show(db: sqlite3.Connection, req_id: str) -> dict: + row = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (req_id,)).fetchone() + if not row: + _fail('UNKNOWN_REQUIREMENT', 'Requirement not found', id=req_id) + return dict(row) + + +def feature_upsert( + db: sqlite3.Connection, + *, + feature_id: str, + title: str, + scope: Optional[str] = None, + mutex: Optional[str] = None, + matrix_status: Optional[str] = None, +) -> dict: + now = _now() + db.execute( + '''INSERT INTO catalog_features(id, title, scope, mutex, matrix_status, updated) + VALUES(?,?,?,?,?,?) + ON CONFLICT(id) DO UPDATE SET + title=excluded.title, + scope=COALESCE(excluded.scope, catalog_features.scope), + mutex=COALESCE(excluded.mutex, catalog_features.mutex), + matrix_status=COALESCE(excluded.matrix_status, catalog_features.matrix_status), + updated=excluded.updated''', + (feature_id, title, scope, mutex, matrix_status, now), + ) + return {'status': 'upserted', 'id': feature_id} + + +def search(db: sqlite3.Connection, query: str, *, top: int = 10) -> list[dict]: + tokens = split_cjk(query).split() + if not tokens: + return [] + # Soft recall: AND of char tokens (all must appear). Phrase is too strict for + # near-paraphrase overlap ("检测与治理" vs "检测治理"). Escape FTS keywords. + safe = [] + for t in tokens: + t = t.replace('"', '""') + if t.upper() in {'AND', 'OR', 'NOT', 'NEAR'}: + t = '"' + t + '"' + else: + t = '"' + t + '"' # quote each token to avoid column/syntax issues + safe.append(t) + # Quoting each token and joining with spaces → AND of tokens in FTS5. + match_expr = ' '.join(safe) + try: + rows = db.execute( + '''SELECT r.id, r.title, r.statement, r.feature, r.source_task, r.state, + bm25(catalog_req_text) AS score + FROM catalog_req_text + JOIN catalog_requirements r ON r.id = catalog_req_text.req_id + WHERE catalog_req_text MATCH ? + ORDER BY score + LIMIT ?''', + (match_expr, top), + ).fetchall() + except sqlite3.OperationalError: + return [] + return [dict(r) for r in rows] + + +def bind_task( + db: sqlite3.Connection, + *, + task: str, + requirement: Optional[str] = None, + intent: Optional[str] = None, + mutex: Optional[str] = None, + feature: Optional[str] = None, +) -> dict: + db.execute( + '''INSERT INTO catalog_task_bindings(task, requirement, intent, mutex, feature, updated) + VALUES(?,?,?,?,?,?) + ON CONFLICT(task) DO UPDATE SET + requirement=COALESCE(excluded.requirement, catalog_task_bindings.requirement), + intent=COALESCE(excluded.intent, catalog_task_bindings.intent), + mutex=COALESCE(excluded.mutex, catalog_task_bindings.mutex), + feature=COALESCE(excluded.feature, catalog_task_bindings.feature), + updated=excluded.updated''', + (task, requirement, intent, mutex, feature, _now()), + ) + return {'status': 'bound', 'task': task} + + +def _active_tasks(db: sqlite3.Connection) -> list[dict]: + try: + rows = db.execute("SELECT * FROM tasks WHERE state='active'").fetchall() + except sqlite3.Error: + return [] + return [dict(r) for r in rows] + + +def _path_overlaps(scopes_a: list[str], scopes_b: list[str], overlaps_fn: Callable) -> bool: + for a in scopes_a: + for b in scopes_b: + if overlaps_fn('path:' + a, 'path:' + b): + return True + return False + + +def probe_overlap( + db: sqlite3.Connection, + *, + task: str, + statement: str, + requirement: Optional[str] = None, + feature: Optional[str] = None, + mutex: Optional[str] = None, + scopes: Optional[list[str]] = None, + top: Optional[int] = None, + overlaps_fn: Optional[Callable] = None, +) -> dict: + """Run L1 hard + L2 soft (+ optional L3) overlap detection; persist soft/hard rows.""" + top = top or int(CATALOG_GATE['semantic_top_k']) + hard_kinds = set(CATALOG_GATE.get('hard_kinds') or DEFAULT_GATE['hard_kinds']) + hard: list[dict] = [] + soft: list[dict] = [] + now = _now() + + # L1: same_req_id + if requirement and 'same_req_id' in hard_kinds: + row = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (requirement,)).fetchone() + if row and row['source_task'] and row['source_task'] != task: + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=requirement, + target_task=row['source_task'], target_feature=row['feature'], + level='hard', kind='same_req_id', score=1.0, + evidence=f'Requirement {requirement} owned by {row["source_task"]}', + created=now, + ) + hard.append(hit) + + # L1: same_feature — another active requirement on same feature (not superseded) + if feature and 'same_feature' in hard_kinds: + rows = db.execute( + '''SELECT * FROM catalog_requirements + WHERE feature=? AND state NOT IN ('superseded','rejected','delivered') + AND (source_task IS NULL OR source_task != ?)''', + (feature, task), + ).fetchall() + for row in rows: + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=row['id'], + target_task=row['source_task'], target_feature=feature, + level='hard', kind='same_feature', score=1.0, + evidence=f'Feature {feature} already claimed by requirement {row["id"]}', + created=now, + ) + hard.append(hit) + + # L1: same_mutex via bindings + CURRENT-style mutex on active tasks + if mutex and 'same_mutex' in hard_kinds: + rows = db.execute( + '''SELECT * FROM catalog_task_bindings + WHERE mutex=? AND task != ?''', + (mutex, task), + ).fetchall() + active_ids = {t['id'] for t in _active_tasks(db)} + for row in rows: + if row['task'] not in active_ids and row['task'] not in active_ids: + # Still hard if binding exists and peer task is active + pass + if row['task'] in active_ids: + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=row['requirement'], + target_task=row['task'], target_feature=row['feature'], + level='hard', kind='same_mutex', score=1.0, + evidence=f'Mutex {mutex} held by active task {row["task"]}', + created=now, + ) + hard.append(hit) + + # L1: same_scope — path overlaps with other active task scopes + if scopes and overlaps_fn and 'same_scope' in hard_kinds: + for peer in _active_tasks(db): + if peer['id'] == task: + continue + peer_scopes = json.loads(peer['scopes']) if isinstance(peer['scopes'], str) else peer['scopes'] + if _path_overlaps(scopes, peer_scopes, overlaps_fn): + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=None, + target_task=peer['id'], target_feature=None, + level='hard', kind='same_scope', score=1.0, + evidence=f'Scope overlaps active task {peer["id"]} scopes={peer_scopes}', + created=now, + ) + hard.append(hit) + + # L2: semantic FTS5 + threshold = float(CATALOG_GATE.get('bm25_threshold', DEFAULT_GATE['bm25_threshold'])) + hits = search(db, statement, top=top) + for row in hits: + if requirement and row['id'] == requirement: + continue + if row.get('source_task') == task: + continue + score = float(row.get('score') or 0.0) + # bm25: lower (more negative) is better; accept scores <= threshold + if score > threshold and score >= 0: + # Positive bm25 is weak; still include if phrase matched and score is finite + pass + # Always record soft semantic hits that MATCH returned (already ranked) + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=row['id'], + target_task=row.get('source_task'), target_feature=row.get('feature'), + level='soft', kind='semantic', score=score, + evidence=f'FTS5 hit title={row["title"]!r} bm25={score:.4f}', + created=now, + ) + soft.append(hit) + + # L3: optional vector enhancement (fail-open) + vector_status = 'disabled' + if CATALOG_VECTOR.get('enabled'): + vector_status, vec_soft = _vector_soft_hits( + db, task=task, statement=statement, requirement=requirement, top=top, created=now, + ) + # Merge unique soft hits + seen = {h['id'] for h in soft} + for h in vec_soft: + if h['id'] not in seen: + soft.append(h) + seen.add(h['id']) + + # Soft hits need ack — filter already-acked + pending_soft = [h for h in soft if not _decision_exists(db, task, h['id'])] + status = 'conflict' if hard else ('ack_required' if pending_soft else 'none') + return { + 'status': status, + 'hard': hard, + 'soft': soft, + 'pending_soft': pending_soft, + 'vector_layer': vector_status, + 'task': task, + 'requirement': requirement, + } + + +def _persist_overlap( + db: sqlite3.Connection, + *, + probe_task: str, + probe_req: Optional[str], + target_req: Optional[str], + target_task: Optional[str], + target_feature: Optional[str], + level: str, + kind: str, + score: Optional[float], + evidence: str, + created: float, +) -> dict: + oid = str(uuid.uuid4()) + db.execute( + 'INSERT INTO catalog_overlaps VALUES(?,?,?,?,?,?,?,?,?,?,?)', + ( + oid, probe_task, probe_req, target_req, target_task, target_feature, + level, kind, score, evidence, created, + ), + ) + return { + 'id': oid, 'probe_task': probe_task, 'probe_req': probe_req, + 'target_req': target_req, 'target_task': target_task, + 'target_feature': target_feature, 'level': level, 'kind': kind, + 'score': score, 'evidence': evidence, 'created': created, + } + + +def _decision_exists(db: sqlite3.Connection, task: str, overlap_id: str) -> bool: + row = db.execute( + 'SELECT id FROM catalog_decisions WHERE task=? AND overlap_id=?', + (task, overlap_id), + ).fetchone() + return bool(row) + + +def overlap_ack( + db: sqlite3.Connection, + *, + task: str, + overlap_id: str, + decision: str, + decided_by: str, + reason: str, + authorization_ref: Optional[str] = None, +) -> dict: + if decision not in DECISIONS: + _fail('INVALID_REQUEST', 'Unknown decision', decision=decision) + if not reason.strip(): + _fail('INVALID_REQUEST', 'reason is required for overlap ack') + ov = db.execute('SELECT * FROM catalog_overlaps WHERE id=?', (overlap_id,)).fetchone() + if not ov: + _fail('UNKNOWN_OVERLAP', 'Overlap id not found', id=overlap_id) + if ov['probe_task'] != task: + _fail('PATH_OUT_OF_SCOPE', 'Overlap belongs to another task', holder=ov['probe_task']) + if ov['level'] == 'hard' and decision == 'proceed': + _fail( + 'REQUIREMENT_CONFLICT', + 'Hard overlaps cannot be acknowledged as proceed; resolve mutex/feature/scope first', + overlap_id=overlap_id, + kind=ov['kind'], + holder=ov['target_task'], + ) + if _decision_exists(db, task, overlap_id): + return {'status': 'replayed', 'overlap_id': overlap_id} + did = str(uuid.uuid4()) + db.execute( + 'INSERT INTO catalog_decisions VALUES(?,?,?,?,?,?,?,?)', + (did, overlap_id, task, decision, decided_by, reason.strip(), authorization_ref, _now()), + ) + return {'status': 'acked', 'id': did, 'overlap_id': overlap_id, 'decision': decision} + + +def gate_task_register( + db: sqlite3.Connection, + *, + task: str, + scopes: list[str], + requirement: Optional[str] = None, + intent: Optional[str] = None, + feature: Optional[str] = None, + mutex: Optional[str] = None, + overlaps_fn: Optional[Callable] = None, + statement: Optional[str] = None, +) -> dict: + """Probe overlaps for task register. Caller must commit then enforce deny codes. + + Returns probe result. Does not raise on soft/hard — register path commits first + so soft overlap rows survive OVERLAP_ACK_REQUIRED. + """ + stmt = statement or intent or '' + if not (requirement or stmt or feature or mutex): + return {'status': 'skipped', 'reason': 'no requirement/intent/feature/mutex provided'} + result = probe_overlap( + db, task=task, statement=stmt or (requirement or feature or mutex or task), + requirement=requirement, feature=feature, mutex=mutex, + scopes=scopes, overlaps_fn=overlaps_fn, + ) + bind_task(db, task=task, requirement=requirement, intent=intent, mutex=mutex, feature=feature) + return result + + +def enforce_gate_result(result: dict) -> None: + """Raise deny codes after the probe transaction has committed.""" + if result.get('status') == 'skipped': + return + hard = result.get('hard') or [] + if hard: + first = hard[0] + _fail( + 'REQUIREMENT_CONFLICT', + first['evidence'], + kind=first['kind'], + holder=first.get('target_task'), + overlap_id=first['id'], + target_req=first.get('target_req'), + ) + if CATALOG_GATE.get('ack_required', True) and result.get('pending_soft'): + _fail( + 'OVERLAP_ACK_REQUIRED', + 'Semantic overlaps require catalog overlap ack before coding', + pending=[h['id'] for h in result['pending_soft']], + count=len(result['pending_soft']), + ) + + +def pending_acks_for_task(db: sqlite3.Connection, task: str) -> list[dict]: + rows = db.execute( + '''SELECT o.* FROM catalog_overlaps o + WHERE o.probe_task=? AND o.level='soft' + AND NOT EXISTS ( + SELECT 1 FROM catalog_decisions d + WHERE d.overlap_id=o.id AND d.task=o.probe_task + ) + ORDER BY o.created''', + (task,), + ).fetchall() + return [dict(r) for r in rows] + + +def assert_acks_clear(db: sqlite3.Connection, task: str) -> None: + """Block first write (with) when soft overlaps are unacked.""" + if not CATALOG_GATE.get('ack_required', True): + return + pending = pending_acks_for_task(db, task) + if pending: + _fail( + 'OVERLAP_ACK_REQUIRED', + 'Unresolved soft overlaps; run catalog overlap ack', + pending=[p['id'] for p in pending], + count=len(pending), + ) + + +def catalog_status(db: sqlite3.Connection) -> dict: + def count(table: str) -> int: + try: + return db.execute(f'SELECT COUNT(*) FROM {table}').fetchone()[0] + except sqlite3.Error: + return 0 + + return { + 'status': 'ok', + 'requirements': count('catalog_requirements'), + 'features': count('catalog_features'), + 'overlaps': count('catalog_overlaps'), + 'decisions': count('catalog_decisions'), + 'issues': count('catalog_issues'), + 'tests': count('catalog_tests'), + 'releases': count('catalog_releases'), + 'embeddings': count('catalog_embeddings'), + 'gate': dict(CATALOG_GATE), + 'vector': { + 'enabled': CATALOG_VECTOR.get('enabled'), + 'fail_open': CATALOG_VECTOR.get('fail_open'), + }, + } + + +def catalog_drift(db: sqlite3.Connection, *, matrix_states: dict[str, str]) -> dict: + """Compare catalog_features.matrix_status to live MATRIX states; never writes docs.""" + regressions = [] + for fid, status in matrix_states.items(): + row = db.execute('SELECT matrix_status FROM catalog_features WHERE id=?', (fid,)).fetchone() + if row and row['matrix_status'] == 'done' and status != 'done': + regressions.append({'record_id': fid, 'catalog': row['matrix_status'], 'matrix': status}) + feature_upsert(db, feature_id=fid, title=fid, matrix_status=status) + open_caps = [k for k, v in matrix_states.items() if v not in ('done', 'removed')] + return { + 'status': 'ok', + 'record_count': len(matrix_states), + 'regressions': regressions, + 'open_capabilities': open_caps, + 'note': 'catalog features refreshed from MATRIX; never auto-promotes done', + } + + +# --- Vector layer (fail-open) ------------------------------------------------- + +def _pack_vec(values: list[float]) -> bytes: + return struct.pack(f'<{len(values)}f', *values) + + +def _unpack_vec(blob: bytes) -> list[float]: + n = len(blob) // 4 + return list(struct.unpack(f'<{n}f', blob)) + + +def _cosine(a: list[float], b: list[float]) -> float: + if not a or not b or len(a) != len(b): + return 0.0 + dot = sum(x * y for x, y in zip(a, b)) + na = math.sqrt(sum(x * x for x in a)) + nb = math.sqrt(sum(y * y for y in b)) + if na == 0 or nb == 0: + return 0.0 + return dot / (na * nb) + + +def _embed_text(text: str) -> tuple[str, Optional[list[float]], str]: + """Return (status, vector_or_None, model). Never raises to caller for network errors.""" + url = os.environ.get('EMBEDDING_ENDPOINT_URL', '').strip() + model = os.environ.get('EMBEDDING_MODEL', 'bge-m3').strip() or 'bge-m3' + key = os.environ.get('EMBEDDING_API_KEY', '').strip() + if not CATALOG_VECTOR.get('enabled'): + return 'disabled', None, model + if not url or not key: + return 'degraded', None, model + timeout = int(CATALOG_VECTOR.get('timeout_seconds', 8)) + payload = json.dumps({'model': model, 'input': text}).encode('utf-8') + req = urllib.request.Request( + url, data=payload, method='POST', + headers={'Content-Type': 'application/json', 'Authorization': f'Bearer {key}'}, + ) + try: + with urllib.request.urlopen(req, timeout=timeout) as resp: + if resp.status != 200: + return 'degraded', None, model + body = json.loads(resp.read().decode('utf-8')) + data = body.get('data') or [] + if not data or 'embedding' not in data[0]: + return 'degraded', None, model + return 'ok', list(data[0]['embedding']), model + except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, json.JSONDecodeError, KeyError, IndexError, TypeError): + return 'degraded', None, model + + +def store_embedding(db: sqlite3.Connection, req_id: str, text: str) -> dict: + status, vec, model = _embed_text(text) + if status != 'ok' or vec is None: + return {'status': status, 'req_id': req_id, 'model': model} + blob = _pack_vec(vec) + sha = hashlib.sha256(text.encode('utf-8')).hexdigest() + db.execute( + '''INSERT INTO catalog_embeddings(req_id, model, dim, vec, source_sha256, status, created) + VALUES(?,?,?,?,?,?,?) + ON CONFLICT(req_id) DO UPDATE SET + model=excluded.model, dim=excluded.dim, vec=excluded.vec, + source_sha256=excluded.source_sha256, status=excluded.status, + created=excluded.created''', + (req_id, model, len(vec), blob, sha, 'ok', _now()), + ) + return {'status': 'ok', 'req_id': req_id, 'dim': len(vec), 'model': model} + + +def _vector_soft_hits( + db: sqlite3.Connection, + *, + task: str, + statement: str, + requirement: Optional[str], + top: int, + created: float, +) -> tuple[str, list[dict]]: + status, probe, model = _embed_text(statement) + if status != 'ok' or probe is None: + return status, [] + threshold = float(CATALOG_VECTOR.get('cosine_threshold', 0.82)) + rows = db.execute( + "SELECT req_id, vec FROM catalog_embeddings WHERE status='ok'" + ).fetchall() + scored: list[tuple[float, str]] = [] + for row in rows: + if requirement and row['req_id'] == requirement: + continue + other = _unpack_vec(row['vec']) + sim = _cosine(probe, other) + if sim >= threshold: + scored.append((sim, row['req_id'])) + scored.sort(reverse=True) + soft = [] + for sim, rid in scored[:top]: + req = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (rid,)).fetchone() + if not req: + continue + if req['source_task'] == task: + continue + hit = _persist_overlap( + db, probe_task=task, probe_req=requirement, target_req=rid, + target_task=req['source_task'], target_feature=req['feature'], + level='soft', kind='semantic', score=sim, + evidence=f'vector cosine={sim:.4f} model={model}', + created=created, + ) + soft.append(hit) + return 'ok', soft diff --git a/scripts/catalog_backfill.py b/scripts/catalog_backfill.py new file mode 100644 index 00000000..f5cfcf38 --- /dev/null +++ b/scripts/catalog_backfill.py @@ -0,0 +1,214 @@ +#!/usr/bin/env python3 +"""One-shot backfill of catalog tables from Markdown / SQLite sources. + +Read-only against .doc and tasks table; writes only catalog_* tables. +Does not touch product PostgreSQL. Safe to re-run (upserts / replaces by PK). +""" +from __future__ import annotations + +import argparse +import json +import re +import sys +import time +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +sys.dont_write_bytecode = True +import agentctl # noqa: E402 +import agentctl_catalog as catalog # noqa: E402 + + +def table_rows(content: str) -> list[list[str]]: + rows = [] + for line in content.splitlines(): + if not line.lstrip().startswith('|'): + continue + cells = [c.strip() for c in re.split(r'(? int: + path = repo.main / '.doc/status/FEATURE-MATRIX.md' + if not path.is_file(): + return 0 + content = path.read_text(encoding='utf-8') + states = agentctl.matrix_states(content) + # Also capture mutex/title from table when possible. + n = 0 + header = None + for cells in table_rows(content): + if header is None: + if cells and cells[0].casefold() in ('record_id', 'id'): + header = [c.casefold() for c in cells] + continue + if len(cells) != len(header): + continue + rid = cells[0] + title = cells[1] if len(cells) > 1 else rid + mutex = None + if 'mutex' in header: + mutex = cells[header.index('mutex')].strip('`') + status = states.get(rid) + catalog.feature_upsert(db, feature_id=rid, title=title, mutex=mutex, matrix_status=status) + n += 1 + return n + + +def backfill_issues(db, repo: agentctl.Repo) -> int: + path = repo.main / '.doc/audit/ISSUES.md' + if not path.is_file(): + return 0 + content = path.read_text(encoding='utf-8') + n = 0 + header = None + now = time.time() + for cells in table_rows(content): + if header is None: + if cells and 'issue_id' in cells[0].casefold(): + header = [c.casefold() for c in cells] + continue + if not cells or not cells[0].startswith('ISSUE-'): + continue + issue_id = cells[0] + module = cells[1] if len(cells) > 1 else '' + severity = cells[3] if len(cells) > 3 else '' + state = cells[4] if len(cells) > 4 else '' + owner = cells[5] if len(cells) > 5 else '' + main_record = cells[7] if len(cells) > 7 else '' + lesson = cells[8] if len(cells) > 8 else '' + db.execute( + '''INSERT INTO catalog_issues( + issue_id, severity, issue_state, owner, module, keywords, + main_record, lesson_ids, requirements, tasks, recurrence_count, + evidence_basis, created, updated) + VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?) + ON CONFLICT(issue_id) DO UPDATE SET + severity=excluded.severity, issue_state=excluded.issue_state, + owner=excluded.owner, module=excluded.module, + main_record=excluded.main_record, lesson_ids=excluded.lesson_ids, + updated=excluded.updated''', + ( + issue_id, severity, state, owner, module, module, + main_record, lesson, None, None, 0, 'historical', now, now, + ), + ) + n += 1 + return n + + +def guess_verdict(text: str) -> str: + upper = text.upper() + if 'NOT REQUIRED' in upper or '不记成产品测试 PASS' in text: + return 'NOT_REQUIRED' + # Prefer explicit Evaluation / Verdict sections. + if re.search(r'\bFAIL\b', upper) and 'PASS' not in upper.split('FAIL')[0][-80:]: + if re.search(r'判定[：:].{0,40}FAIL|Verdict[：:].{0,20}FAIL|评价判定.{0,40}FAIL', text, re.I): + return 'FAIL' + if re.search(r'判定[：:].{0,40}PASS|Verdict[：:].{0,20}PASS|评价判定.{0,40}PASS', text, re.I): + return 'PASS' + if '**PASS**' in text or '| PASS |' in text: + return 'PASS' + return 'unknown' + + +def backfill_tests(db, repo: agentctl.Repo) -> int: + root = repo.main / '.doc/test-reports' + if not root.is_dir(): + return 0 + n = 0 + now = time.time() + for path in sorted(root.glob('*.md')): + if path.name in ('INDEX.md', 'README.md'): + continue + text = path.read_text(encoding='utf-8', errors='replace') + rel = path.relative_to(repo.main).as_posix() + slug = path.stem + verdict = guess_verdict(text) + real = 1 if re.search(r'真实\s*PI|pi-agent|MiniMax', text, re.I) else 0 + level = 'doc_only' if verdict == 'NOT_REQUIRED' else 'e2e' + db.execute( + '''INSERT INTO catalog_tests( + id, report_path, task, requirements, scope, verdict, level, + real_agent, schema_version, env, sha, created) + VALUES(?,?,?,?,?,?,?,?,?,?,?,?) + ON CONFLICT(id) DO UPDATE SET + report_path=excluded.report_path, verdict=excluded.verdict, + real_agent=excluded.real_agent''', + (slug, rel, None, None, None, verdict, level, real, None, None, None, now), + ) + n += 1 + return n + + +def backfill_tasks_as_features(db) -> int: + rows = db.execute('SELECT id, scopes, state FROM tasks').fetchall() + n = 0 + for row in rows: + # Light projection: task id as feature hint only when looks like feature mutex. + scopes = json.loads(row['scopes']) if isinstance(row['scopes'], str) else row['scopes'] + mutex = scopes[0] if scopes else row['id'] + catalog.feature_upsert( + db, + feature_id=f'task:{row["id"]}', + title=row['id'], + scope=','.join(scopes), + mutex=mutex, + matrix_status=None, + ) + n += 1 + return n + + +def backfill_deployments(db) -> int: + try: + rows = db.execute('SELECT * FROM deployments').fetchall() + except Exception: + return 0 + n = 0 + now = time.time() + for row in rows: + db.execute( + '''INSERT INTO catalog_releases( + id, env, branch, commit_sha, tag, deploy_seq, schema_version, + image_digest, source_task, phase, receipt, created) + VALUES(?,?,?,?,?,?,?,?,?,?,?,?) + ON CONFLICT(id) DO UPDATE SET phase=excluded.phase, receipt=excluded.receipt''', + ( + row['id'], row['env'], None, row['expected_commit'], None, None, None, + None, row['task'], row['phase'], row['receipt'], row['created'] or now, + ), + ) + n += 1 + return n + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--repo', default='.') + args = parser.parse_args() + repo = agentctl.Repo(args.repo) + agentctl.require_policy(repo) + catalog.bind_deny(agentctl.deny) + with repo.db(True) as db: + counts = { + 'features_matrix': backfill_matrix(db, repo), + 'issues': backfill_issues(db, repo), + 'tests': backfill_tests(db, repo), + 'task_features': backfill_tasks_as_features(db), + 'releases': backfill_deployments(db), + } + status = catalog.catalog_status(db) + agentctl.emit({'status': 'ok', 'backfill': counts, 'catalog': status}) + return 0 + + +if __name__ == '__main__': + try: + sys.exit(main()) + except agentctl.Denied as exc: + agentctl.emit({'status': 'denied', 'code': exc.code, 'message': exc.message, **exc.details}, sys.stderr) + sys.exit(1) diff --git a/scripts/mcp/agentctl-mcp.mjs b/scripts/mcp/agentctl-mcp.mjs index 4a9b25da..e1ea8d80 100755 --- a/scripts/mcp/agentctl-mcp.mjs +++ b/scripts/mcp/agentctl-mcp.mjs @@ -153,6 +153,76 @@ const TOOLS = [ description: 'List quarantine / suspect runs', inputSchema: { type: 'object', properties: {}, additionalProperties: false }, }, + { + name: 'catalog_requirement_add', + description: 'Add a catalog requirement (R-ID)', + inputSchema: { + type: 'object', + properties: { + id: { type: 'string' }, + title: { type: 'string' }, + statement: { type: 'string' }, + owner: { type: 'string' }, + feature: { type: 'string' }, + source_doc: { type: 'string' }, + source_task: { type: 'string' }, + }, + required: ['id', 'title', 'statement', 'owner'], + additionalProperties: false, + }, + }, + { + name: 'catalog_overlap_probe', + description: 'Probe hard/soft requirement overlaps for a task', + inputSchema: { + type: 'object', + properties: { + task: { type: 'string' }, + statement: { type: 'string' }, + requirement: { type: 'string' }, + feature: { type: 'string' }, + mutex: { type: 'string' }, + top: { type: 'number' }, + }, + required: ['task', 'statement'], + additionalProperties: false, + }, + }, + { + name: 'catalog_overlap_ack', + description: 'Acknowledge a soft overlap with an explicit decision', + inputSchema: { + type: 'object', + properties: { + task: { type: 'string' }, + overlap: { type: 'string' }, + decision: { type: 'string', enum: ['proceed', 'merge_into', 'supersede', 'defer', 'reject'] }, + reason: { type: 'string' }, + decided_by: { type: 'string' }, + authorization_ref: { type: 'string' }, + }, + required: ['task', 'overlap', 'decision', 'reason'], + additionalProperties: false, + }, + }, + { + name: 'catalog_search', + description: 'FTS5 CJK search over catalog requirements', + inputSchema: { + type: 'object', + properties: { + query: { type: 'string' }, + top: { type: 'number' }, + }, + required: ['query'], + additionalProperties: false, + }, + }, + { + name: 'catalog_status', + description: 'Catalog table counts and gate/vector config summary', + inputSchema: { type: 'object', properties: {}, additionalProperties: false }, + }, ]; function callTool(name, args = {}) { @@ -217,6 +287,52 @@ function callTool(name, args = {}) { const r = runAgentctl(['quarantine', 'list']); return r.ok ? textResult(r.stdout.trim() || '{"items":[]}') : errResult(r.stderr || r.stdout || r.error); } + case 'catalog_requirement_add': { + const argv = [ + 'catalog', 'add', + '--id', args.id, + '--title', args.title, + '--statement', args.statement, + '--owner', args.owner, + ]; + if (args.feature) argv.push('--feature', args.feature); + if (args.source_doc) argv.push('--source-doc', args.source_doc); + if (args.source_task) argv.push('--source-task', args.source_task); + const r = runAgentctl(argv); + return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error); + } + case 'catalog_overlap_probe': { + const argv = ['catalog', 'overlap', '--task', args.task, '--statement', args.statement]; + if (args.requirement) argv.push('--requirement', args.requirement); + if (args.feature) argv.push('--feature', args.feature); + if (args.mutex) argv.push('--mutex', args.mutex); + if (args.top != null) argv.push('--top', String(args.top)); + const r = runAgentctl(argv); + return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error); + } + case 'catalog_overlap_ack': { + const argv = [ + 'catalog', 'ack', + '--task', args.task, + '--overlap', args.overlap, + '--decision', args.decision, + '--reason', args.reason, + ]; + if (args.decided_by) argv.push('--decided-by', args.decided_by); + if (args.authorization_ref) argv.push('--authorization-ref', args.authorization_ref); + const r = runAgentctl(argv); + return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error); + } + case 'catalog_search': { + const argv = ['catalog', 'search', '--query', args.query]; + if (args.top != null) argv.push('--top', String(args.top)); + const r = runAgentctl(argv); + return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error); + } + case 'catalog_status': { + const r = runAgentctl(['catalog', 'status']); + return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error); + } default: return errResult(`unknown tool: ${name}`); } diff --git a/scripts/tests/test_agent_worktree.py b/scripts/tests/test_agent_worktree.py index 84d32ec1..f0578e31 100644 --- a/scripts/tests/test_agent_worktree.py +++ b/scripts/tests/test_agent_worktree.py @@ -24,7 +24,7 @@ class WorktreeTest(unittest.TestCase): self.root = Path(self.tmp.name) / "repo" self.root.mkdir() (self.root / "scripts").mkdir() - for name in ("agent-worktree.sh", "agent_worktree.py", "agentctl.py", "agentctl", "agentctl_w4.py"): + for name in ("agent-worktree.sh", "agent_worktree.py", "agentctl.py", "agentctl", "agentctl_w4.py", "agentctl_catalog.py"): shutil.copy2(SOURCE / name, self.root / "scripts" / name) (self.root / "configs").mkdir(parents=True, exist_ok=True) shutil.copy2(SOURCE.parent / "configs" / "agent-governance.json", diff --git a/scripts/tests/test_agentctl.py b/scripts/tests/test_agentctl.py index 86cfa629..65e0c55b 100644 --- a/scripts/tests/test_agentctl.py +++ b/scripts/tests/test_agentctl.py @@ -37,6 +37,7 @@ class AgentctlTests(unittest.TestCase): (self.root / "scripts").mkdir() shutil.copy2(SOURCE, self.root / "scripts" / "agentctl.py") shutil.copy2(SOURCE.parent / "agentctl_w4.py", self.root / "scripts" / "agentctl_w4.py") + shutil.copy2(SOURCE.parent / "agentctl_catalog.py", self.root / "scripts" / "agentctl_catalog.py") install_governance_policy(self.root) (self.root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8") for directory in ("src/a", "src/b", ".doc/ops", ".doc/status"): @@ -1551,6 +1552,7 @@ class ProcessPlatformTests(unittest.TestCase): (root / "scripts").mkdir() shutil.copy2(SOURCE, root / "scripts" / "agentctl.py") shutil.copy2(SOURCE.parent / "agentctl_w4.py", root / "scripts" / "agentctl_w4.py") + shutil.copy2(SOURCE.parent / "agentctl_catalog.py", root / "scripts" / "agentctl_catalog.py") install_governance_policy(root) (root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8") (root / ".doc/ops").mkdir(parents=True) diff --git a/scripts/tests/test_agentctl_catalog.py b/scripts/tests/test_agentctl_catalog.py new file mode 100644 index 00000000..35a735c2 --- /dev/null +++ b/scripts/tests/test_agentctl_catalog.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python3 +"""Catalog governance regressions (schema, FTS5 CJK, overlap gates, vector degrade).""" + +import json +import pathlib +import shutil +import sqlite3 +import subprocess +import tempfile +import unittest + +SOURCE = pathlib.Path(__file__).resolve().parents[1] / "agentctl.py" +CATALOG_SOURCE = pathlib.Path(__file__).resolve().parents[1] / "agentctl_catalog.py" +POLICY_SOURCE = SOURCE.parents[1] / "configs" / "agent-governance.json" + + +def install_governance_policy(root: pathlib.Path) -> None: + dest = root / "configs" + dest.mkdir(parents=True, exist_ok=True) + shutil.copy2(POLICY_SOURCE, dest / "agent-governance.json") + + +class CatalogTests(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory(prefix="agentctl-catalog-") + self.addCleanup(self.temp.cleanup) + self.root = pathlib.Path(self.temp.name) / "repo" + self.root.mkdir() + self.git("init", "-b", "dev") + self.git("config", "user.name", "Catalog Test") + self.git("config", "user.email", "catalog-test@example.invalid") + (self.root / "scripts").mkdir() + shutil.copy2(SOURCE, self.root / "scripts" / "agentctl.py") + shutil.copy2(SOURCE.parent / "agentctl_w4.py", self.root / "scripts" / "agentctl_w4.py") + shutil.copy2(CATALOG_SOURCE, self.root / "scripts" / "agentctl_catalog.py") + install_governance_policy(self.root) + (self.root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8") + for directory in ("src/a", "src/b", ".doc/ops", ".doc/status"): + (self.root / directory).mkdir(parents=True, exist_ok=True) + (self.root / ".doc/ops/CURRENT.md").write_text( + "# Coordination\n\n| owner | task | branch | worktree | mutex | status |\n" + "| --- | --- | --- | --- | --- | --- |\n" + "| Alice | alpha | feature/alpha | .worktrees/alpha | src | in_progress |\n", + encoding="utf-8", + ) + (self.root / ".doc/status/FEATURE-MATRIX.md").write_text( + "# Features\n\n| record_id | 模块 | 状态 |\n| --- | --- | --- |\n" + "| core | core | planned |\n| intent-plan | 多意图 | done |\n", + encoding="utf-8", + ) + self.git("add", ".") + self.git("commit", "-m", "fixture") + self.ctl("init") + + def git(self, *args, cwd=None): + return subprocess.run( + ["git", *args], cwd=cwd or self.root, text=True, capture_output=True, check=True, + ).stdout.strip() + + def ctl(self, *args, check=True, cwd=None): + result = subprocess.run( + ["python3", "-B", str(self.root / "scripts" / "agentctl.py"), *args], + cwd=cwd or self.root, text=True, capture_output=True, + ) + if check and result.returncode != 0: + raise AssertionError(result.stderr or result.stdout or f"exit {result.returncode}") + out = (result.stdout or "").strip() + err = (result.stderr or "").strip() + payload = None + for stream in (out, err): + if not stream: + continue + try: + payload = json.loads(stream.splitlines()[-1]) + except json.JSONDecodeError: + pass + return result.returncode, payload, out, err + + def make_worktree(self, task_id: str): + wt = self.root / ".worktrees" / task_id + self.git("worktree", "add", "-b", f"feature/{task_id}", str(wt)) + return wt + + def test_schema_and_crud(self): + code, payload, _, _ = self.ctl( + "catalog", "add", + "--id", "R-demo-overlap", + "--title", "多意图持久计划", + "--statement", "多意图持久计划 shadow 灰度放量", + "--owner", "Cursor", + "--feature", "intent-plan", + ) + self.assertEqual(code, 0) + self.assertEqual(payload["status"], "added") + _, listed, _, _ = self.ctl("catalog", "list") + self.assertEqual(listed["status"], "ok") + self.assertEqual(len(listed["requirements"]), 1) + _, shown, _, _ = self.ctl("catalog", "show", "--id", "R-demo-overlap") + self.assertEqual(shown["requirement"]["id"], "R-demo-overlap") + _, status, _, _ = self.ctl("catalog", "status") + self.assertGreaterEqual(status["requirements"], 1) + + def test_cjk_fts_digram_recall(self): + """CJK digrams must hit under unicode61 char-split (trigram would miss 灰度).""" + self.ctl( + "catalog", "add", + "--id", "R-gray", + "--title", "灰度放量", + "--statement", "多意图持久计划 shadow 灰度放量", + "--owner", "Cursor", + ) + self.ctl( + "catalog", "add", + "--id", "R-other", + "--title", "语义摘要", + "--statement", "语义摘要 REPLACE fail-closed 灰度", + "--owner", "Cursor", + ) + # Direct FTS verification including trigram contrast. + import sys + sys.path.insert(0, str(self.root / "scripts")) + import agentctl_catalog as cat + db = sqlite3.connect(self.root / ".locks" / "state.sqlite") + db.row_factory = sqlite3.Row + # Create trigram index for contrast. + db.execute("CREATE VIRTUAL TABLE IF NOT EXISTS trigram_probe USING fts5(body, tokenize='trigram')") + db.execute("INSERT INTO trigram_probe VALUES (?)", ("多意图持久计划 shadow 灰度放量",)) + db.execute("INSERT INTO trigram_probe VALUES (?)", ("语义摘要 REPLACE fail-closed 灰度",)) + tri = db.execute("SELECT count(*) FROM trigram_probe WHERE trigram_probe MATCH ?", ('"灰度"',)).fetchone()[0] + self.assertEqual(tri, 0, "trigram must miss digram 灰度") + hits = cat.search(db, "灰度", top=10) + self.assertGreaterEqual(len(hits), 2, f"CJK split must recall digram; got {hits}") + db.close() + _, payload, _, _ = self.ctl("catalog", "search", "--query", "灰度") + self.assertGreaterEqual(len(payload["hits"]), 2) + + def test_hard_same_req_id_conflict(self): + self.ctl( + "catalog", "add", + "--id", "R-owned", + "--title", "owned", + "--statement", "already owned requirement", + "--owner", "Alice", + "--source-task", "alpha-owner", + ) + wt = self.make_worktree("beta-conflict") + code, payload, _, err = self.ctl( + "task", "register", + "--id", "beta-conflict", + "--owner", "Bob", + "--scope", "src/b", + "--requirement", "R-owned", + "--intent", "already owned requirement", + cwd=wt, + check=False, + ) + self.assertNotEqual(code, 0) + denied = payload or json.loads(err.splitlines()[-1]) + self.assertEqual(denied["code"], "REQUIREMENT_CONFLICT") + self.assertEqual(denied["kind"], "same_req_id") + + def test_soft_overlap_requires_ack(self): + self.ctl( + "catalog", "add", + "--id", "R-existing", + "--title", "治理目录库", + "--statement", "跨对话需求重叠检测与治理目录库", + "--owner", "Alice", + "--source-task", "alpha-owner", + ) + wt = self.make_worktree("soft-probe") + # Register without catalog args first. + self.ctl( + "task", "register", + "--id", "soft-probe", + "--owner", "Bob", + "--scope", "src/b", + cwd=wt, + ) + code, payload, _, _ = self.ctl( + "catalog", "overlap", + "--task", "soft-probe", + "--statement", "跨对话需求重叠检测治理目录", + "--requirement", "R-new-soft", + ) + self.assertEqual(code, 0) + self.assertEqual(payload["status"], "ack_required") + self.assertTrue(payload["pending_soft"]) + oid = payload["pending_soft"][0]["id"] + # Exclusive with blocked until ack. + code, denied, _, err = self.ctl( + "with", "--task", "soft-probe", "--resource", "path:src/b", "--", "true", + cwd=wt, check=False, + ) + self.assertNotEqual(code, 0) + body = denied or json.loads(err.splitlines()[-1]) + self.assertEqual(body["code"], "OVERLAP_ACK_REQUIRED") + self.ctl( + "catalog", "ack", + "--task", "soft-probe", + "--overlap", oid, + "--decision", "proceed", + "--reason", "incremental extension of same theme", + "--decided-by", "Bob", + ) + code, _, _, _ = self.ctl( + "with", "--task", "soft-probe", "--resource", "path:src/b", "--", "true", + cwd=wt, + ) + self.assertEqual(code, 0) + + def test_vector_layer_degrades_without_key(self): + # Enable vector in policy for this repo. + policy_path = self.root / "configs" / "agent-governance.json" + data = json.loads(policy_path.read_text(encoding="utf-8")) + data["catalog_vector"] = { + "enabled": True, "timeout_seconds": 2, "fail_open": True, "cosine_threshold": 0.82, + } + policy_path.write_text(json.dumps(data, indent=2), encoding="utf-8") + self.ctl( + "catalog", "add", + "--id", "R-vec", + "--title", "向量降级", + "--statement", "向量库可降级不阻断门禁", + "--owner", "Cursor", + ) + # No EMBEDDING_API_KEY → degraded, still returns FTS soft or none without crash. + env = {"EMBEDDING_ENDPOINT_URL": "http://127.0.0.1:9/embeddings", "EMBEDDING_API_KEY": ""} + result = subprocess.run( + ["python3", "-B", str(self.root / "scripts" / "agentctl.py"), + "catalog", "overlap", "--task", "no-task-yet", "--statement", "向量库可降级不阻断"], + cwd=self.root, text=True, capture_output=True, + env={**dict(**{k: v for k, v in __import__('os').environ.items()}), **env}, + ) + self.assertEqual(result.returncode, 0, result.stderr) + payload = json.loads(result.stdout.strip().splitlines()[-1]) + self.assertIn(payload.get("vector_layer"), ("degraded", "disabled", "ok")) + + def test_hard_proceed_ack_rejected(self): + self.ctl( + "catalog", "add", + "--id", "R-hard", + "--title", "mutex feature", + "--statement", "same feature ownership", + "--owner", "Alice", + "--feature", "core", + "--source-task", "holder-a", + ) + wt = self.make_worktree("hard-ack") + self.ctl("task", "register", "--id", "hard-ack", "--owner", "Bob", "--scope", "src/b", cwd=wt) + _, payload, _, _ = self.ctl( + "catalog", "overlap", + "--task", "hard-ack", + "--statement", "same feature ownership", + "--feature", "core", + "--requirement", "R-hard-b", + ) + self.assertTrue(payload["hard"]) + oid = payload["hard"][0]["id"] + code, denied, _, err = self.ctl( + "catalog", "ack", + "--task", "hard-ack", + "--overlap", oid, + "--decision", "proceed", + "--reason", "should fail", + check=False, + ) + self.assertNotEqual(code, 0) + body = denied or json.loads(err.splitlines()[-1]) + self.assertEqual(body["code"], "REQUIREMENT_CONFLICT") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/tests/test_codegraph_governance.py b/scripts/tests/test_codegraph_governance.py index 0af57a72..0387b78d 100644 --- a/scripts/tests/test_codegraph_governance.py +++ b/scripts/tests/test_codegraph_governance.py @@ -21,7 +21,7 @@ class CodegraphGovernanceTest(unittest.TestCase): self.root.mkdir() (self.root / 'scripts').mkdir() for name in ('agent-worktree.sh', 'agent_worktree.py', 'agentctl.py', - 'agentctl', 'agentctl_w4.py'): + 'agentctl', 'agentctl_w4.py', 'agentctl_catalog.py'): shutil.copy2(SOURCE / name, self.root / 'scripts' / name) (self.root / 'configs').mkdir(parents=True, exist_ok=True) shutil.copy2(SOURCE.parent / 'configs' / 'agent-governance.json', diff --git a/scripts/tests/test_ops_guards.py b/scripts/tests/test_ops_guards.py index ec82474d..4bad4894 100644 --- a/scripts/tests/test_ops_guards.py +++ b/scripts/tests/test_ops_guards.py @@ -19,7 +19,7 @@ class OpsGuardsTest(unittest.TestCase): self.root = Path(self.tmp.name) / "repo" self.root.mkdir() (self.root / "scripts").mkdir() - for name in ("env-service.sh", "candidate-images-build.sh", "agentctl.py", "agentctl", "agentctl_w4.py"): + for name in ("env-service.sh", "candidate-images-build.sh", "agentctl.py", "agentctl", "agentctl_w4.py", "agentctl_catalog.py"): shutil.copy2(SOURCE / name, self.root / "scripts" / name) (self.root / ".gitignore").write_text(".worktrees/\n.locks/\nconfigs/.env.*\n!configs/.env.*.example\ndata/\n") (self.root / ".dockerignore").write_text(".doc\n.locks\n.worktrees\n") diff --git a/scripts/tests/test_pi_subagents.py b/scripts/tests/test_pi_subagents.py index 018dd424..e1ba8c9d 100644 --- a/scripts/tests/test_pi_subagents.py +++ b/scripts/tests/test_pi_subagents.py @@ -146,6 +146,9 @@ class PiSubagentsTests(unittest.TestCase): shutil.copy2(SOURCE_AGENTCTL, self.root / "scripts" / "agentctl.py") if SOURCE_W4.exists(): shutil.copy2(SOURCE_W4, self.root / "scripts" / "agentctl_w4.py") + catalog_src = ROOT / "scripts" / "agentctl_catalog.py" + if catalog_src.exists(): + shutil.copy2(catalog_src, self.root / "scripts" / "agentctl_catalog.py") if SOURCE_EXEC.exists(): shutil.copy2(SOURCE_EXEC, self.root / "scripts" / "pi_subagents.py") if SOURCE_MCP.exists():