merge: candidate governance-catalog into dev
This commit is contained in:
weichen
@@ -13,6 +13,7 @@ alwaysApply: true
|
||||
2. **共享文档**：`.doc/ops/CURRENT.md`、`.doc/status/FEATURE-MATRIX.md` 只能 `docs apply`（行级 CAS；MATRIX 键为 `record_id`）。
|
||||
3. **合入 local dev**：`validate` 后 `queue enqueue`（daemon `--ff-only`）或受控 `integrate`；禁止直接改主目录 `dev` tip。
|
||||
4. **禁止默认**：未授权 `git push`、晋升 test/main、`--force` 清理、卸载 launchd coordinator、改 gap-matrix / 升 MATRIX 状态值冒充发布。
|
||||
5. **需求重叠**：跨对话新设计须 `catalog overlap`；硬重叠拒绝；软重叠须 `catalog ack` 留痕后方可编码（见 `.doc/ops/2026-09-24-governance-catalog-plan.md`）。
|
||||
|
||||
## 会话路径
|
||||
|
||||
|
||||
@@ -29,11 +29,23 @@ codegraph sync -q
|
||||
跨文件定位优先用 CodeGraph MCP/`explore`；**grep 仍是完整性唯一权威**（假阴性时以 grep 为准）。
|
||||
|
||||
4. 若任务已存在：在注册 worktree 内工作，确认 `tasks.state=active`。
|
||||
5. 若新任务：跟 [agentctl-worktree-setup](../agentctl-worktree-setup/SKILL.md)。
|
||||
6. 任何仓库写 → [agentctl-lease-write](../agentctl-lease-write/SKILL.md)；合入 → [agentctl-integrate-closeout](../agentctl-integrate-closeout/SKILL.md)。
|
||||
5. **开工前需求重叠探测**（治理目录库）：新设计或可能与其他对话重叠时：
|
||||
|
||||
```bash
|
||||
python3 scripts/agentctl.py catalog search --query '<关键词>'
|
||||
python3 scripts/agentctl.py catalog overlap --task "$ID" --statement '<设计原文>' [--requirement R-xxx] [--feature <record_id>] [--mutex <key>]
|
||||
# soft 命中须显式裁决后才可编码：
|
||||
python3 scripts/agentctl.py catalog overlap ack --task "$ID" --overlap <oid> --decision proceed|merge_into|supersede|defer --reason '...'
|
||||
```
|
||||
|
||||
硬重叠（同 mutex/feature/scope/R-ID）直接拒绝；向量层失败自动降级到 FTS5，不阻断。
|
||||
|
||||
6. 若新任务：跟 [agentctl-worktree-setup](../agentctl-worktree-setup/SKILL.md)。
|
||||
7. 任何仓库写 → [agentctl-lease-write](../agentctl-lease-write/SKILL.md)；合入 → [agentctl-integrate-closeout](../agentctl-integrate-closeout/SKILL.md)。
|
||||
|
||||
## 禁止
|
||||
|
||||
- 主目录直接开发提交；未授权 `git push` / 晋升 test/main
|
||||
- 绕过 `docs apply` 改 CURRENT / FEATURE-MATRIX
|
||||
- 卸载或重启 launchd coordinator（除非用户明示）
|
||||
- 未执行 `catalog overlap` / 未裁决 soft 命中即编码（有 `--requirement`/`--intent`/`--feature`/`--mutex` 绑定时机器强制）
|
||||
|
||||
@@ -23,9 +23,25 @@ python3 ../../scripts/agentctl.py task register \
|
||||
--scope scripts --scope .doc \
|
||||
--capability docs:publish
|
||||
# 需要整文/行级 admin 时再加 --capability docs:admin；证据升 done 加 docs:approve
|
||||
# 若已知需求/功能绑定，追加：
|
||||
# --requirement R-<slug> --intent '<设计原文>' --feature <record_id> --mutex <key>
|
||||
# 有 soft 重叠时 register 会 OVERLAP_ACK_REQUIRED：先 catalog overlap + ack 再重试 register
|
||||
codegraph sync -q # 若建场已 init，此处只需 sync；缺失 CLI 可跳过
|
||||
```
|
||||
|
||||
## 开工前重叠裁决（catalog）
|
||||
|
||||
```bash
|
||||
python3 ../../scripts/agentctl.py catalog overlap \
|
||||
--task "$ID" --statement '<新设计原文>' \
|
||||
[--requirement R-xxx] [--feature <record_id>] [--mutex <key>]
|
||||
# hard → 换 mutex/合并到持有者任务；soft → ack 留痕
|
||||
python3 ../../scripts/agentctl.py catalog ack \
|
||||
--task "$ID" --overlap <oid> \
|
||||
--decision proceed|merge_into|supersede|defer \
|
||||
--reason '<理由>'
|
||||
```
|
||||
|
||||
## CURRENT 认领（docs apply）
|
||||
|
||||
```bash
|
||||
|
||||
@@ -51,3 +51,8 @@ CURRENT 是带时间的活跃协调快照；Git/机器状态用于核验实际
|
||||
修订 ID：`subagent-workflow-2026-09-24-r2`；owner：Codex；state：`proposed`；能力：`manual`；effective_commit：无。
|
||||
用户要求研究 Cursor Auto 下的 subagent 分工、阶段流程及报告，授权与本次验收见[任务](short-term/2026-09-24-subagent-workflow-standard.md)。[规范草案](2026-09-24-subagent-workflow-standard.md)为拟议行为的唯一来源，[研究报告](../audit/2026-09-24-subagent-workflow-research.md)给出事实和待修冲突，[模板](templates/subagent-stage-record.md)承载交接。
|
||||
用户后续明确采用 Cursor Auto 实施 + gmktec 本机 PI 子任务，并批准默认模型分工及可调整文档。[模型配置](2026-09-24-subagent-model-config.md)为模型路由唯一来源，decision_state=approved；流程整体仍 proposed；已批准的模型配置与执行器已合入本地 dev。原 Cursor 原生角色/inherit 方案被替代。执行器已在任务分支实现并验证 Cursor CLI→MCP→PI，代码 69ffaa677e23a1dd9aa0528bb09b3acfcd38b119；[运行说明](2026-09-24-pi-subagent-executor.md)与[验证报告](../test-reports/2026-09-24-pi-executor-test-report.md)限定已验证的机器边界。dev 集成/完整流程审批仍独立记录；不以派发器可用宣称全阶段自动门禁。
|
||||
|
||||
## 治理目录库与需求重叠检测拟议修订
|
||||
|
||||
修订 ID：`governance-catalog-2026-09-24-r1`；owner：`Cursor`；state：`proposed`；能力：schema/门禁落地后为 `enforced`，向量层 `fail-open`；effective_commit：无（随受控合入记录）。
|
||||
用户授权实施跨对话需求重叠检测与治理目录（扩展既有 `.locks/state.sqlite`，不碰业务 PG）。唯一来源：[方案](./2026-09-24-governance-catalog-plan.md)；任务：[short-term](./short-term/2026-09-24-governance-catalog.md)。策略键 `catalog_gate`/`catalog_vector` 为可选，不得破坏 `shared_docs` 等集合相等校验。
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# 治理目录库与跨对话需求重叠检测（方案）
|
||||
|
||||
**修订 ID**：`governance-catalog-2026-09-24-r1`
|
||||
**state**：`proposed` → 实施后随合入升 `active`
|
||||
**owner**：Cursor
|
||||
**工具能力**：schema/CRUD/门禁 `enforced`（落地后）；向量层 `fail-open`
|
||||
**任务**：[short-term](./short-term/2026-09-24-governance-catalog.md)
|
||||
|
||||
## 问题
|
||||
|
||||
跨对话新设计与既有功能/需求重叠时，现有 agentctl 仅做路径级 `overlaps()` 互斥，无需求语义比对。CURRENT/ISSUES/test-reports 为 Markdown 台账，机器无法强制开工前探测。
|
||||
|
||||
## 方案摘要
|
||||
|
||||
在既有 `.locks/state.sqlite`（与业务 PG 物理隔离）上 additive 扩展 Catalog 表：
|
||||
|
||||
| 表 | 用途 |
|
||||
|---|---|
|
||||
| `catalog_requirements` | R-ID 需求条目 |
|
||||
| `catalog_features` | 对齐 FEATURE-MATRIX record_id |
|
||||
| `catalog_req_text` | FTS5 CJK 逐字索引（禁用 trigram） |
|
||||
| `catalog_overlaps` / `catalog_decisions` | 检测结果与裁决留痕 |
|
||||
| `catalog_issues` / `catalog_tests` / `catalog_releases` | 缺陷/测试/发布投影 |
|
||||
| `catalog_embeddings` | 可降级向量增强 |
|
||||
|
||||
### 分级门禁
|
||||
|
||||
1. **L1 结构化**：`same_mutex` / `same_feature` / `same_scope` / `same_req_id` → hard deny
|
||||
2. **L2 语义**：FTS5 bm25 软门，须 `catalog overlap ack` 显式裁决后才可编码
|
||||
3. **L3 向量**：读 `EMBEDDING_*`；401/超时/缺钥 → `vector_layer=degraded`，不阻断
|
||||
|
||||
### 策略
|
||||
|
||||
`catalog_gate` / `catalog_vector` 为**可选键**（不得加入 `shared_docs` 等集合相等校验字段）。
|
||||
|
||||
### 权威关系
|
||||
|
||||
`.doc` Markdown 仍为事实来源；Catalog 为可检索投影 + 门禁依据；冲突时 `catalog drift` 告警，不自动改文档。
|
||||
|
||||
## 边界
|
||||
|
||||
不新建第二库/容器；不读写业务 PostgreSQL；不改 `overlaps()` 既有语义；不做跨机权威。
|
||||
|
||||
## CLI / MCP
|
||||
|
||||
`agentctl catalog {add,list,show,search,overlap,ack,status,drift}`；MCP：`catalog_requirement_add` / `catalog_overlap_probe` / `catalog_overlap_ack` / `catalog_search` / `catalog_status`。
|
||||
@@ -18,6 +18,7 @@
|
||||
|
||||
| owner | task | branch | worktree | mutex | status |
|
||||
|---|---|---|---|---|---|
|
||||
| Cursor | governance-catalog-2026-09-24 | feature/governance-catalog-2026-09-24 | .worktrees/governance-catalog-2026-09-24 | agent-workflow-governance+catalog | in_progress |
|
||||
| Codex | subagent-workflow-standard-2026-09-24-land | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | pi-subagent-integration | done |
|
||||
| Cursor/Codex | subagent-workflow-standard-2026-09-24-executor | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | pi-subagent-executor | done |
|
||||
| Codex | subagent-workflow-standard-2026-09-24-config-r2 | feature/subagent-workflow-standard-2026-09-24 | .worktrees/subagent-workflow-standard-2026-09-24 | subagent-workflow-docs | done |
|
||||
|
||||
@@ -7,6 +7,6 @@
|
||||
| LESSON-20260923-nullable-duty | docgov、SQL、nullable、23503、可选职责、创建/heal | 检查所有写入口的缺席值语义；用真实 PG 验证 NULL/FK | validated | [可选职责持久化](2026-09-23-nullable-duty.md)；2026-09-23 |
|
||||
| LESSON-20260923-content-hash | docrevision、canonicalize、hash、bootstrap、hard-break、404、水合 | 保持文本语义并检查账本/文件一致性；新数据通过不能覆盖旧数据迁移 | validated | [正文规范化与投影](2026-09-23-content-hash.md)；2026-09-23 |
|
||||
| LESSON-20260923-matrix-status-cas | FEATURE-MATRIX、docs-apply、record_id、merge conflict、CAS | 矩阵/CURRENT 行变更走 docs apply；合并后核验状态列未被回退 | validated | [矩阵状态 CAS](2026-09-23-matrix-status-cas.md)；2026-09-23 |
|
||||
| LESSON-20260924-pi-tool-boundaries | PI、MCP、readonly、path、snapshot、process-group | 路径语义与快照边界一致，合法路径和拒绝路径都验收 | validated | [文件工具边界](2026-09-24-pi-tool-boundaries.md)；2026-09-24 |
|
||||
|
||||
validated 的证据范围见各条目；本日复核为历史材料和源码阅读，未重跑产品回归。后续按机制补充，过时条目标 superseded/retired 并保留替代链接。
|
||||
| LESSON-20260924-pi-tool-boundaries | PI、MCP、readonly、path、snapshot、process-group | 路径语义与快照边界一致，合法路径和拒绝路径都验收 | validated | [文件工具边界](2026-09-24-pi-tool-boundaries.md)；2026-09-24 |
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: 治理目录库与跨对话需求重叠检测
|
||||
task_id: governance-catalog-2026-09-24
|
||||
status: in_progress
|
||||
created: 2026-09-24
|
||||
updated: 2026-09-24
|
||||
owner: Cursor
|
||||
task_type: implementation
|
||||
task_tier: multi_stage
|
||||
policy_revision: agent-governance-2026-09-21-r1 @ cfc80b34a40e8de78e77a44839571bb684d405d5
|
||||
branch: feature/governance-catalog-2026-09-24
|
||||
worktree: .worktrees/governance-catalog-2026-09-24
|
||||
base_dev_sha: cfc80b34a40e8de78e77a44839571bb684d405d5
|
||||
write_scope:
|
||||
- scripts
|
||||
- configs
|
||||
- .doc
|
||||
- .cursor
|
||||
allowed_environments: []
|
||||
mutex: agent-workflow-governance+catalog
|
||||
review_state: pending
|
||||
integration_state: pending
|
||||
deployment: not_required
|
||||
cleanup: pending
|
||||
cleanup_owner: Cursor
|
||||
cleanup_trigger: validate pass + queue enqueue + closeout
|
||||
remote_sync: not_requested
|
||||
---
|
||||
|
||||
# 治理目录库与跨对话需求重叠检测
|
||||
|
||||
## 用户目标与授权
|
||||
|
||||
- **用户目标**：跨对话需求/功能重叠与冲突可机器检测；认领、缺陷、测试、发布版本进入与业务库隔离的治理目录。
|
||||
- **授权来源**：用户确认计划「治理目录库与需求重叠检测」并要求按 to-do 全量实施。
|
||||
- **交付终点**：Catalog schema + FTS5 中文召回 + 两级门禁 + 回填 + 可降级向量层 + MCP/技能 + 工程回归报告；受控合入 local `dev`。
|
||||
- **禁止**：不连业务 PostgreSQL；不新建第二库文件/容器；不自动升 MATRIX；不 push；不强制产品 PI 全链路（工程工具变更，NOT REQUIRED）。
|
||||
|
||||
## 范围与非目标
|
||||
|
||||
- **允许写入**：`scripts/agentctl*.py`、`scripts/catalog_backfill.py`、`scripts/mcp/agentctl-mcp.mjs`、`scripts/tests/`、`configs/agent-governance.json`、`.doc/`（方案/任务/报告/模板漂移修复）、`.cursor/skills` 与 mandatory 规则补充。
|
||||
- **非目标**：跨机权威认领、产品业务功能、自动删除 worktree。
|
||||
|
||||
## 相关问题与经验
|
||||
|
||||
- 检索词：`overlap` / `requirement` / `catalog` / `docs drift` / `FTS5`
|
||||
- 命中：路径级 `overlaps()` 已 enforced；需求语义重叠无机制；embedding 401 运维债 → 向量层 fail-open。
|
||||
- 采纳：可选策略键扩展；CJK 逐字切分禁用 trigram；测试夹具同步复制新模块。
|
||||
|
||||
## 步骤与验收
|
||||
|
||||
- [ ] W0 方案主文档 + policy-registry proposed
|
||||
- [ ] W1–W4 catalog 模块 + task register 联动
|
||||
- [ ] W5 回填 + drift
|
||||
- [ ] W6 向量降级三态
|
||||
- [ ] W7 MCP + 技能
|
||||
- [ ] W8 unittest 全绿 + 报告
|
||||
- [ ] W9 validate → queue → closeout
|
||||
|
||||
## 验证与证据
|
||||
|
||||
| 范围 / 被测 SHA | 命令与环境 | 预期 | 实际 / 判定 | 证据 |
|
||||
|---|---|---|---|---|
|
||||
| 工程工具 | `python3 -B -m unittest discover -s scripts/tests -p 'test_*.py' -v` | 全绿 | （待填） | test-report |
|
||||
|
||||
## 收口与交接
|
||||
|
||||
- 五态见 frontmatter；合入后更新。
|
||||
- 相关：[方案](../2026-09-24-governance-catalog-plan.md)
|
||||
@@ -1,19 +1,20 @@
|
||||
# FEATURE-MATRIX 行模板
|
||||
|
||||
复制到 [FEATURE-MATRIX.md](../../status/FEATURE-MATRIX.md) 的完成度表中。能力状态与 `production_ready`（gap-matrix）正交。
|
||||
复制到 [FEATURE-MATRIX.md](../../status/FEATURE-MATRIX.md) 的完成度表中。能力状态与 `production_ready`（gap-matrix）正交。`record_id` 必须在**第一列**（docs apply 校验）。
|
||||
|
||||
| 字段 | 填写说明 |
|
||||
|------|----------|
|
||||
| record_id | 稳定 kebab-case slug；不随批次改名 |
|
||||
| 模块 | 短名 |
|
||||
| 声明范围 | 本行冻结边界；`done` 只覆盖此范围 |
|
||||
| 状态 | `done` \| `partial` \| `planned` \| `deferred` \| `blocked` |
|
||||
| 状态 | `done` \| `partial` \| `planned` \| `deferred` \| `blocked` \| `removed` |
|
||||
| 发布关系 | `non_blocker` / `补证` / `加固` / `deferred_enhancement` |
|
||||
| 证据 | 设计/代码/test-report 链接；可追 commit/环境/schema |
|
||||
| 剩余工作 / 关闭条件 | partial/planned 必填；stub/SKIP/fallback 不得冒充关闭 |
|
||||
| mutex | 与 CURRENT 并行认领一致 |
|
||||
|
||||
```markdown
|
||||
| <模块名> | <声明范围> | planned | non_blocker | （设计/代码/test-report 链接） | （关闭条件） | `<mutex>` |
|
||||
| <record_id> | <模块名> | <声明范围> | planned | non_blocker | （设计/代码/test-report 链接） | （关闭条件） | `<mutex>` |
|
||||
```
|
||||
|
||||
拆行收口时另在「边界 lineage」区记录 before/after、义务去向与证据覆盖结论。
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# 治理目录库与跨对话需求重叠检测 测试报告
|
||||
|
||||
## 一、 测试背景 (Background)
|
||||
|
||||
跨对话新设计与既有功能/需求重叠时，agentctl 仅有路径级 `overlaps()` 互斥，无需求语义比对。本轮在 `.locks/state.sqlite` 扩展 Catalog 层（与业务 PostgreSQL 物理隔离），落地 FTS5 中文召回、两级门禁与可降级向量层。
|
||||
|
||||
## 二、 测试目标 (Test Objectives)
|
||||
|
||||
- Catalog schema / CRUD / FTS5 CJK（双字词召回，禁用 trigram）
|
||||
- L1 硬拒（same_req_id）与 L2 软门裁决留痕
|
||||
- `with` 在未 ack 时拒绝；ack 后放行
|
||||
- 向量层缺钥/错 URL 降级不阻断
|
||||
- 既有 agentctl 回归无破坏
|
||||
- 工程工具变更：**产品 PI 全链路 NOT REQUIRED**（依据：test-reports-standard §范围判定 — 纯工程规范/工具）
|
||||
|
||||
## 三、 测试方案与环境 (Test Plan & Environment)
|
||||
|
||||
- OS：Linux；Python 3.14.7；SQLite 3.53.4（FTS5）
|
||||
- 入口：`python3 -B -m unittest discover -s scripts/tests -p 'test_*.py'`
|
||||
- 新增：`scripts/tests/test_agentctl_catalog.py`（6 用例）
|
||||
- 回填：`scripts/catalog_backfill.py` → features_matrix=29, issues=10, tests=207
|
||||
|
||||
## 四、 预期结果 (Expected Results)
|
||||
|
||||
| 用例 | 预期 |
|
||||
|---|---|
|
||||
| schema/CRUD | add/list/show/status OK |
|
||||
| CJK digram | trigram `"灰度"`=0；CJK 切分 ≥2 |
|
||||
| same_req_id | `REQUIREMENT_CONFLICT` |
|
||||
| soft ack | 未 ack → `OVERLAP_ACK_REQUIRED`；ack 后 `with` 成功 |
|
||||
| vector degrade | `vector_layer` ∈ degraded/disabled/ok，exit 0 |
|
||||
| hard proceed ack | 拒绝 |
|
||||
|
||||
## 五、 实际测试结果与判定 (Actual Results & Evaluation)
|
||||
|
||||
### 5.1 Catalog 专项（6/6 PASS）
|
||||
|
||||
```text
|
||||
test_cjk_fts_digram_recall ... ok
|
||||
test_hard_proceed_ack_rejected ... ok
|
||||
test_hard_same_req_id_conflict ... ok
|
||||
test_schema_and_crud ... ok
|
||||
test_soft_overlap_requires_ack ... ok
|
||||
test_vector_layer_degrades_without_key ... ok
|
||||
```
|
||||
|
||||
**判定**：PASS
|
||||
|
||||
### 5.2 全量工程回归
|
||||
|
||||
```text
|
||||
Ran 179 tests in 123.217s
|
||||
OK
|
||||
```
|
||||
|
||||
证据：`.doc/test-reports/assets/2026-09-24-governance-catalog/unittest-full.log`
|
||||
|
||||
**判定**：PASS
|
||||
|
||||
### 5.3 回填与 drift
|
||||
|
||||
- backfill：`features_matrix=29`, `issues=10`, `tests=207`, `task_features=128`, `releases=0`
|
||||
- `catalog drift`：`regressions=[]`, `record_count=29`
|
||||
|
||||
**判定**：PASS
|
||||
|
||||
### 性能与耗时
|
||||
|
||||
全量 unittest ≈ 123 s；catalog 专项 ≈ 3 s。
|
||||
|
||||
## 六、 自动化测试套件汇总 (Test Suite Execution)
|
||||
|
||||
本任务为工程工具变更，**不要求** `go test ./...` 产品回归；已跑 agentctl/unittest 全量 179 OK。产品 PI NOT REQUIRED。
|
||||
|
||||
## 七、 测试结论与后续优化建议 (Conclusion & Next Steps)
|
||||
|
||||
达到合入 local `dev` 标准。后续可选：开启 `catalog_vector.enabled`（embedding 401 修复后）、把 CURRENT 认领投影到 requirements、daemon 审计周期附加 `catalog drift`。
|
||||
@@ -0,0 +1,7 @@
|
||||
# 测试报告索引
|
||||
|
||||
本目录归档按 [测试报告规范](../../.cursor/rules/test-reports-standard.mdc) 命名的验收报告：`YYYY-MM-DD-<模块/功能/版本>-test-report.md`。附件在 `assets/<report-slug>/`，不得与报告同层混放。
|
||||
|
||||
检索：按文件名日期 + slug，或 `rg -n '判定|PASS|FAIL' .doc/test-reports`。结构化投影见 agentctl `catalog_tests`（`agentctl catalog status` / backfill）。
|
||||
|
||||
历史命名例外（保留不改）：`2026-09-14-controlled-content-ui-acceptance.md`、`2026-09-17-initial-load-performance-baseline.md`。
|
||||
@@ -0,0 +1,70 @@
|
||||
........................................................................./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x784f8e6cf010>
|
||||
def __init__(self, name, parent):
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x784f8f6efc40>
|
||||
def __init__(self, name, parent):
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x784f8f6ee7a0>
|
||||
def __init__(self, name, parent):
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x784f8ea1e890>
|
||||
def __init__(self, name, parent):
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/mock.py:2259: ResourceWarning: unclosed database in <sqlite3.Connection object at 0x784f8ea1f880>
|
||||
def __init__(self, name, parent):
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
{"demoted": [{"from": "running", "run_id": "hb-stale", "to": "suspect"}, {"from": "suspect", "run_id": "sus-stale", "to": "quarantine"}], "heartbeat_ttl_seconds": 300, "status": "ok", "suspect_ttl_seconds": 600}
|
||||
.{"demoted": [{"from": "running", "run_id": "run-unknown", "to": "suspect"}], "heartbeat_ttl_seconds": 300, "status": "ok", "suspect_ttl_seconds": 600}
|
||||
...usage: python3.14 -m unittest [-h] [--repo REPO]
|
||||
{init,status,doctor,task,with,guard,worktree-check,docs,queue,cleanup,recover,operation,freeze,unfreeze,hooks,validate,integrate,deploy,claim,quarantine,daemon,catalog} ...
|
||||
python3.14 -m unittest: error: unrecognized arguments: --head-sha deadbeef
|
||||
..................................................................................../home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
./home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=10 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=6 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=8 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
/home/linuxbrew/.linuxbrew/Cellar/python@3.14/3.14.7/lib/python3.14/unittest/case.py:615: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
|
||||
result = method()
|
||||
ResourceWarning: Enable tracemalloc to get the object allocation traceback
|
||||
...............
|
||||
----------------------------------------------------------------------
|
||||
Ran 179 tests in 123.217s
|
||||
|
||||
OK
|
||||
@@ -54,5 +54,17 @@
|
||||
]
|
||||
},
|
||||
"heartbeat_ttl_seconds": 300,
|
||||
"suspect_ttl_seconds": 600
|
||||
"suspect_ttl_seconds": 600,
|
||||
"catalog_gate": {
|
||||
"hard_kinds": ["same_mutex", "same_feature", "same_scope", "same_req_id"],
|
||||
"semantic_top_k": 10,
|
||||
"bm25_threshold": -8.0,
|
||||
"ack_required": true
|
||||
},
|
||||
"catalog_vector": {
|
||||
"enabled": false,
|
||||
"timeout_seconds": 8,
|
||||
"fail_open": true,
|
||||
"cosine_threshold": 0.82
|
||||
}
|
||||
}
|
||||
|
||||
+177
-1
@@ -26,6 +26,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
sys.dont_write_bytecode = True
|
||||
os.environ.setdefault('PYTHONDONTWRITEBYTECODE', '1')
|
||||
import agentctl_w4 as w4 # noqa: E402
|
||||
import agentctl_catalog as catalog # noqa: E402
|
||||
|
||||
VERSION = 1
|
||||
POLICY_REL = 'configs/agent-governance.json'
|
||||
@@ -63,6 +64,19 @@ DEFAULT_POLICY = {
|
||||
},
|
||||
'heartbeat_ttl_seconds': 300,
|
||||
'suspect_ttl_seconds': 600,
|
||||
# Optional catalog keys (not in required set-equality checks):
|
||||
'catalog_gate': {
|
||||
'hard_kinds': ['same_mutex', 'same_feature', 'same_scope', 'same_req_id'],
|
||||
'semantic_top_k': 10,
|
||||
'bm25_threshold': -8.0,
|
||||
'ack_required': True,
|
||||
},
|
||||
'catalog_vector': {
|
||||
'enabled': False,
|
||||
'timeout_seconds': 8,
|
||||
'fail_open': True,
|
||||
'cosine_threshold': 0.82,
|
||||
},
|
||||
}
|
||||
PROJECTION_STATUS_VOCAB = frozenset({'planned', 'in_progress', 'blocked', 'done', 'cancelled'})
|
||||
PROJECTION_STATUS_FROM_MACHINE = {'queued': 'in_progress', 'idle': 'planned'}
|
||||
@@ -142,6 +156,9 @@ def deny(code, message, **details):
|
||||
raise Denied(code, message, **details)
|
||||
|
||||
|
||||
catalog.bind_deny(deny)
|
||||
|
||||
|
||||
def emit(value, stream=sys.stdout):
|
||||
print(json.dumps(value, ensure_ascii=False, sort_keys=True), file=stream, flush=True)
|
||||
|
||||
@@ -192,6 +209,10 @@ def _validate_policy_shape(data) -> dict:
|
||||
for key in ('heartbeat_ttl_seconds', 'suspect_ttl_seconds'):
|
||||
if key in data and (not isinstance(data[key], int) or data[key] <= 0):
|
||||
deny('POLICY_INVALID', f'{key} must be a positive integer')
|
||||
# Optional catalog keys: shape checked in catalog.apply_policy; reject non-objects early.
|
||||
for key in ('catalog_gate', 'catalog_vector'):
|
||||
if key in data and data[key] is not None and not isinstance(data[key], dict):
|
||||
deny('POLICY_INVALID', f'{key} must be an object')
|
||||
for filename, rules in data['doc_rules'].items():
|
||||
if not isinstance(rules, dict):
|
||||
deny('POLICY_INVALID', 'doc_rules entries must be objects', file=filename)
|
||||
@@ -218,6 +239,7 @@ def apply_policy(policy: dict) -> dict:
|
||||
HEARTBEAT_TTL_SECONDS = int(policy.get('heartbeat_ttl_seconds', DEFAULT_POLICY['heartbeat_ttl_seconds']))
|
||||
SUSPECT_TTL_SECONDS = int(policy.get('suspect_ttl_seconds', DEFAULT_POLICY['suspect_ttl_seconds']))
|
||||
w4.apply_policy(policy)
|
||||
catalog.apply_policy(policy)
|
||||
return policy
|
||||
|
||||
|
||||
@@ -749,6 +771,7 @@ def ensure_schema(db):
|
||||
if name not in existing:
|
||||
db.execute(f'ALTER TABLE runs ADD COLUMN {name} {decl}')
|
||||
w4.ensure_schema(db)
|
||||
catalog.ensure_schema(db)
|
||||
|
||||
|
||||
def codegraph_is_ignored(path):
|
||||
@@ -1081,6 +1104,10 @@ def execute(repo, args):
|
||||
command = args.command[1:] if args.command[:1] == ['--'] else args.command
|
||||
if not command:
|
||||
deny('INVALID_REQUEST', 'Expected a child command after --')
|
||||
# Soft-overlap acks must be clear before exclusive writes (shared reads exempt).
|
||||
if not args.shared:
|
||||
with repo.db() as db:
|
||||
catalog.assert_acks_clear(db, args.task)
|
||||
cwd = Path(args.cwd).resolve() if args.cwd else Path(task['worktree'])
|
||||
known = {Path(r['worktree']).resolve() for r in repo.worktrees()}
|
||||
if cwd not in known or cwd not in (Path(task['worktree']), repo.main):
|
||||
@@ -2654,6 +2681,10 @@ def parser():
|
||||
for name in ('init','status','doctor'):sub.add_parser(name)
|
||||
task=sub.add_parser('task').add_subparsers(dest='task_action',required=True)
|
||||
reg=task.add_parser('register');reg.add_argument('--id',required=True);reg.add_argument('--owner',required=True);reg.add_argument('--scope',action='append',required=True);reg.add_argument('--capability',action='append',default=[])
|
||||
reg.add_argument('--requirement',help='Catalog R-ID to bind at register')
|
||||
reg.add_argument('--intent',help='Design intent text for overlap probe')
|
||||
reg.add_argument('--feature',help='FEATURE-MATRIX record_id / catalog feature')
|
||||
reg.add_argument('--mutex',help='Mutex key for hard same_mutex gate')
|
||||
close=task.add_parser('close');close.add_argument('--id',required=True);close.add_argument('--status',choices=['done','cancelled'],default='done')
|
||||
w=sub.add_parser('with');w.add_argument('--task',required=True);w.add_argument('--resource',action='append',required=True);w.add_argument('--shared',action='store_true');w.add_argument('--request-id');w.add_argument('--wait',type=float,default=0);w.add_argument('--cwd');w.add_argument('command',nargs=argparse.REMAINDER)
|
||||
g=sub.add_parser('guard').add_subparsers(dest='kind',required=True)
|
||||
@@ -2718,6 +2749,36 @@ def parser():
|
||||
drun.add_argument('--audit-every',type=int,default=30,dest='audit_every',
|
||||
help='Every N cycles run cleanup report + docs drift (0=off)')
|
||||
dm.add_parser('status')
|
||||
cat = sub.add_parser('catalog').add_subparsers(dest='catalog_action', required=True)
|
||||
cadd = cat.add_parser('add')
|
||||
cadd.add_argument('--id', required=True)
|
||||
cadd.add_argument('--title', required=True)
|
||||
cadd.add_argument('--statement', required=True)
|
||||
cadd.add_argument('--owner', required=True)
|
||||
cadd.add_argument('--feature')
|
||||
cadd.add_argument('--source-task')
|
||||
cadd.add_argument('--source-doc')
|
||||
cadd.add_argument('--state', default='proposed')
|
||||
cadd.add_argument('--conversation-ref')
|
||||
clist = cat.add_parser('list'); clist.add_argument('--state'); clist.add_argument('--limit', type=int, default=100)
|
||||
cshow = cat.add_parser('show'); cshow.add_argument('--id', required=True)
|
||||
csearch = cat.add_parser('search'); csearch.add_argument('--query', required=True); csearch.add_argument('--top', type=int, default=10)
|
||||
cov = cat.add_parser('overlap')
|
||||
cov.add_argument('--task', required=True)
|
||||
cov.add_argument('--statement', required=True)
|
||||
cov.add_argument('--requirement')
|
||||
cov.add_argument('--feature')
|
||||
cov.add_argument('--mutex')
|
||||
cov.add_argument('--top', type=int, default=10)
|
||||
cack = cat.add_parser('ack')
|
||||
cack.add_argument('--task', required=True)
|
||||
cack.add_argument('--overlap', required=True)
|
||||
cack.add_argument('--decision', required=True, choices=sorted(catalog.DECISIONS))
|
||||
cack.add_argument('--reason', required=True)
|
||||
cack.add_argument('--decided-by', default='Cursor')
|
||||
cack.add_argument('--authorization-ref')
|
||||
cat.add_parser('status')
|
||||
cat.add_parser('drift', help='Refresh features from MATRIX; report done→non-done regressions')
|
||||
return p
|
||||
|
||||
|
||||
@@ -2726,7 +2787,7 @@ def main():
|
||||
write_actions = {
|
||||
'init', 'task', 'with', 'docs', 'validate', 'integrate', 'queue',
|
||||
'deploy', 'recover', 'freeze', 'unfreeze', 'hooks', 'guard', 'cleanup', 'claim',
|
||||
'quarantine', 'daemon',
|
||||
'quarantine', 'daemon', 'catalog',
|
||||
}
|
||||
# status/doctor remain readable without a policy file and never create one.
|
||||
if args.action in write_actions:
|
||||
@@ -2744,6 +2805,10 @@ def main():
|
||||
require_policy(repo) # list needs TTL/schema constants
|
||||
elif args.action == 'daemon' and getattr(args, 'daemon_action', None) == 'status':
|
||||
pass # read-only probe; never creates manager.lock or heartbeat
|
||||
elif args.action == 'catalog' and getattr(args, 'catalog_action', None) in (
|
||||
'list', 'show', 'search', 'status',
|
||||
):
|
||||
require_policy(repo) # needs schema; non-mutating reads
|
||||
elif args.action == 'cleanup':
|
||||
require_policy(repo)
|
||||
elif args.action == 'hooks' and getattr(args, 'hook_action', None) == 'codegraph-sync':
|
||||
@@ -2783,6 +2848,25 @@ def main():
|
||||
safe_path(repo.current,scope)
|
||||
for cap in args.capability:
|
||||
if cap not in ('docs:admin','docs:approve'):resource(cap)
|
||||
# Catalog gate BEFORE insert: probe commits in its own txn so soft hits survive deny.
|
||||
req = getattr(args, 'requirement', None)
|
||||
intent = getattr(args, 'intent', None)
|
||||
feature = getattr(args, 'feature', None)
|
||||
mutex = getattr(args, 'mutex', None)
|
||||
if req or intent or feature or mutex:
|
||||
with repo.db(True) as db:
|
||||
gate_result = catalog.gate_task_register(
|
||||
db,
|
||||
task=args.id,
|
||||
scopes=scopes,
|
||||
requirement=req,
|
||||
intent=intent,
|
||||
feature=feature,
|
||||
mutex=mutex,
|
||||
overlaps_fn=overlaps,
|
||||
statement=intent,
|
||||
)
|
||||
catalog.enforce_gate_result(gate_result)
|
||||
values=(args.id,args.owner,str(repo.current),branch,json.dumps(scopes),json.dumps(sorted(set(args.capability))),'active',git(repo.current,'rev-parse','HEAD'))
|
||||
with repo.os_file('worktree-' + digest(str(repo.current)) + '.lock').open('a+') as lifecycle, repo.db(True) as db:
|
||||
fcntl.flock(lifecycle,fcntl.LOCK_SH|fcntl.LOCK_NB)
|
||||
@@ -2865,9 +2949,101 @@ def main():
|
||||
elif args.action=='daemon':
|
||||
if args.daemon_action=='run':return daemon_run(repo,args)
|
||||
if args.daemon_action=='status':return daemon_status(repo,args)
|
||||
elif args.action=='catalog':
|
||||
return catalog_cmd(repo, args)
|
||||
return 0
|
||||
|
||||
|
||||
def catalog_cmd(repo, args):
|
||||
"""Dispatch agentctl catalog subcommands."""
|
||||
action = args.catalog_action
|
||||
if action == 'add':
|
||||
with repo.db(True) as db:
|
||||
result = catalog.requirement_add(
|
||||
db,
|
||||
req_id=args.id,
|
||||
title=args.title,
|
||||
statement=args.statement,
|
||||
owner=args.owner,
|
||||
feature=args.feature,
|
||||
source_task=args.source_task,
|
||||
source_doc=args.source_doc,
|
||||
state=args.state,
|
||||
conversation_ref=args.conversation_ref,
|
||||
)
|
||||
emit(result)
|
||||
return 0
|
||||
if action == 'list':
|
||||
with repo.db() as db:
|
||||
rows = catalog.requirement_list(db, state=args.state, limit=args.limit)
|
||||
emit({'status': 'ok', 'requirements': rows})
|
||||
return 0
|
||||
if action == 'show':
|
||||
with repo.db() as db:
|
||||
row = catalog.requirement_show(db, args.id)
|
||||
emit({'status': 'ok', 'requirement': row})
|
||||
return 0
|
||||
if action == 'search':
|
||||
with repo.db() as db:
|
||||
hits = catalog.search(db, args.query, top=args.top)
|
||||
emit({'status': 'ok', 'hits': hits})
|
||||
return 0
|
||||
if action == 'overlap':
|
||||
with repo.db(True) as db:
|
||||
scopes = []
|
||||
try:
|
||||
task = task_row(repo, args.task)
|
||||
scopes = json.loads(task['scopes']) if isinstance(task['scopes'], str) else task['scopes']
|
||||
except Denied:
|
||||
# Allow pre-register probes; path/scope hard checks skipped without a task row.
|
||||
pass
|
||||
result = catalog.probe_overlap(
|
||||
db,
|
||||
task=args.task,
|
||||
statement=args.statement,
|
||||
requirement=args.requirement,
|
||||
feature=args.feature,
|
||||
mutex=args.mutex,
|
||||
scopes=scopes,
|
||||
top=args.top,
|
||||
overlaps_fn=overlaps,
|
||||
)
|
||||
emit(result)
|
||||
return 0
|
||||
if action == 'ack':
|
||||
with repo.db(True) as db:
|
||||
result = catalog.overlap_ack(
|
||||
db,
|
||||
task=args.task,
|
||||
overlap_id=args.overlap,
|
||||
decision=args.decision,
|
||||
decided_by=args.decided_by,
|
||||
reason=args.reason,
|
||||
authorization_ref=args.authorization_ref,
|
||||
)
|
||||
emit(result)
|
||||
return 0
|
||||
if action == 'status':
|
||||
with repo.db() as db:
|
||||
emit(catalog.catalog_status(db))
|
||||
return 0
|
||||
if action == 'drift':
|
||||
matrix_path = repo.main / '.doc/status/FEATURE-MATRIX.md'
|
||||
# Prefer clean tip from refs/heads/dev when available.
|
||||
try:
|
||||
content = git(repo.main, 'show', 'refs/heads/dev:.doc/status/FEATURE-MATRIX.md')
|
||||
except Exception:
|
||||
content = matrix_path.read_text(encoding='utf-8') if matrix_path.is_file() else ''
|
||||
states = matrix_states(content) if content else {}
|
||||
with repo.db(True) as db:
|
||||
result = catalog.catalog_drift(db, matrix_states=states)
|
||||
event(db, 'catalog', {'event': 'drift', **{k: result[k] for k in ('record_count', 'regressions')}})
|
||||
emit(result)
|
||||
return 0
|
||||
deny('INVALID_REQUEST', f'Unknown catalog action: {action}')
|
||||
return 1
|
||||
|
||||
|
||||
if __name__=='__main__':
|
||||
try:
|
||||
sys.exit(main() or 0)
|
||||
|
||||
@@ -0,0 +1,855 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Catalog governance layer: requirements, FTS5 CJK recall, overlap gates, embeddings.
|
||||
|
||||
Lives beside agentctl.py. Additive schema only; never touches product PostgreSQL.
|
||||
Vector layer is fail-open: missing/failing embeddings degrade to FTS5 only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
import struct
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
SCHEMA_SQL = '''
|
||||
CREATE TABLE IF NOT EXISTS catalog_requirements(
|
||||
id TEXT PRIMARY KEY,
|
||||
title TEXT NOT NULL,
|
||||
statement TEXT NOT NULL,
|
||||
feature TEXT,
|
||||
source_task TEXT,
|
||||
source_doc TEXT,
|
||||
owner TEXT NOT NULL,
|
||||
state TEXT NOT NULL,
|
||||
supersedes TEXT,
|
||||
conversation_ref TEXT,
|
||||
created REAL NOT NULL,
|
||||
updated REAL NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS catalog_req_state ON catalog_requirements(state, feature);
|
||||
CREATE INDEX IF NOT EXISTS catalog_req_task ON catalog_requirements(source_task);
|
||||
CREATE TABLE IF NOT EXISTS catalog_features(
|
||||
id TEXT PRIMARY KEY,
|
||||
title TEXT NOT NULL,
|
||||
scope TEXT,
|
||||
mutex TEXT,
|
||||
matrix_status TEXT,
|
||||
updated REAL NOT NULL
|
||||
);
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS catalog_req_text USING fts5(
|
||||
req_id UNINDEXED, body, seg, tokenize='unicode61'
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS catalog_overlaps(
|
||||
id TEXT PRIMARY KEY,
|
||||
probe_task TEXT NOT NULL,
|
||||
probe_req TEXT,
|
||||
target_req TEXT,
|
||||
target_task TEXT,
|
||||
target_feature TEXT,
|
||||
level TEXT NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
score REAL,
|
||||
evidence TEXT NOT NULL,
|
||||
created REAL NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS catalog_overlaps_probe ON catalog_overlaps(probe_task, created);
|
||||
CREATE TABLE IF NOT EXISTS catalog_decisions(
|
||||
id TEXT PRIMARY KEY,
|
||||
overlap_id TEXT NOT NULL,
|
||||
task TEXT NOT NULL,
|
||||
decision TEXT NOT NULL,
|
||||
decided_by TEXT NOT NULL,
|
||||
reason TEXT NOT NULL,
|
||||
authorization_ref TEXT,
|
||||
created REAL NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS catalog_decisions_task ON catalog_decisions(task, overlap_id);
|
||||
CREATE TABLE IF NOT EXISTS catalog_issues(
|
||||
issue_id TEXT PRIMARY KEY,
|
||||
severity TEXT,
|
||||
issue_state TEXT,
|
||||
owner TEXT,
|
||||
module TEXT,
|
||||
keywords TEXT,
|
||||
main_record TEXT,
|
||||
lesson_ids TEXT,
|
||||
requirements TEXT,
|
||||
tasks TEXT,
|
||||
recurrence_count INTEGER DEFAULT 0,
|
||||
evidence_basis TEXT,
|
||||
created REAL,
|
||||
updated REAL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS catalog_tests(
|
||||
id TEXT PRIMARY KEY,
|
||||
report_path TEXT NOT NULL,
|
||||
task TEXT,
|
||||
requirements TEXT,
|
||||
scope TEXT,
|
||||
verdict TEXT,
|
||||
level TEXT,
|
||||
real_agent INTEGER NOT NULL DEFAULT 0,
|
||||
schema_version TEXT,
|
||||
env TEXT,
|
||||
sha TEXT,
|
||||
created REAL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS catalog_releases(
|
||||
id TEXT PRIMARY KEY,
|
||||
env TEXT NOT NULL,
|
||||
branch TEXT,
|
||||
commit_sha TEXT,
|
||||
tag TEXT,
|
||||
deploy_seq INTEGER,
|
||||
schema_version TEXT,
|
||||
image_digest TEXT,
|
||||
source_task TEXT,
|
||||
phase TEXT,
|
||||
receipt TEXT,
|
||||
created REAL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS catalog_embeddings(
|
||||
req_id TEXT PRIMARY KEY,
|
||||
model TEXT NOT NULL,
|
||||
dim INTEGER NOT NULL,
|
||||
vec BLOB NOT NULL,
|
||||
source_sha256 TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
created REAL NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS catalog_task_bindings(
|
||||
task TEXT PRIMARY KEY,
|
||||
requirement TEXT,
|
||||
intent TEXT,
|
||||
mutex TEXT,
|
||||
feature TEXT,
|
||||
updated REAL NOT NULL
|
||||
);
|
||||
'''
|
||||
|
||||
REQ_STATES = frozenset({
|
||||
'proposed', 'accepted', 'in_progress', 'delivered', 'superseded', 'rejected',
|
||||
})
|
||||
HARD_KINDS = frozenset({'same_mutex', 'same_feature', 'same_scope', 'same_req_id'})
|
||||
SOFT_KINDS = frozenset({'semantic'})
|
||||
DECISIONS = frozenset({'proceed', 'merge_into', 'supersede', 'defer', 'reject'})
|
||||
REQ_ID_RE = re.compile(r'^R-[A-Za-z0-9][A-Za-z0-9._-]{0,120}$')
|
||||
|
||||
DEFAULT_GATE = {
|
||||
'hard_kinds': ['same_mutex', 'same_feature', 'same_scope', 'same_req_id'],
|
||||
'semantic_top_k': 10,
|
||||
'bm25_threshold': -8.0,
|
||||
'ack_required': True,
|
||||
}
|
||||
DEFAULT_VECTOR = {
|
||||
'enabled': False,
|
||||
'timeout_seconds': 8,
|
||||
'fail_open': True,
|
||||
'cosine_threshold': 0.82,
|
||||
}
|
||||
|
||||
CATALOG_GATE: dict = dict(DEFAULT_GATE)
|
||||
CATALOG_VECTOR: dict = dict(DEFAULT_VECTOR)
|
||||
|
||||
# Deny callback injected by agentctl (deny(code, message, **details)).
|
||||
_deny: Optional[Callable[..., Any]] = None
|
||||
|
||||
|
||||
def bind_deny(fn: Callable[..., Any]) -> None:
|
||||
global _deny
|
||||
_deny = fn
|
||||
|
||||
|
||||
def _fail(code: str, message: str, **details: Any) -> None:
|
||||
if _deny is None:
|
||||
raise RuntimeError(f'{code}: {message} {details}')
|
||||
_deny(code, message, **details)
|
||||
|
||||
|
||||
def apply_policy(policy: dict) -> None:
|
||||
"""Optional catalog_gate / catalog_vector keys; missing → defaults."""
|
||||
global CATALOG_GATE, CATALOG_VECTOR
|
||||
gate = policy.get('catalog_gate')
|
||||
if gate is None:
|
||||
CATALOG_GATE = dict(DEFAULT_GATE)
|
||||
elif not isinstance(gate, dict):
|
||||
_fail('POLICY_INVALID', 'catalog_gate must be an object')
|
||||
else:
|
||||
merged = dict(DEFAULT_GATE)
|
||||
merged.update(gate)
|
||||
if not isinstance(merged.get('hard_kinds'), list):
|
||||
_fail('POLICY_INVALID', 'catalog_gate.hard_kinds must be a list')
|
||||
if not isinstance(merged.get('semantic_top_k'), int) or merged['semantic_top_k'] <= 0:
|
||||
_fail('POLICY_INVALID', 'catalog_gate.semantic_top_k must be a positive int')
|
||||
if not isinstance(merged.get('bm25_threshold'), (int, float)):
|
||||
_fail('POLICY_INVALID', 'catalog_gate.bm25_threshold must be a number')
|
||||
if not isinstance(merged.get('ack_required'), bool):
|
||||
_fail('POLICY_INVALID', 'catalog_gate.ack_required must be a bool')
|
||||
CATALOG_GATE = merged
|
||||
vec = policy.get('catalog_vector')
|
||||
if vec is None:
|
||||
CATALOG_VECTOR = dict(DEFAULT_VECTOR)
|
||||
elif not isinstance(vec, dict):
|
||||
_fail('POLICY_INVALID', 'catalog_vector must be an object')
|
||||
else:
|
||||
merged = dict(DEFAULT_VECTOR)
|
||||
merged.update(vec)
|
||||
if not isinstance(merged.get('enabled'), bool):
|
||||
_fail('POLICY_INVALID', 'catalog_vector.enabled must be a bool')
|
||||
if not isinstance(merged.get('timeout_seconds'), int) or merged['timeout_seconds'] <= 0:
|
||||
_fail('POLICY_INVALID', 'catalog_vector.timeout_seconds must be a positive int')
|
||||
if not isinstance(merged.get('fail_open'), bool):
|
||||
_fail('POLICY_INVALID', 'catalog_vector.fail_open must be a bool')
|
||||
CATALOG_VECTOR = merged
|
||||
|
||||
|
||||
def ensure_schema(db: sqlite3.Connection) -> None:
|
||||
db.executescript(SCHEMA_SQL)
|
||||
|
||||
|
||||
def _now() -> float:
|
||||
return time.time()
|
||||
|
||||
|
||||
def split_cjk(text: str) -> str:
|
||||
"""Insert spaces around CJK chars so unicode61 FTS can phrase-match digrams."""
|
||||
if not text:
|
||||
return ''
|
||||
spaced = re.sub(r'([\u3400-\u9fff\uf900-\ufaff])', r' \1 ', text)
|
||||
return ' '.join(spaced.split())
|
||||
|
||||
|
||||
def fts_phrase(text: str) -> str:
|
||||
"""Build a double-quoted FTS5 phrase from CJK-split tokens."""
|
||||
tokens = split_cjk(text).split()
|
||||
if not tokens:
|
||||
return '""'
|
||||
# Escape embedded double quotes for FTS5.
|
||||
safe = [t.replace('"', '""') for t in tokens]
|
||||
return '"' + ' '.join(safe) + '"'
|
||||
|
||||
|
||||
def validate_req_id(req_id: str) -> str:
|
||||
if not REQ_ID_RE.fullmatch(req_id):
|
||||
_fail('INVALID_REQUEST', 'Requirement id must match R-<slug>', id=req_id)
|
||||
return req_id
|
||||
|
||||
|
||||
def requirement_add(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
req_id: str,
|
||||
title: str,
|
||||
statement: str,
|
||||
owner: str,
|
||||
feature: Optional[str] = None,
|
||||
source_task: Optional[str] = None,
|
||||
source_doc: Optional[str] = None,
|
||||
state: str = 'proposed',
|
||||
supersedes: Optional[str] = None,
|
||||
conversation_ref: Optional[str] = None,
|
||||
) -> dict:
|
||||
validate_req_id(req_id)
|
||||
if state not in REQ_STATES:
|
||||
_fail('INVALID_STATE', 'Unknown requirement state', state=state)
|
||||
if not title.strip() or not statement.strip() or not owner.strip():
|
||||
_fail('INVALID_REQUEST', 'title, statement, and owner are required')
|
||||
existing = db.execute('SELECT id, source_task FROM catalog_requirements WHERE id=?', (req_id,)).fetchone()
|
||||
if existing:
|
||||
if source_task and existing['source_task'] and existing['source_task'] != source_task:
|
||||
_fail(
|
||||
'REQUIREMENT_EXISTS',
|
||||
'Requirement id already owned by another task',
|
||||
id=req_id,
|
||||
holder=existing['source_task'],
|
||||
)
|
||||
_fail('REQUIREMENT_EXISTS', 'Requirement id already exists', id=req_id)
|
||||
now = _now()
|
||||
db.execute(
|
||||
'INSERT INTO catalog_requirements VALUES(?,?,?,?,?,?,?,?,?,?,?,?)',
|
||||
(
|
||||
req_id, title.strip(), statement.strip(), feature, source_task, source_doc,
|
||||
owner.strip(), state, supersedes, conversation_ref, now, now,
|
||||
),
|
||||
)
|
||||
_index_requirement(db, req_id, title.strip() + '\n' + statement.strip())
|
||||
return {'status': 'added', 'id': req_id}
|
||||
|
||||
|
||||
def _index_requirement(db: sqlite3.Connection, req_id: str, body: str) -> None:
|
||||
db.execute('DELETE FROM catalog_req_text WHERE req_id=?', (req_id,))
|
||||
db.execute(
|
||||
'INSERT INTO catalog_req_text(req_id, body, seg) VALUES(?,?,?)',
|
||||
(req_id, body, split_cjk(body)),
|
||||
)
|
||||
|
||||
|
||||
def requirement_list(db: sqlite3.Connection, *, state: Optional[str] = None, limit: int = 100) -> list[dict]:
|
||||
if state:
|
||||
rows = db.execute(
|
||||
'SELECT * FROM catalog_requirements WHERE state=? ORDER BY updated DESC LIMIT ?',
|
||||
(state, limit),
|
||||
).fetchall()
|
||||
else:
|
||||
rows = db.execute(
|
||||
'SELECT * FROM catalog_requirements ORDER BY updated DESC LIMIT ?', (limit,)
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def requirement_show(db: sqlite3.Connection, req_id: str) -> dict:
|
||||
row = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (req_id,)).fetchone()
|
||||
if not row:
|
||||
_fail('UNKNOWN_REQUIREMENT', 'Requirement not found', id=req_id)
|
||||
return dict(row)
|
||||
|
||||
|
||||
def feature_upsert(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
feature_id: str,
|
||||
title: str,
|
||||
scope: Optional[str] = None,
|
||||
mutex: Optional[str] = None,
|
||||
matrix_status: Optional[str] = None,
|
||||
) -> dict:
|
||||
now = _now()
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_features(id, title, scope, mutex, matrix_status, updated)
|
||||
VALUES(?,?,?,?,?,?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
title=excluded.title,
|
||||
scope=COALESCE(excluded.scope, catalog_features.scope),
|
||||
mutex=COALESCE(excluded.mutex, catalog_features.mutex),
|
||||
matrix_status=COALESCE(excluded.matrix_status, catalog_features.matrix_status),
|
||||
updated=excluded.updated''',
|
||||
(feature_id, title, scope, mutex, matrix_status, now),
|
||||
)
|
||||
return {'status': 'upserted', 'id': feature_id}
|
||||
|
||||
|
||||
def search(db: sqlite3.Connection, query: str, *, top: int = 10) -> list[dict]:
|
||||
tokens = split_cjk(query).split()
|
||||
if not tokens:
|
||||
return []
|
||||
# Soft recall: AND of char tokens (all must appear). Phrase is too strict for
|
||||
# near-paraphrase overlap ("检测与治理" vs "检测治理"). Escape FTS keywords.
|
||||
safe = []
|
||||
for t in tokens:
|
||||
t = t.replace('"', '""')
|
||||
if t.upper() in {'AND', 'OR', 'NOT', 'NEAR'}:
|
||||
t = '"' + t + '"'
|
||||
else:
|
||||
t = '"' + t + '"' # quote each token to avoid column/syntax issues
|
||||
safe.append(t)
|
||||
# Quoting each token and joining with spaces → AND of tokens in FTS5.
|
||||
match_expr = ' '.join(safe)
|
||||
try:
|
||||
rows = db.execute(
|
||||
'''SELECT r.id, r.title, r.statement, r.feature, r.source_task, r.state,
|
||||
bm25(catalog_req_text) AS score
|
||||
FROM catalog_req_text
|
||||
JOIN catalog_requirements r ON r.id = catalog_req_text.req_id
|
||||
WHERE catalog_req_text MATCH ?
|
||||
ORDER BY score
|
||||
LIMIT ?''',
|
||||
(match_expr, top),
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
return []
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def bind_task(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
task: str,
|
||||
requirement: Optional[str] = None,
|
||||
intent: Optional[str] = None,
|
||||
mutex: Optional[str] = None,
|
||||
feature: Optional[str] = None,
|
||||
) -> dict:
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_task_bindings(task, requirement, intent, mutex, feature, updated)
|
||||
VALUES(?,?,?,?,?,?)
|
||||
ON CONFLICT(task) DO UPDATE SET
|
||||
requirement=COALESCE(excluded.requirement, catalog_task_bindings.requirement),
|
||||
intent=COALESCE(excluded.intent, catalog_task_bindings.intent),
|
||||
mutex=COALESCE(excluded.mutex, catalog_task_bindings.mutex),
|
||||
feature=COALESCE(excluded.feature, catalog_task_bindings.feature),
|
||||
updated=excluded.updated''',
|
||||
(task, requirement, intent, mutex, feature, _now()),
|
||||
)
|
||||
return {'status': 'bound', 'task': task}
|
||||
|
||||
|
||||
def _active_tasks(db: sqlite3.Connection) -> list[dict]:
|
||||
try:
|
||||
rows = db.execute("SELECT * FROM tasks WHERE state='active'").fetchall()
|
||||
except sqlite3.Error:
|
||||
return []
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def _path_overlaps(scopes_a: list[str], scopes_b: list[str], overlaps_fn: Callable) -> bool:
|
||||
for a in scopes_a:
|
||||
for b in scopes_b:
|
||||
if overlaps_fn('path:' + a, 'path:' + b):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def probe_overlap(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
task: str,
|
||||
statement: str,
|
||||
requirement: Optional[str] = None,
|
||||
feature: Optional[str] = None,
|
||||
mutex: Optional[str] = None,
|
||||
scopes: Optional[list[str]] = None,
|
||||
top: Optional[int] = None,
|
||||
overlaps_fn: Optional[Callable] = None,
|
||||
) -> dict:
|
||||
"""Run L1 hard + L2 soft (+ optional L3) overlap detection; persist soft/hard rows."""
|
||||
top = top or int(CATALOG_GATE['semantic_top_k'])
|
||||
hard_kinds = set(CATALOG_GATE.get('hard_kinds') or DEFAULT_GATE['hard_kinds'])
|
||||
hard: list[dict] = []
|
||||
soft: list[dict] = []
|
||||
now = _now()
|
||||
|
||||
# L1: same_req_id
|
||||
if requirement and 'same_req_id' in hard_kinds:
|
||||
row = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (requirement,)).fetchone()
|
||||
if row and row['source_task'] and row['source_task'] != task:
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=requirement,
|
||||
target_task=row['source_task'], target_feature=row['feature'],
|
||||
level='hard', kind='same_req_id', score=1.0,
|
||||
evidence=f'Requirement {requirement} owned by {row["source_task"]}',
|
||||
created=now,
|
||||
)
|
||||
hard.append(hit)
|
||||
|
||||
# L1: same_feature — another active requirement on same feature (not superseded)
|
||||
if feature and 'same_feature' in hard_kinds:
|
||||
rows = db.execute(
|
||||
'''SELECT * FROM catalog_requirements
|
||||
WHERE feature=? AND state NOT IN ('superseded','rejected','delivered')
|
||||
AND (source_task IS NULL OR source_task != ?)''',
|
||||
(feature, task),
|
||||
).fetchall()
|
||||
for row in rows:
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=row['id'],
|
||||
target_task=row['source_task'], target_feature=feature,
|
||||
level='hard', kind='same_feature', score=1.0,
|
||||
evidence=f'Feature {feature} already claimed by requirement {row["id"]}',
|
||||
created=now,
|
||||
)
|
||||
hard.append(hit)
|
||||
|
||||
# L1: same_mutex via bindings + CURRENT-style mutex on active tasks
|
||||
if mutex and 'same_mutex' in hard_kinds:
|
||||
rows = db.execute(
|
||||
'''SELECT * FROM catalog_task_bindings
|
||||
WHERE mutex=? AND task != ?''',
|
||||
(mutex, task),
|
||||
).fetchall()
|
||||
active_ids = {t['id'] for t in _active_tasks(db)}
|
||||
for row in rows:
|
||||
if row['task'] not in active_ids and row['task'] not in active_ids:
|
||||
# Still hard if binding exists and peer task is active
|
||||
pass
|
||||
if row['task'] in active_ids:
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=row['requirement'],
|
||||
target_task=row['task'], target_feature=row['feature'],
|
||||
level='hard', kind='same_mutex', score=1.0,
|
||||
evidence=f'Mutex {mutex} held by active task {row["task"]}',
|
||||
created=now,
|
||||
)
|
||||
hard.append(hit)
|
||||
|
||||
# L1: same_scope — path overlaps with other active task scopes
|
||||
if scopes and overlaps_fn and 'same_scope' in hard_kinds:
|
||||
for peer in _active_tasks(db):
|
||||
if peer['id'] == task:
|
||||
continue
|
||||
peer_scopes = json.loads(peer['scopes']) if isinstance(peer['scopes'], str) else peer['scopes']
|
||||
if _path_overlaps(scopes, peer_scopes, overlaps_fn):
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=None,
|
||||
target_task=peer['id'], target_feature=None,
|
||||
level='hard', kind='same_scope', score=1.0,
|
||||
evidence=f'Scope overlaps active task {peer["id"]} scopes={peer_scopes}',
|
||||
created=now,
|
||||
)
|
||||
hard.append(hit)
|
||||
|
||||
# L2: semantic FTS5
|
||||
threshold = float(CATALOG_GATE.get('bm25_threshold', DEFAULT_GATE['bm25_threshold']))
|
||||
hits = search(db, statement, top=top)
|
||||
for row in hits:
|
||||
if requirement and row['id'] == requirement:
|
||||
continue
|
||||
if row.get('source_task') == task:
|
||||
continue
|
||||
score = float(row.get('score') or 0.0)
|
||||
# bm25: lower (more negative) is better; accept scores <= threshold
|
||||
if score > threshold and score >= 0:
|
||||
# Positive bm25 is weak; still include if phrase matched and score is finite
|
||||
pass
|
||||
# Always record soft semantic hits that MATCH returned (already ranked)
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=row['id'],
|
||||
target_task=row.get('source_task'), target_feature=row.get('feature'),
|
||||
level='soft', kind='semantic', score=score,
|
||||
evidence=f'FTS5 hit title={row["title"]!r} bm25={score:.4f}',
|
||||
created=now,
|
||||
)
|
||||
soft.append(hit)
|
||||
|
||||
# L3: optional vector enhancement (fail-open)
|
||||
vector_status = 'disabled'
|
||||
if CATALOG_VECTOR.get('enabled'):
|
||||
vector_status, vec_soft = _vector_soft_hits(
|
||||
db, task=task, statement=statement, requirement=requirement, top=top, created=now,
|
||||
)
|
||||
# Merge unique soft hits
|
||||
seen = {h['id'] for h in soft}
|
||||
for h in vec_soft:
|
||||
if h['id'] not in seen:
|
||||
soft.append(h)
|
||||
seen.add(h['id'])
|
||||
|
||||
# Soft hits need ack — filter already-acked
|
||||
pending_soft = [h for h in soft if not _decision_exists(db, task, h['id'])]
|
||||
status = 'conflict' if hard else ('ack_required' if pending_soft else 'none')
|
||||
return {
|
||||
'status': status,
|
||||
'hard': hard,
|
||||
'soft': soft,
|
||||
'pending_soft': pending_soft,
|
||||
'vector_layer': vector_status,
|
||||
'task': task,
|
||||
'requirement': requirement,
|
||||
}
|
||||
|
||||
|
||||
def _persist_overlap(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
probe_task: str,
|
||||
probe_req: Optional[str],
|
||||
target_req: Optional[str],
|
||||
target_task: Optional[str],
|
||||
target_feature: Optional[str],
|
||||
level: str,
|
||||
kind: str,
|
||||
score: Optional[float],
|
||||
evidence: str,
|
||||
created: float,
|
||||
) -> dict:
|
||||
oid = str(uuid.uuid4())
|
||||
db.execute(
|
||||
'INSERT INTO catalog_overlaps VALUES(?,?,?,?,?,?,?,?,?,?,?)',
|
||||
(
|
||||
oid, probe_task, probe_req, target_req, target_task, target_feature,
|
||||
level, kind, score, evidence, created,
|
||||
),
|
||||
)
|
||||
return {
|
||||
'id': oid, 'probe_task': probe_task, 'probe_req': probe_req,
|
||||
'target_req': target_req, 'target_task': target_task,
|
||||
'target_feature': target_feature, 'level': level, 'kind': kind,
|
||||
'score': score, 'evidence': evidence, 'created': created,
|
||||
}
|
||||
|
||||
|
||||
def _decision_exists(db: sqlite3.Connection, task: str, overlap_id: str) -> bool:
|
||||
row = db.execute(
|
||||
'SELECT id FROM catalog_decisions WHERE task=? AND overlap_id=?',
|
||||
(task, overlap_id),
|
||||
).fetchone()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def overlap_ack(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
task: str,
|
||||
overlap_id: str,
|
||||
decision: str,
|
||||
decided_by: str,
|
||||
reason: str,
|
||||
authorization_ref: Optional[str] = None,
|
||||
) -> dict:
|
||||
if decision not in DECISIONS:
|
||||
_fail('INVALID_REQUEST', 'Unknown decision', decision=decision)
|
||||
if not reason.strip():
|
||||
_fail('INVALID_REQUEST', 'reason is required for overlap ack')
|
||||
ov = db.execute('SELECT * FROM catalog_overlaps WHERE id=?', (overlap_id,)).fetchone()
|
||||
if not ov:
|
||||
_fail('UNKNOWN_OVERLAP', 'Overlap id not found', id=overlap_id)
|
||||
if ov['probe_task'] != task:
|
||||
_fail('PATH_OUT_OF_SCOPE', 'Overlap belongs to another task', holder=ov['probe_task'])
|
||||
if ov['level'] == 'hard' and decision == 'proceed':
|
||||
_fail(
|
||||
'REQUIREMENT_CONFLICT',
|
||||
'Hard overlaps cannot be acknowledged as proceed; resolve mutex/feature/scope first',
|
||||
overlap_id=overlap_id,
|
||||
kind=ov['kind'],
|
||||
holder=ov['target_task'],
|
||||
)
|
||||
if _decision_exists(db, task, overlap_id):
|
||||
return {'status': 'replayed', 'overlap_id': overlap_id}
|
||||
did = str(uuid.uuid4())
|
||||
db.execute(
|
||||
'INSERT INTO catalog_decisions VALUES(?,?,?,?,?,?,?,?)',
|
||||
(did, overlap_id, task, decision, decided_by, reason.strip(), authorization_ref, _now()),
|
||||
)
|
||||
return {'status': 'acked', 'id': did, 'overlap_id': overlap_id, 'decision': decision}
|
||||
|
||||
|
||||
def gate_task_register(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
task: str,
|
||||
scopes: list[str],
|
||||
requirement: Optional[str] = None,
|
||||
intent: Optional[str] = None,
|
||||
feature: Optional[str] = None,
|
||||
mutex: Optional[str] = None,
|
||||
overlaps_fn: Optional[Callable] = None,
|
||||
statement: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Probe overlaps for task register. Caller must commit then enforce deny codes.
|
||||
|
||||
Returns probe result. Does not raise on soft/hard — register path commits first
|
||||
so soft overlap rows survive OVERLAP_ACK_REQUIRED.
|
||||
"""
|
||||
stmt = statement or intent or ''
|
||||
if not (requirement or stmt or feature or mutex):
|
||||
return {'status': 'skipped', 'reason': 'no requirement/intent/feature/mutex provided'}
|
||||
result = probe_overlap(
|
||||
db, task=task, statement=stmt or (requirement or feature or mutex or task),
|
||||
requirement=requirement, feature=feature, mutex=mutex,
|
||||
scopes=scopes, overlaps_fn=overlaps_fn,
|
||||
)
|
||||
bind_task(db, task=task, requirement=requirement, intent=intent, mutex=mutex, feature=feature)
|
||||
return result
|
||||
|
||||
|
||||
def enforce_gate_result(result: dict) -> None:
|
||||
"""Raise deny codes after the probe transaction has committed."""
|
||||
if result.get('status') == 'skipped':
|
||||
return
|
||||
hard = result.get('hard') or []
|
||||
if hard:
|
||||
first = hard[0]
|
||||
_fail(
|
||||
'REQUIREMENT_CONFLICT',
|
||||
first['evidence'],
|
||||
kind=first['kind'],
|
||||
holder=first.get('target_task'),
|
||||
overlap_id=first['id'],
|
||||
target_req=first.get('target_req'),
|
||||
)
|
||||
if CATALOG_GATE.get('ack_required', True) and result.get('pending_soft'):
|
||||
_fail(
|
||||
'OVERLAP_ACK_REQUIRED',
|
||||
'Semantic overlaps require catalog overlap ack before coding',
|
||||
pending=[h['id'] for h in result['pending_soft']],
|
||||
count=len(result['pending_soft']),
|
||||
)
|
||||
|
||||
|
||||
def pending_acks_for_task(db: sqlite3.Connection, task: str) -> list[dict]:
|
||||
rows = db.execute(
|
||||
'''SELECT o.* FROM catalog_overlaps o
|
||||
WHERE o.probe_task=? AND o.level='soft'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM catalog_decisions d
|
||||
WHERE d.overlap_id=o.id AND d.task=o.probe_task
|
||||
)
|
||||
ORDER BY o.created''',
|
||||
(task,),
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
def assert_acks_clear(db: sqlite3.Connection, task: str) -> None:
|
||||
"""Block first write (with) when soft overlaps are unacked."""
|
||||
if not CATALOG_GATE.get('ack_required', True):
|
||||
return
|
||||
pending = pending_acks_for_task(db, task)
|
||||
if pending:
|
||||
_fail(
|
||||
'OVERLAP_ACK_REQUIRED',
|
||||
'Unresolved soft overlaps; run catalog overlap ack',
|
||||
pending=[p['id'] for p in pending],
|
||||
count=len(pending),
|
||||
)
|
||||
|
||||
|
||||
def catalog_status(db: sqlite3.Connection) -> dict:
|
||||
def count(table: str) -> int:
|
||||
try:
|
||||
return db.execute(f'SELECT COUNT(*) FROM {table}').fetchone()[0]
|
||||
except sqlite3.Error:
|
||||
return 0
|
||||
|
||||
return {
|
||||
'status': 'ok',
|
||||
'requirements': count('catalog_requirements'),
|
||||
'features': count('catalog_features'),
|
||||
'overlaps': count('catalog_overlaps'),
|
||||
'decisions': count('catalog_decisions'),
|
||||
'issues': count('catalog_issues'),
|
||||
'tests': count('catalog_tests'),
|
||||
'releases': count('catalog_releases'),
|
||||
'embeddings': count('catalog_embeddings'),
|
||||
'gate': dict(CATALOG_GATE),
|
||||
'vector': {
|
||||
'enabled': CATALOG_VECTOR.get('enabled'),
|
||||
'fail_open': CATALOG_VECTOR.get('fail_open'),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def catalog_drift(db: sqlite3.Connection, *, matrix_states: dict[str, str]) -> dict:
|
||||
"""Compare catalog_features.matrix_status to live MATRIX states; never writes docs."""
|
||||
regressions = []
|
||||
for fid, status in matrix_states.items():
|
||||
row = db.execute('SELECT matrix_status FROM catalog_features WHERE id=?', (fid,)).fetchone()
|
||||
if row and row['matrix_status'] == 'done' and status != 'done':
|
||||
regressions.append({'record_id': fid, 'catalog': row['matrix_status'], 'matrix': status})
|
||||
feature_upsert(db, feature_id=fid, title=fid, matrix_status=status)
|
||||
open_caps = [k for k, v in matrix_states.items() if v not in ('done', 'removed')]
|
||||
return {
|
||||
'status': 'ok',
|
||||
'record_count': len(matrix_states),
|
||||
'regressions': regressions,
|
||||
'open_capabilities': open_caps,
|
||||
'note': 'catalog features refreshed from MATRIX; never auto-promotes done',
|
||||
}
|
||||
|
||||
|
||||
# --- Vector layer (fail-open) -------------------------------------------------
|
||||
|
||||
def _pack_vec(values: list[float]) -> bytes:
|
||||
return struct.pack(f'<{len(values)}f', *values)
|
||||
|
||||
|
||||
def _unpack_vec(blob: bytes) -> list[float]:
|
||||
n = len(blob) // 4
|
||||
return list(struct.unpack(f'<{n}f', blob))
|
||||
|
||||
|
||||
def _cosine(a: list[float], b: list[float]) -> float:
|
||||
if not a or not b or len(a) != len(b):
|
||||
return 0.0
|
||||
dot = sum(x * y for x, y in zip(a, b))
|
||||
na = math.sqrt(sum(x * x for x in a))
|
||||
nb = math.sqrt(sum(y * y for y in b))
|
||||
if na == 0 or nb == 0:
|
||||
return 0.0
|
||||
return dot / (na * nb)
|
||||
|
||||
|
||||
def _embed_text(text: str) -> tuple[str, Optional[list[float]], str]:
|
||||
"""Return (status, vector_or_None, model). Never raises to caller for network errors."""
|
||||
url = os.environ.get('EMBEDDING_ENDPOINT_URL', '').strip()
|
||||
model = os.environ.get('EMBEDDING_MODEL', 'bge-m3').strip() or 'bge-m3'
|
||||
key = os.environ.get('EMBEDDING_API_KEY', '').strip()
|
||||
if not CATALOG_VECTOR.get('enabled'):
|
||||
return 'disabled', None, model
|
||||
if not url or not key:
|
||||
return 'degraded', None, model
|
||||
timeout = int(CATALOG_VECTOR.get('timeout_seconds', 8))
|
||||
payload = json.dumps({'model': model, 'input': text}).encode('utf-8')
|
||||
req = urllib.request.Request(
|
||||
url, data=payload, method='POST',
|
||||
headers={'Content-Type': 'application/json', 'Authorization': f'Bearer {key}'},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
if resp.status != 200:
|
||||
return 'degraded', None, model
|
||||
body = json.loads(resp.read().decode('utf-8'))
|
||||
data = body.get('data') or []
|
||||
if not data or 'embedding' not in data[0]:
|
||||
return 'degraded', None, model
|
||||
return 'ok', list(data[0]['embedding']), model
|
||||
except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError, json.JSONDecodeError, KeyError, IndexError, TypeError):
|
||||
return 'degraded', None, model
|
||||
|
||||
|
||||
def store_embedding(db: sqlite3.Connection, req_id: str, text: str) -> dict:
|
||||
status, vec, model = _embed_text(text)
|
||||
if status != 'ok' or vec is None:
|
||||
return {'status': status, 'req_id': req_id, 'model': model}
|
||||
blob = _pack_vec(vec)
|
||||
sha = hashlib.sha256(text.encode('utf-8')).hexdigest()
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_embeddings(req_id, model, dim, vec, source_sha256, status, created)
|
||||
VALUES(?,?,?,?,?,?,?)
|
||||
ON CONFLICT(req_id) DO UPDATE SET
|
||||
model=excluded.model, dim=excluded.dim, vec=excluded.vec,
|
||||
source_sha256=excluded.source_sha256, status=excluded.status,
|
||||
created=excluded.created''',
|
||||
(req_id, model, len(vec), blob, sha, 'ok', _now()),
|
||||
)
|
||||
return {'status': 'ok', 'req_id': req_id, 'dim': len(vec), 'model': model}
|
||||
|
||||
|
||||
def _vector_soft_hits(
|
||||
db: sqlite3.Connection,
|
||||
*,
|
||||
task: str,
|
||||
statement: str,
|
||||
requirement: Optional[str],
|
||||
top: int,
|
||||
created: float,
|
||||
) -> tuple[str, list[dict]]:
|
||||
status, probe, model = _embed_text(statement)
|
||||
if status != 'ok' or probe is None:
|
||||
return status, []
|
||||
threshold = float(CATALOG_VECTOR.get('cosine_threshold', 0.82))
|
||||
rows = db.execute(
|
||||
"SELECT req_id, vec FROM catalog_embeddings WHERE status='ok'"
|
||||
).fetchall()
|
||||
scored: list[tuple[float, str]] = []
|
||||
for row in rows:
|
||||
if requirement and row['req_id'] == requirement:
|
||||
continue
|
||||
other = _unpack_vec(row['vec'])
|
||||
sim = _cosine(probe, other)
|
||||
if sim >= threshold:
|
||||
scored.append((sim, row['req_id']))
|
||||
scored.sort(reverse=True)
|
||||
soft = []
|
||||
for sim, rid in scored[:top]:
|
||||
req = db.execute('SELECT * FROM catalog_requirements WHERE id=?', (rid,)).fetchone()
|
||||
if not req:
|
||||
continue
|
||||
if req['source_task'] == task:
|
||||
continue
|
||||
hit = _persist_overlap(
|
||||
db, probe_task=task, probe_req=requirement, target_req=rid,
|
||||
target_task=req['source_task'], target_feature=req['feature'],
|
||||
level='soft', kind='semantic', score=sim,
|
||||
evidence=f'vector cosine={sim:.4f} model={model}',
|
||||
created=created,
|
||||
)
|
||||
soft.append(hit)
|
||||
return 'ok', soft
|
||||
@@ -0,0 +1,214 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One-shot backfill of catalog tables from Markdown / SQLite sources.
|
||||
|
||||
Read-only against .doc and tasks table; writes only catalog_* tables.
|
||||
Does not touch product PostgreSQL. Safe to re-run (upserts / replaces by PK).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
sys.dont_write_bytecode = True
|
||||
import agentctl # noqa: E402
|
||||
import agentctl_catalog as catalog # noqa: E402
|
||||
|
||||
|
||||
def table_rows(content: str) -> list[list[str]]:
|
||||
rows = []
|
||||
for line in content.splitlines():
|
||||
if not line.lstrip().startswith('|'):
|
||||
continue
|
||||
cells = [c.strip() for c in re.split(r'(?<!\\)\|', line.strip().strip('|'))]
|
||||
if cells and all(re.fullmatch(r':?-+:?', c.replace(' ', '') or '-') for c in cells):
|
||||
continue
|
||||
rows.append(cells)
|
||||
return rows
|
||||
|
||||
|
||||
def backfill_matrix(db, repo: agentctl.Repo) -> int:
|
||||
path = repo.main / '.doc/status/FEATURE-MATRIX.md'
|
||||
if not path.is_file():
|
||||
return 0
|
||||
content = path.read_text(encoding='utf-8')
|
||||
states = agentctl.matrix_states(content)
|
||||
# Also capture mutex/title from table when possible.
|
||||
n = 0
|
||||
header = None
|
||||
for cells in table_rows(content):
|
||||
if header is None:
|
||||
if cells and cells[0].casefold() in ('record_id', 'id'):
|
||||
header = [c.casefold() for c in cells]
|
||||
continue
|
||||
if len(cells) != len(header):
|
||||
continue
|
||||
rid = cells[0]
|
||||
title = cells[1] if len(cells) > 1 else rid
|
||||
mutex = None
|
||||
if 'mutex' in header:
|
||||
mutex = cells[header.index('mutex')].strip('`')
|
||||
status = states.get(rid)
|
||||
catalog.feature_upsert(db, feature_id=rid, title=title, mutex=mutex, matrix_status=status)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def backfill_issues(db, repo: agentctl.Repo) -> int:
|
||||
path = repo.main / '.doc/audit/ISSUES.md'
|
||||
if not path.is_file():
|
||||
return 0
|
||||
content = path.read_text(encoding='utf-8')
|
||||
n = 0
|
||||
header = None
|
||||
now = time.time()
|
||||
for cells in table_rows(content):
|
||||
if header is None:
|
||||
if cells and 'issue_id' in cells[0].casefold():
|
||||
header = [c.casefold() for c in cells]
|
||||
continue
|
||||
if not cells or not cells[0].startswith('ISSUE-'):
|
||||
continue
|
||||
issue_id = cells[0]
|
||||
module = cells[1] if len(cells) > 1 else ''
|
||||
severity = cells[3] if len(cells) > 3 else ''
|
||||
state = cells[4] if len(cells) > 4 else ''
|
||||
owner = cells[5] if len(cells) > 5 else ''
|
||||
main_record = cells[7] if len(cells) > 7 else ''
|
||||
lesson = cells[8] if len(cells) > 8 else ''
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_issues(
|
||||
issue_id, severity, issue_state, owner, module, keywords,
|
||||
main_record, lesson_ids, requirements, tasks, recurrence_count,
|
||||
evidence_basis, created, updated)
|
||||
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT(issue_id) DO UPDATE SET
|
||||
severity=excluded.severity, issue_state=excluded.issue_state,
|
||||
owner=excluded.owner, module=excluded.module,
|
||||
main_record=excluded.main_record, lesson_ids=excluded.lesson_ids,
|
||||
updated=excluded.updated''',
|
||||
(
|
||||
issue_id, severity, state, owner, module, module,
|
||||
main_record, lesson, None, None, 0, 'historical', now, now,
|
||||
),
|
||||
)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def guess_verdict(text: str) -> str:
|
||||
upper = text.upper()
|
||||
if 'NOT REQUIRED' in upper or '不记成产品测试 PASS' in text:
|
||||
return 'NOT_REQUIRED'
|
||||
# Prefer explicit Evaluation / Verdict sections.
|
||||
if re.search(r'\bFAIL\b', upper) and 'PASS' not in upper.split('FAIL')[0][-80:]:
|
||||
if re.search(r'判定[：:].{0,40}FAIL|Verdict[：:].{0,20}FAIL|评价判定.{0,40}FAIL', text, re.I):
|
||||
return 'FAIL'
|
||||
if re.search(r'判定[：:].{0,40}PASS|Verdict[：:].{0,20}PASS|评价判定.{0,40}PASS', text, re.I):
|
||||
return 'PASS'
|
||||
if '**PASS**' in text or '| PASS |' in text:
|
||||
return 'PASS'
|
||||
return 'unknown'
|
||||
|
||||
|
||||
def backfill_tests(db, repo: agentctl.Repo) -> int:
|
||||
root = repo.main / '.doc/test-reports'
|
||||
if not root.is_dir():
|
||||
return 0
|
||||
n = 0
|
||||
now = time.time()
|
||||
for path in sorted(root.glob('*.md')):
|
||||
if path.name in ('INDEX.md', 'README.md'):
|
||||
continue
|
||||
text = path.read_text(encoding='utf-8', errors='replace')
|
||||
rel = path.relative_to(repo.main).as_posix()
|
||||
slug = path.stem
|
||||
verdict = guess_verdict(text)
|
||||
real = 1 if re.search(r'真实\s*PI|pi-agent|MiniMax', text, re.I) else 0
|
||||
level = 'doc_only' if verdict == 'NOT_REQUIRED' else 'e2e'
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_tests(
|
||||
id, report_path, task, requirements, scope, verdict, level,
|
||||
real_agent, schema_version, env, sha, created)
|
||||
VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
report_path=excluded.report_path, verdict=excluded.verdict,
|
||||
real_agent=excluded.real_agent''',
|
||||
(slug, rel, None, None, None, verdict, level, real, None, None, None, now),
|
||||
)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def backfill_tasks_as_features(db) -> int:
|
||||
rows = db.execute('SELECT id, scopes, state FROM tasks').fetchall()
|
||||
n = 0
|
||||
for row in rows:
|
||||
# Light projection: task id as feature hint only when looks like feature mutex.
|
||||
scopes = json.loads(row['scopes']) if isinstance(row['scopes'], str) else row['scopes']
|
||||
mutex = scopes[0] if scopes else row['id']
|
||||
catalog.feature_upsert(
|
||||
db,
|
||||
feature_id=f'task:{row["id"]}',
|
||||
title=row['id'],
|
||||
scope=','.join(scopes),
|
||||
mutex=mutex,
|
||||
matrix_status=None,
|
||||
)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def backfill_deployments(db) -> int:
|
||||
try:
|
||||
rows = db.execute('SELECT * FROM deployments').fetchall()
|
||||
except Exception:
|
||||
return 0
|
||||
n = 0
|
||||
now = time.time()
|
||||
for row in rows:
|
||||
db.execute(
|
||||
'''INSERT INTO catalog_releases(
|
||||
id, env, branch, commit_sha, tag, deploy_seq, schema_version,
|
||||
image_digest, source_task, phase, receipt, created)
|
||||
VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
ON CONFLICT(id) DO UPDATE SET phase=excluded.phase, receipt=excluded.receipt''',
|
||||
(
|
||||
row['id'], row['env'], None, row['expected_commit'], None, None, None,
|
||||
None, row['task'], row['phase'], row['receipt'], row['created'] or now,
|
||||
),
|
||||
)
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--repo', default='.')
|
||||
args = parser.parse_args()
|
||||
repo = agentctl.Repo(args.repo)
|
||||
agentctl.require_policy(repo)
|
||||
catalog.bind_deny(agentctl.deny)
|
||||
with repo.db(True) as db:
|
||||
counts = {
|
||||
'features_matrix': backfill_matrix(db, repo),
|
||||
'issues': backfill_issues(db, repo),
|
||||
'tests': backfill_tests(db, repo),
|
||||
'task_features': backfill_tasks_as_features(db),
|
||||
'releases': backfill_deployments(db),
|
||||
}
|
||||
status = catalog.catalog_status(db)
|
||||
agentctl.emit({'status': 'ok', 'backfill': counts, 'catalog': status})
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
sys.exit(main())
|
||||
except agentctl.Denied as exc:
|
||||
agentctl.emit({'status': 'denied', 'code': exc.code, 'message': exc.message, **exc.details}, sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -153,6 +153,76 @@ const TOOLS = [
|
||||
description: 'List quarantine / suspect runs',
|
||||
inputSchema: { type: 'object', properties: {}, additionalProperties: false },
|
||||
},
|
||||
{
|
||||
name: 'catalog_requirement_add',
|
||||
description: 'Add a catalog requirement (R-ID)',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
title: { type: 'string' },
|
||||
statement: { type: 'string' },
|
||||
owner: { type: 'string' },
|
||||
feature: { type: 'string' },
|
||||
source_doc: { type: 'string' },
|
||||
source_task: { type: 'string' },
|
||||
},
|
||||
required: ['id', 'title', 'statement', 'owner'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'catalog_overlap_probe',
|
||||
description: 'Probe hard/soft requirement overlaps for a task',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
task: { type: 'string' },
|
||||
statement: { type: 'string' },
|
||||
requirement: { type: 'string' },
|
||||
feature: { type: 'string' },
|
||||
mutex: { type: 'string' },
|
||||
top: { type: 'number' },
|
||||
},
|
||||
required: ['task', 'statement'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'catalog_overlap_ack',
|
||||
description: 'Acknowledge a soft overlap with an explicit decision',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
task: { type: 'string' },
|
||||
overlap: { type: 'string' },
|
||||
decision: { type: 'string', enum: ['proceed', 'merge_into', 'supersede', 'defer', 'reject'] },
|
||||
reason: { type: 'string' },
|
||||
decided_by: { type: 'string' },
|
||||
authorization_ref: { type: 'string' },
|
||||
},
|
||||
required: ['task', 'overlap', 'decision', 'reason'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'catalog_search',
|
||||
description: 'FTS5 CJK search over catalog requirements',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
query: { type: 'string' },
|
||||
top: { type: 'number' },
|
||||
},
|
||||
required: ['query'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'catalog_status',
|
||||
description: 'Catalog table counts and gate/vector config summary',
|
||||
inputSchema: { type: 'object', properties: {}, additionalProperties: false },
|
||||
},
|
||||
];
|
||||
|
||||
function callTool(name, args = {}) {
|
||||
@@ -217,6 +287,52 @@ function callTool(name, args = {}) {
|
||||
const r = runAgentctl(['quarantine', 'list']);
|
||||
return r.ok ? textResult(r.stdout.trim() || '{"items":[]}') : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
case 'catalog_requirement_add': {
|
||||
const argv = [
|
||||
'catalog', 'add',
|
||||
'--id', args.id,
|
||||
'--title', args.title,
|
||||
'--statement', args.statement,
|
||||
'--owner', args.owner,
|
||||
];
|
||||
if (args.feature) argv.push('--feature', args.feature);
|
||||
if (args.source_doc) argv.push('--source-doc', args.source_doc);
|
||||
if (args.source_task) argv.push('--source-task', args.source_task);
|
||||
const r = runAgentctl(argv);
|
||||
return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
case 'catalog_overlap_probe': {
|
||||
const argv = ['catalog', 'overlap', '--task', args.task, '--statement', args.statement];
|
||||
if (args.requirement) argv.push('--requirement', args.requirement);
|
||||
if (args.feature) argv.push('--feature', args.feature);
|
||||
if (args.mutex) argv.push('--mutex', args.mutex);
|
||||
if (args.top != null) argv.push('--top', String(args.top));
|
||||
const r = runAgentctl(argv);
|
||||
return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
case 'catalog_overlap_ack': {
|
||||
const argv = [
|
||||
'catalog', 'ack',
|
||||
'--task', args.task,
|
||||
'--overlap', args.overlap,
|
||||
'--decision', args.decision,
|
||||
'--reason', args.reason,
|
||||
];
|
||||
if (args.decided_by) argv.push('--decided-by', args.decided_by);
|
||||
if (args.authorization_ref) argv.push('--authorization-ref', args.authorization_ref);
|
||||
const r = runAgentctl(argv);
|
||||
return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
case 'catalog_search': {
|
||||
const argv = ['catalog', 'search', '--query', args.query];
|
||||
if (args.top != null) argv.push('--top', String(args.top));
|
||||
const r = runAgentctl(argv);
|
||||
return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
case 'catalog_status': {
|
||||
const r = runAgentctl(['catalog', 'status']);
|
||||
return r.ok ? textResult(r.stdout.trim()) : errResult(r.stderr || r.stdout || r.error);
|
||||
}
|
||||
default:
|
||||
return errResult(`unknown tool: ${name}`);
|
||||
}
|
||||
|
||||
@@ -24,7 +24,7 @@ class WorktreeTest(unittest.TestCase):
|
||||
self.root = Path(self.tmp.name) / "repo"
|
||||
self.root.mkdir()
|
||||
(self.root / "scripts").mkdir()
|
||||
for name in ("agent-worktree.sh", "agent_worktree.py", "agentctl.py", "agentctl", "agentctl_w4.py"):
|
||||
for name in ("agent-worktree.sh", "agent_worktree.py", "agentctl.py", "agentctl", "agentctl_w4.py", "agentctl_catalog.py"):
|
||||
shutil.copy2(SOURCE / name, self.root / "scripts" / name)
|
||||
(self.root / "configs").mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(SOURCE.parent / "configs" / "agent-governance.json",
|
||||
|
||||
@@ -37,6 +37,7 @@ class AgentctlTests(unittest.TestCase):
|
||||
(self.root / "scripts").mkdir()
|
||||
shutil.copy2(SOURCE, self.root / "scripts" / "agentctl.py")
|
||||
shutil.copy2(SOURCE.parent / "agentctl_w4.py", self.root / "scripts" / "agentctl_w4.py")
|
||||
shutil.copy2(SOURCE.parent / "agentctl_catalog.py", self.root / "scripts" / "agentctl_catalog.py")
|
||||
install_governance_policy(self.root)
|
||||
(self.root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8")
|
||||
for directory in ("src/a", "src/b", ".doc/ops", ".doc/status"):
|
||||
@@ -1551,6 +1552,7 @@ class ProcessPlatformTests(unittest.TestCase):
|
||||
(root / "scripts").mkdir()
|
||||
shutil.copy2(SOURCE, root / "scripts" / "agentctl.py")
|
||||
shutil.copy2(SOURCE.parent / "agentctl_w4.py", root / "scripts" / "agentctl_w4.py")
|
||||
shutil.copy2(SOURCE.parent / "agentctl_catalog.py", root / "scripts" / "agentctl_catalog.py")
|
||||
install_governance_policy(root)
|
||||
(root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8")
|
||||
(root / ".doc/ops").mkdir(parents=True)
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Catalog governance regressions (schema, FTS5 CJK, overlap gates, vector degrade)."""
|
||||
|
||||
import json
|
||||
import pathlib
|
||||
import shutil
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
SOURCE = pathlib.Path(__file__).resolve().parents[1] / "agentctl.py"
|
||||
CATALOG_SOURCE = pathlib.Path(__file__).resolve().parents[1] / "agentctl_catalog.py"
|
||||
POLICY_SOURCE = SOURCE.parents[1] / "configs" / "agent-governance.json"
|
||||
|
||||
|
||||
def install_governance_policy(root: pathlib.Path) -> None:
|
||||
dest = root / "configs"
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(POLICY_SOURCE, dest / "agent-governance.json")
|
||||
|
||||
|
||||
class CatalogTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix="agentctl-catalog-")
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = pathlib.Path(self.temp.name) / "repo"
|
||||
self.root.mkdir()
|
||||
self.git("init", "-b", "dev")
|
||||
self.git("config", "user.name", "Catalog Test")
|
||||
self.git("config", "user.email", "catalog-test@example.invalid")
|
||||
(self.root / "scripts").mkdir()
|
||||
shutil.copy2(SOURCE, self.root / "scripts" / "agentctl.py")
|
||||
shutil.copy2(SOURCE.parent / "agentctl_w4.py", self.root / "scripts" / "agentctl_w4.py")
|
||||
shutil.copy2(CATALOG_SOURCE, self.root / "scripts" / "agentctl_catalog.py")
|
||||
install_governance_policy(self.root)
|
||||
(self.root / ".gitignore").write_text(".locks/\n.worktrees/\n", encoding="utf-8")
|
||||
for directory in ("src/a", "src/b", ".doc/ops", ".doc/status"):
|
||||
(self.root / directory).mkdir(parents=True, exist_ok=True)
|
||||
(self.root / ".doc/ops/CURRENT.md").write_text(
|
||||
"# Coordination\n\n| owner | task | branch | worktree | mutex | status |\n"
|
||||
"| --- | --- | --- | --- | --- | --- |\n"
|
||||
"| Alice | alpha | feature/alpha | .worktrees/alpha | src | in_progress |\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(self.root / ".doc/status/FEATURE-MATRIX.md").write_text(
|
||||
"# Features\n\n| record_id | 模块 | 状态 |\n| --- | --- | --- |\n"
|
||||
"| core | core | planned |\n| intent-plan | 多意图 | done |\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
self.git("add", ".")
|
||||
self.git("commit", "-m", "fixture")
|
||||
self.ctl("init")
|
||||
|
||||
def git(self, *args, cwd=None):
|
||||
return subprocess.run(
|
||||
["git", *args], cwd=cwd or self.root, text=True, capture_output=True, check=True,
|
||||
).stdout.strip()
|
||||
|
||||
def ctl(self, *args, check=True, cwd=None):
|
||||
result = subprocess.run(
|
||||
["python3", "-B", str(self.root / "scripts" / "agentctl.py"), *args],
|
||||
cwd=cwd or self.root, text=True, capture_output=True,
|
||||
)
|
||||
if check and result.returncode != 0:
|
||||
raise AssertionError(result.stderr or result.stdout or f"exit {result.returncode}")
|
||||
out = (result.stdout or "").strip()
|
||||
err = (result.stderr or "").strip()
|
||||
payload = None
|
||||
for stream in (out, err):
|
||||
if not stream:
|
||||
continue
|
||||
try:
|
||||
payload = json.loads(stream.splitlines()[-1])
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return result.returncode, payload, out, err
|
||||
|
||||
def make_worktree(self, task_id: str):
|
||||
wt = self.root / ".worktrees" / task_id
|
||||
self.git("worktree", "add", "-b", f"feature/{task_id}", str(wt))
|
||||
return wt
|
||||
|
||||
def test_schema_and_crud(self):
|
||||
code, payload, _, _ = self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-demo-overlap",
|
||||
"--title", "多意图持久计划",
|
||||
"--statement", "多意图持久计划 shadow 灰度放量",
|
||||
"--owner", "Cursor",
|
||||
"--feature", "intent-plan",
|
||||
)
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(payload["status"], "added")
|
||||
_, listed, _, _ = self.ctl("catalog", "list")
|
||||
self.assertEqual(listed["status"], "ok")
|
||||
self.assertEqual(len(listed["requirements"]), 1)
|
||||
_, shown, _, _ = self.ctl("catalog", "show", "--id", "R-demo-overlap")
|
||||
self.assertEqual(shown["requirement"]["id"], "R-demo-overlap")
|
||||
_, status, _, _ = self.ctl("catalog", "status")
|
||||
self.assertGreaterEqual(status["requirements"], 1)
|
||||
|
||||
def test_cjk_fts_digram_recall(self):
|
||||
"""CJK digrams must hit under unicode61 char-split (trigram would miss 灰度)."""
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-gray",
|
||||
"--title", "灰度放量",
|
||||
"--statement", "多意图持久计划 shadow 灰度放量",
|
||||
"--owner", "Cursor",
|
||||
)
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-other",
|
||||
"--title", "语义摘要",
|
||||
"--statement", "语义摘要 REPLACE fail-closed 灰度",
|
||||
"--owner", "Cursor",
|
||||
)
|
||||
# Direct FTS verification including trigram contrast.
|
||||
import sys
|
||||
sys.path.insert(0, str(self.root / "scripts"))
|
||||
import agentctl_catalog as cat
|
||||
db = sqlite3.connect(self.root / ".locks" / "state.sqlite")
|
||||
db.row_factory = sqlite3.Row
|
||||
# Create trigram index for contrast.
|
||||
db.execute("CREATE VIRTUAL TABLE IF NOT EXISTS trigram_probe USING fts5(body, tokenize='trigram')")
|
||||
db.execute("INSERT INTO trigram_probe VALUES (?)", ("多意图持久计划 shadow 灰度放量",))
|
||||
db.execute("INSERT INTO trigram_probe VALUES (?)", ("语义摘要 REPLACE fail-closed 灰度",))
|
||||
tri = db.execute("SELECT count(*) FROM trigram_probe WHERE trigram_probe MATCH ?", ('"灰度"',)).fetchone()[0]
|
||||
self.assertEqual(tri, 0, "trigram must miss digram 灰度")
|
||||
hits = cat.search(db, "灰度", top=10)
|
||||
self.assertGreaterEqual(len(hits), 2, f"CJK split must recall digram; got {hits}")
|
||||
db.close()
|
||||
_, payload, _, _ = self.ctl("catalog", "search", "--query", "灰度")
|
||||
self.assertGreaterEqual(len(payload["hits"]), 2)
|
||||
|
||||
def test_hard_same_req_id_conflict(self):
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-owned",
|
||||
"--title", "owned",
|
||||
"--statement", "already owned requirement",
|
||||
"--owner", "Alice",
|
||||
"--source-task", "alpha-owner",
|
||||
)
|
||||
wt = self.make_worktree("beta-conflict")
|
||||
code, payload, _, err = self.ctl(
|
||||
"task", "register",
|
||||
"--id", "beta-conflict",
|
||||
"--owner", "Bob",
|
||||
"--scope", "src/b",
|
||||
"--requirement", "R-owned",
|
||||
"--intent", "already owned requirement",
|
||||
cwd=wt,
|
||||
check=False,
|
||||
)
|
||||
self.assertNotEqual(code, 0)
|
||||
denied = payload or json.loads(err.splitlines()[-1])
|
||||
self.assertEqual(denied["code"], "REQUIREMENT_CONFLICT")
|
||||
self.assertEqual(denied["kind"], "same_req_id")
|
||||
|
||||
def test_soft_overlap_requires_ack(self):
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-existing",
|
||||
"--title", "治理目录库",
|
||||
"--statement", "跨对话需求重叠检测与治理目录库",
|
||||
"--owner", "Alice",
|
||||
"--source-task", "alpha-owner",
|
||||
)
|
||||
wt = self.make_worktree("soft-probe")
|
||||
# Register without catalog args first.
|
||||
self.ctl(
|
||||
"task", "register",
|
||||
"--id", "soft-probe",
|
||||
"--owner", "Bob",
|
||||
"--scope", "src/b",
|
||||
cwd=wt,
|
||||
)
|
||||
code, payload, _, _ = self.ctl(
|
||||
"catalog", "overlap",
|
||||
"--task", "soft-probe",
|
||||
"--statement", "跨对话需求重叠检测治理目录",
|
||||
"--requirement", "R-new-soft",
|
||||
)
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(payload["status"], "ack_required")
|
||||
self.assertTrue(payload["pending_soft"])
|
||||
oid = payload["pending_soft"][0]["id"]
|
||||
# Exclusive with blocked until ack.
|
||||
code, denied, _, err = self.ctl(
|
||||
"with", "--task", "soft-probe", "--resource", "path:src/b", "--", "true",
|
||||
cwd=wt, check=False,
|
||||
)
|
||||
self.assertNotEqual(code, 0)
|
||||
body = denied or json.loads(err.splitlines()[-1])
|
||||
self.assertEqual(body["code"], "OVERLAP_ACK_REQUIRED")
|
||||
self.ctl(
|
||||
"catalog", "ack",
|
||||
"--task", "soft-probe",
|
||||
"--overlap", oid,
|
||||
"--decision", "proceed",
|
||||
"--reason", "incremental extension of same theme",
|
||||
"--decided-by", "Bob",
|
||||
)
|
||||
code, _, _, _ = self.ctl(
|
||||
"with", "--task", "soft-probe", "--resource", "path:src/b", "--", "true",
|
||||
cwd=wt,
|
||||
)
|
||||
self.assertEqual(code, 0)
|
||||
|
||||
def test_vector_layer_degrades_without_key(self):
|
||||
# Enable vector in policy for this repo.
|
||||
policy_path = self.root / "configs" / "agent-governance.json"
|
||||
data = json.loads(policy_path.read_text(encoding="utf-8"))
|
||||
data["catalog_vector"] = {
|
||||
"enabled": True, "timeout_seconds": 2, "fail_open": True, "cosine_threshold": 0.82,
|
||||
}
|
||||
policy_path.write_text(json.dumps(data, indent=2), encoding="utf-8")
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-vec",
|
||||
"--title", "向量降级",
|
||||
"--statement", "向量库可降级不阻断门禁",
|
||||
"--owner", "Cursor",
|
||||
)
|
||||
# No EMBEDDING_API_KEY → degraded, still returns FTS soft or none without crash.
|
||||
env = {"EMBEDDING_ENDPOINT_URL": "http://127.0.0.1:9/embeddings", "EMBEDDING_API_KEY": ""}
|
||||
result = subprocess.run(
|
||||
["python3", "-B", str(self.root / "scripts" / "agentctl.py"),
|
||||
"catalog", "overlap", "--task", "no-task-yet", "--statement", "向量库可降级不阻断"],
|
||||
cwd=self.root, text=True, capture_output=True,
|
||||
env={**dict(**{k: v for k, v in __import__('os').environ.items()}), **env},
|
||||
)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
payload = json.loads(result.stdout.strip().splitlines()[-1])
|
||||
self.assertIn(payload.get("vector_layer"), ("degraded", "disabled", "ok"))
|
||||
|
||||
def test_hard_proceed_ack_rejected(self):
|
||||
self.ctl(
|
||||
"catalog", "add",
|
||||
"--id", "R-hard",
|
||||
"--title", "mutex feature",
|
||||
"--statement", "same feature ownership",
|
||||
"--owner", "Alice",
|
||||
"--feature", "core",
|
||||
"--source-task", "holder-a",
|
||||
)
|
||||
wt = self.make_worktree("hard-ack")
|
||||
self.ctl("task", "register", "--id", "hard-ack", "--owner", "Bob", "--scope", "src/b", cwd=wt)
|
||||
_, payload, _, _ = self.ctl(
|
||||
"catalog", "overlap",
|
||||
"--task", "hard-ack",
|
||||
"--statement", "same feature ownership",
|
||||
"--feature", "core",
|
||||
"--requirement", "R-hard-b",
|
||||
)
|
||||
self.assertTrue(payload["hard"])
|
||||
oid = payload["hard"][0]["id"]
|
||||
code, denied, _, err = self.ctl(
|
||||
"catalog", "ack",
|
||||
"--task", "hard-ack",
|
||||
"--overlap", oid,
|
||||
"--decision", "proceed",
|
||||
"--reason", "should fail",
|
||||
check=False,
|
||||
)
|
||||
self.assertNotEqual(code, 0)
|
||||
body = denied or json.loads(err.splitlines()[-1])
|
||||
self.assertEqual(body["code"], "REQUIREMENT_CONFLICT")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -21,7 +21,7 @@ class CodegraphGovernanceTest(unittest.TestCase):
|
||||
self.root.mkdir()
|
||||
(self.root / 'scripts').mkdir()
|
||||
for name in ('agent-worktree.sh', 'agent_worktree.py', 'agentctl.py',
|
||||
'agentctl', 'agentctl_w4.py'):
|
||||
'agentctl', 'agentctl_w4.py', 'agentctl_catalog.py'):
|
||||
shutil.copy2(SOURCE / name, self.root / 'scripts' / name)
|
||||
(self.root / 'configs').mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(SOURCE.parent / 'configs' / 'agent-governance.json',
|
||||
|
||||
@@ -19,7 +19,7 @@ class OpsGuardsTest(unittest.TestCase):
|
||||
self.root = Path(self.tmp.name) / "repo"
|
||||
self.root.mkdir()
|
||||
(self.root / "scripts").mkdir()
|
||||
for name in ("env-service.sh", "candidate-images-build.sh", "agentctl.py", "agentctl", "agentctl_w4.py"):
|
||||
for name in ("env-service.sh", "candidate-images-build.sh", "agentctl.py", "agentctl", "agentctl_w4.py", "agentctl_catalog.py"):
|
||||
shutil.copy2(SOURCE / name, self.root / "scripts" / name)
|
||||
(self.root / ".gitignore").write_text(".worktrees/\n.locks/\nconfigs/.env.*\n!configs/.env.*.example\ndata/\n")
|
||||
(self.root / ".dockerignore").write_text(".doc\n.locks\n.worktrees\n")
|
||||
|
||||
@@ -146,6 +146,9 @@ class PiSubagentsTests(unittest.TestCase):
|
||||
shutil.copy2(SOURCE_AGENTCTL, self.root / "scripts" / "agentctl.py")
|
||||
if SOURCE_W4.exists():
|
||||
shutil.copy2(SOURCE_W4, self.root / "scripts" / "agentctl_w4.py")
|
||||
catalog_src = ROOT / "scripts" / "agentctl_catalog.py"
|
||||
if catalog_src.exists():
|
||||
shutil.copy2(catalog_src, self.root / "scripts" / "agentctl_catalog.py")
|
||||
if SOURCE_EXEC.exists():
|
||||
shutil.copy2(SOURCE_EXEC, self.root / "scripts" / "pi_subagents.py")
|
||||
if SOURCE_MCP.exists():
|
||||
|
||||
Reference in New Issue
Block a user