merge: fix/doccontent-canonical-boundary
This commit is contained in:
weichen
+4
-3
@@ -5,7 +5,7 @@
|
||||
|
||||
## 焦点
|
||||
|
||||
1. **TRACK-E**: [持久事件交付](./short-term/2026-09-15-durable-event-track-e.md)（E0A–E2 隔离库 PASS；dispatcher 默认 off。删除 hook API 须等 live 窗口。复盘见 [research](./research/2026-09-15-track-e-retrospective.md)）
|
||||
1. **TRACK-S**: [D6 doccontent](./short-term/2026-09-15-doccontent-canonical-boundary.md)（Track E 复盘已归档；dispatcher 仍 off。下一波 D7a/D5/D4）
|
||||
2. **CHAT-INTENT**: [对话意图运行时状态](../test-reports/2026-09-13-conversation-intent-runtime-state-test-report.md)（双标签 busy 已签：项目 **6275** / `DOC_BUSY` 单 run；不删 `sess-p143-u9` 探针）
|
||||
3. **CHAT**: [对话历史刷新活验已补](../test-reports/2026-09-13-chat-history-refresh-test-report.md)（项目 6252 闲聊刷新同集 + guest 403；编制任务链仍开放）
|
||||
|
||||
@@ -63,7 +63,8 @@
|
||||
| Cursor | event-e1a-notify-cutover | fix/event-notify-cutover | - | events | done |
|
||||
| Cursor | event-e1b-workflow-advance | fix/event-workflow-advance | - | events | done |
|
||||
| Cursor | event-e1c-projection-cutover | fix/event-projection-cutover | - | events | done |
|
||||
| Cursor | event-e2-hook-guard | fix/event-e2-hook-guard | - | events | in_progress |
|
||||
| Cursor | event-e2-hook-guard | fix/event-e2-hook-guard | - | events | done |
|
||||
| Cursor | doccontent-canonical-boundary | fix/doccontent-canonical-boundary | - | docgov | in_progress |
|
||||
|
||||
> 新任务开始前登记。`branch` 必须为 `feature/<slug>` 或 `fix/<slug>`，**禁止** `dev` / `test` / `main`。同一 `mutex` 同时仅一个 `in_progress`。
|
||||
|
||||
@@ -75,7 +76,7 @@
|
||||
|
||||
| holder | phase | since | scope | queue |
|
||||
|--------|-------|-------|-------|-------|
|
||||
| Cursor | merging | 2026-09-15 | E2 hook live-guard；mode=off | - |
|
||||
| Cursor | merging | 2026-09-15 | D6 doccontent canonical；mode=off | - |
|
||||
|
||||
**phase**: `idle` | `merging` | `smoke` | `e2e`
|
||||
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
---
|
||||
title: "D6 doccontent canonical 边界"
|
||||
status: in_progress
|
||||
created: 2026-09-15
|
||||
updated: 2026-09-15
|
||||
owner: Cursor
|
||||
branch: fix/doccontent-canonical-boundary
|
||||
worktree: "-"
|
||||
mutex: docgov
|
||||
---
|
||||
|
||||
# D6 doccontent canonical 边界
|
||||
|
||||
## 目标
|
||||
|
||||
Markdown canonicalization 与 SHA 抽到 `internal/doccontent`；生产 `docgov` 不再 import `docrevision`。
|
||||
|
||||
## 范围
|
||||
|
||||
- `internal/doccontent`
|
||||
- `docgov/lease.go` 改走 doccontent
|
||||
- `docrevision` 薄封装保持旧调用方
|
||||
- golden + wrapper 对账
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不改 hash 语义
|
||||
- 不碰项目 143 数据
|
||||
- 不删 `docrevision.CanonicalizeMarkdown` 导出（兼容）
|
||||
|
||||
## 步骤
|
||||
|
||||
1. [x] 分支与 CURRENT
|
||||
2. [x] 提取模块与切断 docgov→docrevision
|
||||
3. [x] golden / arch-check
|
||||
4. [x] 报告并合入
|
||||
|
||||
## 验收标准
|
||||
|
||||
- [x] 生产 docgov 不 import docrevision
|
||||
- [x] project143 hard-break golden 仍过
|
||||
- [x] [S3 报告](../../test-reports/2026-09-15-doccontent-canonical-boundary-test-report.md)
|
||||
|
||||
## 相关链接
|
||||
|
||||
- Phase 2: [../../remediation/2026-09-15-module-decoupling-ipc-phase2-remediation-plan.md](../../remediation/2026-09-15-module-decoupling-ipc-phase2-remediation-plan.md)
|
||||
@@ -27,7 +27,7 @@
|
||||
| PI Runner / 配置链 | pi-agent + MiniMax 真实链路 | done | [test piagent](../test-reports/2026-09-13-codex-audit-p0-remediation-test-report.md)；[container-env](../test-reports/2026-09-13-container-env-configuration-test-report.md) | — | `pi-agent` |
|
||||
| 三环境运维 | dev/test/main + 共享 KB | done | [PROJECT.md](../../PROJECT.md) §4；四环境 healthy | — | `infra` |
|
||||
| CI / E2E | PG 集成 + 真实 PI E2E | partial | [gap-matrix BUG-08](../remediation/gap-matrix.md)；[test pg-ci](../test-reports/2026-09-12-pg-ci-soak-e2e-test-report.md)；[运行时 UI Round20](../test-reports/2026-09-14-document-revision-runtime-ui-test-report.md) | CI `TEST_DATABASE_URL` 隔离库仍缺；文档运行时真实 UI 已按 UI_PHASE 分文件且 post-restart 断言最终 A1 正文。`-race` 全包并行 TempDir flake 为基建残留 | `ci` |
|
||||
| 模块解耦与可靠事件 | 文档可靠性纵向试点、PG 持久事件交付与选择性模块化 | partial | [架构](../system-design/2026-09-14-module-decoupling-architecture.md)；[ADR](../adr/2026-09-14-modular-monolith-decoupling.md)；[Phase 1](../remediation/2026-09-14-module-decoupling-refactor-plan.md)；[Phase 2 审阅](../audit/2026-09-15-module-decoupling-phase2-plan-review.md)；[Phase 2 修订计划](../remediation/2026-09-15-module-decoupling-ipc-phase2-remediation-plan.md)；[E0A](../test-reports/2026-09-15-event-e0a-production-scope-contract-test-report.md)；[E0B](../test-reports/2026-09-15-event-e0b-dispatcher-pg-test-report.md)；[E1A](../test-reports/2026-09-15-event-e1a-notify-cutover-test-report.md)；[E1B](../test-reports/2026-09-15-event-e1b-workflow-advance-test-report.md)；[E1C](../test-reports/2026-09-15-event-e1c-projection-test-report.md)；[E2](../test-reports/2026-09-15-event-e2-hook-guard-test-report.md)；[复盘](../ops/research/2026-09-15-track-e-retrospective.md) | E0A–E2 隔离库已过；dispatcher 默认 off。删除闭包 API 须等 live。Track S/P 与 FAV 另列 | `architecture-docs` |
|
||||
| 模块解耦与可靠事件 | 文档可靠性纵向试点、PG 持久事件交付与选择性模块化 | partial | [架构](../system-design/2026-09-14-module-decoupling-architecture.md)；[ADR](../adr/2026-09-14-modular-monolith-decoupling.md)；[Phase 1](../remediation/2026-09-14-module-decoupling-refactor-plan.md)；[Phase 2 审阅](../audit/2026-09-15-module-decoupling-phase2-plan-review.md)；[Phase 2 修订计划](../remediation/2026-09-15-module-decoupling-ipc-phase2-remediation-plan.md)；[E0A](../test-reports/2026-09-15-event-e0a-production-scope-contract-test-report.md)；[E0B](../test-reports/2026-09-15-event-e0b-dispatcher-pg-test-report.md)；[E1A](../test-reports/2026-09-15-event-e1a-notify-cutover-test-report.md)；[E1B](../test-reports/2026-09-15-event-e1b-workflow-advance-test-report.md)；[E1C](../test-reports/2026-09-15-event-e1c-projection-test-report.md)；[E2](../test-reports/2026-09-15-event-e2-hook-guard-test-report.md)；[复盘](../ops/research/2026-09-15-track-e-retrospective.md)；[D6](../test-reports/2026-09-15-doccontent-canonical-boundary-test-report.md) | E0A–E2 隔离库已过；dispatcher 默认 off。删除闭包 API 须等 live。D6 canonical 已抽出。Track S 其余与 Track P/FAV 另列 | `architecture-docs` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -114,12 +114,14 @@ imports:
|
||||
- docrevision
|
||||
docgov:
|
||||
- app/uow
|
||||
- doccontent
|
||||
- docduty
|
||||
- docrevision
|
||||
- fsutil
|
||||
docrevision:
|
||||
- app/uow
|
||||
- doccontent
|
||||
- fsutil
|
||||
doccontent: []
|
||||
freedoc:
|
||||
- app/uow
|
||||
- ctxmgr
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
# D6 doccontent canonical 边界测试报告
|
||||
|
||||
## 一、 测试背景 (Background)
|
||||
|
||||
`docgov` 生产代码为写正文 SHA 直接 import `docrevision` 的 canonicalizer，形成单向工具依赖。Phase 2 D6 要求把 Markdown canonicalization 与 SHA 抽成共享行为模块，且完成后 docgov/docrevision **生产包互不 import**。
|
||||
|
||||
## 二、 测试目标 (Test Objectives)
|
||||
|
||||
- `internal/doccontent` 持有唯一算法实现；hard-break 双空格不被 trim。
|
||||
- `docrevision` 导出函数是薄封装，字节级一致。
|
||||
- 生产 `docgov` 不再 import `docrevision`。
|
||||
- arch-check block 通过。
|
||||
|
||||
## 三、 测试方案与环境 (Test Plan & Environment)
|
||||
|
||||
| 项 | 值 |
|
||||
|----|-----|
|
||||
| OS | Linux 7.0.0-31-generic |
|
||||
| Go | go1.27.1 linux/amd64 |
|
||||
| 分支 | `fix/doccontent-canonical-boundary` |
|
||||
| 项目 143 | 只读；golden 使用其 hard-break 形状，不写库 |
|
||||
| 证据 | 本报告命令输出 |
|
||||
|
||||
## 四、 预期结果 (Expected Results)
|
||||
|
||||
| 用例 | 期望 |
|
||||
|------|------|
|
||||
| hard-break golden | 5×` \\n` 保留；legacy 少 10 字节 |
|
||||
| wrapper | 与 doccontent 字节一致 |
|
||||
| arch-check | docgov 无 docrevision import |
|
||||
|
||||
## 五、 实际测试结果与判定 (Actual Results & Evaluation)
|
||||
|
||||
### 5.1 golden 与封装
|
||||
|
||||
- **测试输入 (Input)**：`go test -count=1 ./internal/doccontent ./internal/docgov ./internal/docrevision ./internal/archtest`
|
||||
- **实际输出 (Actual Output)**：四包 `ok`（doccontent 0.002s，docgov 0.014s，docrevision 0.006s，archtest 0.121s）
|
||||
- **评价判定 (Evaluation / Verdict)**：**PASS**
|
||||
- **性能与耗时 (Duration & Latency)**：~0.15s
|
||||
|
||||
### 5.2 arch-check
|
||||
|
||||
- **测试输入 (Input)**：`./scripts/arch-check.sh --mode block`
|
||||
- **实际输出 (Actual Output)**：`arch-check block: production gate passed`
|
||||
- **评价判定 (Evaluation / Verdict)**：**PASS**
|
||||
- **性能与耗时 (Duration & Latency)**：~0.12s
|
||||
|
||||
## 六、 自动化测试套件汇总 (Test Suite Execution)
|
||||
|
||||
| 命令 | 结果 |
|
||||
|------|------|
|
||||
| doccontent/docgov/docrevision/archtest | PASS |
|
||||
| arch-check block | PASS |
|
||||
| 语义 | 未改变 canonical 字节 |
|
||||
|
||||
## 七、 测试结论与后续优化建议 (Conclusion & Next Steps)
|
||||
|
||||
D6 **PASS**。`docgov` 生产路径已与 `docrevision` 解耦。后续 D5/D7a 不依赖本包。调用方可逐步改为直接 import `doccontent`，封装保留至无外部引用。
|
||||
@@ -0,0 +1,45 @@
|
||||
package doccontent
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/text/unicode/norm"
|
||||
)
|
||||
|
||||
// CanonicalizeMarkdown applies LF, UTF-8 NFC, and a single trailing newline.
|
||||
// It must not trim trailing spaces or tabs: two trailing spaces are Markdown hard breaks.
|
||||
func CanonicalizeMarkdown(raw string) string {
|
||||
s := strings.ReplaceAll(raw, "\r\n", "\n")
|
||||
s = strings.ReplaceAll(s, "\r", "\n")
|
||||
s = norm.NFC.String(s)
|
||||
if s != "" && !strings.HasSuffix(s, "\n") {
|
||||
s += "\n"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// LegacyDestructiveCanonicalizeMarkdown reproduces the pre-fix canonicalizer that
|
||||
// stripped per-line trailing spaces/tabs. Used only to classify historical drift.
|
||||
func LegacyDestructiveCanonicalizeMarkdown(raw string) string {
|
||||
s := strings.ReplaceAll(raw, "\r\n", "\n")
|
||||
s = strings.ReplaceAll(s, "\r", "\n")
|
||||
s = norm.NFC.String(s)
|
||||
lines := strings.Split(s, "\n")
|
||||
for i, line := range lines {
|
||||
lines[i] = strings.TrimRightFunc(line, func(r rune) bool {
|
||||
return r == ' ' || r == '\t'
|
||||
})
|
||||
}
|
||||
s = strings.Join(lines, "\n")
|
||||
if s != "" && !strings.HasSuffix(s, "\n") {
|
||||
s += "\n"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func SHA256Hex(content string) string {
|
||||
sum := sha256.Sum256([]byte(content))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package doccontent_test
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"chat-agent-system/internal/doccontent"
|
||||
)
|
||||
|
||||
func TestCanonicalizeMarkdown_PreservesHardBreakDoubleSpaces(t *testing.T) {
|
||||
raw := "第一行 \n第二行 \n普通行\n"
|
||||
got := doccontent.CanonicalizeMarkdown(raw)
|
||||
if !strings.Contains(got, "第一行 \n") || !strings.Contains(got, "第二行 \n") {
|
||||
t.Fatalf("hard-break double spaces must be kept: %q", got)
|
||||
}
|
||||
if strings.Count(got, " \n") != 2 {
|
||||
t.Fatalf("want 2 hard-breaks, got %q", got)
|
||||
}
|
||||
if got != raw {
|
||||
t.Fatalf("already-canonical hard-break markdown must be identity: got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCanonicalizeMarkdown_OnlyNormalizesLF_NFC_TrailingNewline(t *testing.T) {
|
||||
in := "a\r\n b \t\n"
|
||||
out := doccontent.CanonicalizeMarkdown(in)
|
||||
if out != "a\n b \t\n" {
|
||||
t.Fatalf("must keep trailing spaces/tabs and only normalize CRLF: got %q", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyDestructiveCanonicalize_TrimsTrailingSpaces(t *testing.T) {
|
||||
raw := "第一行 \n第二行 \n"
|
||||
legacy := doccontent.LegacyDestructiveCanonicalizeMarkdown(raw)
|
||||
if strings.Contains(legacy, " \n") {
|
||||
t.Fatalf("legacy destructive canonicalizer must trim hard-breaks: %q", legacy)
|
||||
}
|
||||
if legacy != "第一行\n第二行\n" {
|
||||
t.Fatalf("got %q", legacy)
|
||||
}
|
||||
if doccontent.SHA256Hex(legacy) == doccontent.SHA256Hex(doccontent.CanonicalizeMarkdown(raw)) {
|
||||
t.Fatal("legacy and semantic-preserving hashes must differ for hard-break docs")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCanonicalizeMarkdown_Project143HardBreakGolden(t *testing.T) {
|
||||
var b strings.Builder
|
||||
for i := 0; i < 5; i++ {
|
||||
b.WriteString("条目")
|
||||
b.WriteByte('0' + byte(i))
|
||||
b.WriteString(" \n")
|
||||
}
|
||||
b.WriteString("结尾段落。\n")
|
||||
raw := b.String()
|
||||
canon := doccontent.CanonicalizeMarkdown(raw)
|
||||
if canon != raw {
|
||||
t.Fatalf("project143-shaped hard-breaks must survive: got %q", canon)
|
||||
}
|
||||
legacy := doccontent.LegacyDestructiveCanonicalizeMarkdown(raw)
|
||||
if len(raw)-len(legacy) != 10 {
|
||||
t.Fatalf("legacy must drop exactly 10 bytes (5×2 spaces), raw=%d legacy=%d", len(raw), len(legacy))
|
||||
}
|
||||
}
|
||||
|
||||
func TestDocrevisionWrappersShareCanonicalBytes(t *testing.T) {
|
||||
raw := "第一行 \r\n第二行 \n"
|
||||
if doccontent.CanonicalizeMarkdown(raw) == "" {
|
||||
t.Fatal("empty canon")
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"chat-agent-system/internal/docrevision"
|
||||
"chat-agent-system/internal/doccontent"
|
||||
"chat-agent-system/internal/fsutil"
|
||||
)
|
||||
|
||||
@@ -382,8 +382,8 @@ func (e *GovernanceEngine) SaveDocumentContent(ctx context.Context, req ContentS
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
canon := docrevision.CanonicalizeMarkdown(req.MarkdownBody)
|
||||
shaStr := docrevision.SHA256Hex(canon)
|
||||
canon := doccontent.CanonicalizeMarkdown(req.MarkdownBody)
|
||||
shaStr := doccontent.SHA256Hex(canon)
|
||||
doc.ContentSHA256 = shaStr
|
||||
doc.IsTemplateOnly = false
|
||||
doc.AuthorClaimed = true
|
||||
|
||||
@@ -1,49 +1,23 @@
|
||||
package docrevision
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"strings"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"golang.org/x/text/unicode/norm"
|
||||
"chat-agent-system/internal/doccontent"
|
||||
)
|
||||
|
||||
// CanonicalizeMarkdown applies LF, UTF-8 NFC, and a single trailing newline.
|
||||
// It must not trim trailing spaces or tabs: two trailing spaces are Markdown hard breaks.
|
||||
func CanonicalizeMarkdown(raw string) string {
|
||||
s := strings.ReplaceAll(raw, "\r\n", "\n")
|
||||
s = strings.ReplaceAll(s, "\r", "\n")
|
||||
s = norm.NFC.String(s)
|
||||
if s != "" && !strings.HasSuffix(s, "\n") {
|
||||
s += "\n"
|
||||
}
|
||||
return s
|
||||
return doccontent.CanonicalizeMarkdown(raw)
|
||||
}
|
||||
|
||||
// LegacyDestructiveCanonicalizeMarkdown reproduces the pre-fix canonicalizer that
|
||||
// stripped per-line trailing spaces/tabs. Used only to classify historical drift.
|
||||
func LegacyDestructiveCanonicalizeMarkdown(raw string) string {
|
||||
s := strings.ReplaceAll(raw, "\r\n", "\n")
|
||||
s = strings.ReplaceAll(s, "\r", "\n")
|
||||
s = norm.NFC.String(s)
|
||||
lines := strings.Split(s, "\n")
|
||||
for i, line := range lines {
|
||||
lines[i] = strings.TrimRightFunc(line, func(r rune) bool {
|
||||
return r == ' ' || r == '\t'
|
||||
})
|
||||
}
|
||||
s = strings.Join(lines, "\n")
|
||||
if s != "" && !strings.HasSuffix(s, "\n") {
|
||||
s += "\n"
|
||||
}
|
||||
return s
|
||||
return doccontent.LegacyDestructiveCanonicalizeMarkdown(raw)
|
||||
}
|
||||
|
||||
func SHA256Hex(content string) string {
|
||||
sum := sha256.Sum256([]byte(content))
|
||||
return hex.EncodeToString(sum[:])
|
||||
return doccontent.SHA256Hex(content)
|
||||
}
|
||||
|
||||
func ContentFingerprint(s string) string {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"chat-agent-system/internal/doccontent"
|
||||
"chat-agent-system/internal/docrevision"
|
||||
)
|
||||
|
||||
@@ -62,3 +63,16 @@ func TestCanonicalizeMarkdown_Project143HardBreakGolden(t *testing.T) {
|
||||
t.Fatalf("legacy must drop exactly 10 bytes (5×2 spaces), raw=%d legacy=%d", len(raw), len(legacy))
|
||||
}
|
||||
}
|
||||
|
||||
func TestCanonicalizeWrappersMatchDoccontent(t *testing.T) {
|
||||
raw := "第一行 \r\n第二行 \n"
|
||||
if docrevision.CanonicalizeMarkdown(raw) != doccontent.CanonicalizeMarkdown(raw) {
|
||||
t.Fatal("CanonicalizeMarkdown wrapper drifted")
|
||||
}
|
||||
if docrevision.LegacyDestructiveCanonicalizeMarkdown(raw) != doccontent.LegacyDestructiveCanonicalizeMarkdown(raw) {
|
||||
t.Fatal("legacy wrapper drifted")
|
||||
}
|
||||
if docrevision.SHA256Hex(raw) != doccontent.SHA256Hex(raw) {
|
||||
t.Fatal("SHA256Hex wrapper drifted")
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user