fix(intent): 收紧安全红线正则动词与桥接,消除正常撰写请求误拦
裸单字「改/换」加无界桥接会把「整改/角色」等正常撰写句误判为 SECURITY_FORBIDDEN。改为复合动词 + 短窗不跨标点桥接,受控名词须与写动词共现;补误报负例回归。
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
# 安全红线正则误拦 测试报告
|
||||
|
||||
## 一、 测试背景 (Background)
|
||||
|
||||
用户发送正常文档撰写请求「请撰写详尽合规手册:…每项含理由/证据/失败模式/整改/角色/抽样方法…」时,助手返回固定文案「⚠️ 合规权限拦截…」。根因是 `isSecurityViolation` 正则 2 用裸单字动词「改」加无界 `.*`,把「整改/角色」误配为越权写操作。
|
||||
|
||||
规范归档路径应为 `.doc/test-reports/`;本报告因并行任务 `server-image-field-closeout-r3-2026-09-25` 占用该 scope,先落盘于 `.doc/ops/evidence/` 与 agent store,待 scope 释放后再迁移。
|
||||
|
||||
## 二、 测试目标 (Test Objectives)
|
||||
|
||||
1. 原句与同类误报变体不得返回 `SECURITY_FORBIDDEN`。
|
||||
2. 既有真阳性用例仍须拦截(confidence=1.0)。
|
||||
3. 收紧机制不破坏 `internal/intent` / `internal/coordinator` 回归。
|
||||
|
||||
## 三、 测试方案与环境 (Test Plan & Environment)
|
||||
|
||||
- OS: Linux;Go 1.27.1;worktree `feature/security-redline-false-positive-2026-09-25` @ base `44161d2a`
|
||||
- 层级:Go/接口(验证契约)
|
||||
- 范围:`internal/intent`、`internal/coordinator`;全量 `go test -short ./...`(旁路失败单独记)
|
||||
|
||||
## 四、 预期结果 (Expected Results)
|
||||
|
||||
| 用例 | 预期 |
|
||||
|---|---|
|
||||
| 原句合规手册撰写 | 非 SECURITY_FORBIDDEN |
|
||||
| 整改计划…角色职责 | 非 SECURITY_FORBIDDEN |
|
||||
| 改一下文档标题…角色 | 非 SECURITY_FORBIDDEN |
|
||||
| 换一份清单…姓名字段 | 非 SECURITY_FORBIDDEN |
|
||||
| 起草系统配置说明文档 | 非 SECURITY_FORBIDDEN |
|
||||
| 知识库连接失败怎么排查 | 非 SECURITY_FORBIDDEN |
|
||||
| 既有 SecurityForbidden / Extended | 仍 SECURITY_FORBIDDEN |
|
||||
|
||||
## 五、 实际测试结果与判定 (Actual Results & Evaluation)
|
||||
|
||||
### 5.1 修复前(对照)
|
||||
|
||||
命令:`go test ./internal/intent/ -run TestIntentRecognition_SecurityFalsePositives_NotBlocked -count=1`
|
||||
|
||||
- **实际输出**:6/6 子用例 FAIL,均 `got SECURITY_FORBIDDEN`
|
||||
- **判定**:FAIL(预期:用于确认红)
|
||||
- **耗时**:约 4ms
|
||||
|
||||
### 5.2 修复后 — 误报负例
|
||||
|
||||
命令:`go test ./internal/intent/ ./internal/coordinator/ -count=1 -v -run 'Security'`
|
||||
|
||||
```
|
||||
--- PASS: TestIntentRecognition_SecurityFalsePositives_NotBlocked (0.00s)
|
||||
--- PASS: .../合规手册撰写含整改角色字段
|
||||
--- PASS: .../整改+角色远距共现
|
||||
--- PASS: .../改一下文档+角色跨句
|
||||
--- PASS: .../换一份清单+姓名字段
|
||||
--- PASS: .../系统配置说明文档撰写
|
||||
--- PASS: .../知识库连接故障排查询问
|
||||
--- PASS: TestIntentRecognition_SecurityForbidden
|
||||
--- PASS: TestIntentRecognition_SecurityForbidden_Extended (全部子用例)
|
||||
--- PASS: TestCoordinatorEngine_SecurityForbidden
|
||||
ok chat-agent-system/internal/intent
|
||||
ok chat-agent-system/internal/coordinator
|
||||
```
|
||||
|
||||
- **判定**:PASS
|
||||
- **耗时**:intent ~7ms;coordinator ~4ms
|
||||
|
||||
### 5.3 门禁
|
||||
|
||||
| 检查 | 判定 |
|
||||
|---|---|
|
||||
| `./scripts/arch-check.sh --mode block` | PASS |
|
||||
| `go vet ./...` | PASS |
|
||||
| `go test ./internal/intent/ ./internal/coordinator/ -count=1` | PASS |
|
||||
| `go test -short ./...` | 旁路 FAIL:`cmd/bizdev-cli` goose duplicate version 71(`00071_...maintenance-windows` 与 `00071_...context-conflict-adjudications`),与本次 intent 改动无关;其余包 ok |
|
||||
|
||||
## 六、 自动化测试套件汇总 (Test Suite Execution)
|
||||
|
||||
- 适用并实际运行:arch-check、go vet、intent/coordinator 全量、short 全仓(旁路失败已记录)
|
||||
- 新增用例:`TestIntentRecognition_SecurityFalsePositives_NotBlocked`(6 子例)
|
||||
- 真阳性零回退
|
||||
|
||||
## 七、 测试结论与后续优化建议 (Conclusion & Next Steps)
|
||||
|
||||
- **结论**:机制级收紧后,原句与同类变体不再误拦;真阳性仍拦截。授权范围内验收满足。
|
||||
- **后续**:① scope 释放后将本报告迁入 `.doc/test-reports/2026-09-25-security-redline-false-positive-test-report.md`;② 独立处理 goose 71 重复(非本任务);③ 可选:对「越权」裸字面量做同类负例评估(如「防越权培训」)。
|
||||
@@ -6,7 +6,9 @@ import (
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// isSecurityViolation 检查是否命中敏感安全与权限红线
|
||||
// isSecurityViolation 检查是否命中敏感安全与权限红线。
|
||||
// 中文无词边界:动词仅用复合形(改成/改为/换成/重置/设为/修改等),桥接限制在不跨子句标点的短窗;
|
||||
// 受控资产名词(系统配置/知识库连接等)须与写动词共现,避免正常撰写/排查误拦。
|
||||
func isSecurityViolation(input string) bool {
|
||||
lower := strings.ToLower(input)
|
||||
patterns := []string{
|
||||
@@ -16,11 +18,11 @@ func isSecurityViolation(input string) bool {
|
||||
"修改资质", "资质修改",
|
||||
"改成批准人", "改成审核人", "改成编制人",
|
||||
"工号改成", "修改工号", "工号修改",
|
||||
"修改设置", "修改配置", "系统设置", "系统配置",
|
||||
"修改网关", "网关配置", "网关密钥", "大模型网关", "模型网关",
|
||||
"修改设置", "修改配置", "修改系统设置", "修改系统配置",
|
||||
"修改网关", "修改网关配置", "修改网关密钥", "修改大模型网关", "修改模型网关",
|
||||
"修改密码", "重置密码", "提升权限", "越权",
|
||||
"修改向量库", "向量库连接", "向量库配置", "修改向量库连接",
|
||||
"修改知识库连接", "知识库连接", "知识库配置",
|
||||
"修改向量库", "修改向量库连接", "修改向量库配置",
|
||||
"修改知识库连接", "修改知识库配置",
|
||||
}
|
||||
for _, p := range patterns {
|
||||
if strings.Contains(lower, p) {
|
||||
@@ -28,10 +30,13 @@ func isSecurityViolation(input string) bool {
|
||||
}
|
||||
}
|
||||
|
||||
// 桥接窗:不跨子句标点(含斜杠),避免「整改/角色」类远距误配。
|
||||
const bridge = `[一-鿿A-Za-z0-9_、 ]{0,16}`
|
||||
secRegexes := []string{
|
||||
`把.*(?:身份|职位|角色|工号|权限|姓名|名字|部门|资质).*(?:修改|改|换|重置|设为|变为)`,
|
||||
`(?:修改|改|换|重置|设为).*(?:我的)?(?:身份|职位|角色|工号|权限|姓名|名字|部门|资质|网关|向量库|知识库连接|密钥|apikey|api_key)`,
|
||||
`.*(?:向量库连接|修改向量库|知识库连接|大模型网关密钥|网关密钥).*`,
|
||||
`把` + bridge + `(?:身份|职位|角色|工号|权限|姓名|名字|部门|资质)` + bridge + `(?:修改|改成|改为|换成|换为|重置|设为|变为)`,
|
||||
`把` + bridge + `(?:向量库连接|向量库|知识库连接|大模型网关|模型网关|网关密钥|网关|密钥|apikey|api_key)` + bridge + `(?:修改|改成|改为|换成|换为|重置|设为|变为)`,
|
||||
`(?:修改|改成|改为|换成|换为|更换|重置|设为)` + bridge + `(?:我的)?(?:身份|职位|角色|工号|权限|姓名|名字|部门|资质|网关|向量库|知识库连接|密钥|apikey|api_key)`,
|
||||
`(?:修改|改成|改为|换成|换为|更换|重置|设为)` + bridge + `(?:向量库连接|向量库|知识库连接|大模型网关密钥|网关密钥)`,
|
||||
}
|
||||
for _, r := range secRegexes {
|
||||
if matched, _ := regexp.MatchString(r, input); matched {
|
||||
|
||||
@@ -602,6 +602,36 @@ func TestIntentRecognition_FinalReject_MissingReason(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
func TestIntentRecognition_SecurityFalsePositives_NotBlocked(t *testing.T) {
|
||||
rec := intent.NewRecognizer()
|
||||
testInputs := []struct {
|
||||
input string
|
||||
desc string
|
||||
}{
|
||||
{"请撰写详尽合规手册:IEC 62304、ISO 13485、ISO 14971、IEC 62366 各写 20 个检查项,每项含理由/证据/失败模式/整改/角色/抽样方法,全文尽量超过 8000 字。", "合规手册撰写含整改角色字段"},
|
||||
{"整改计划中列明各角色职责", "整改+角色远距共现"},
|
||||
{"帮我改一下文档标题,再说明各审批角色", "改一下文档+角色跨句"},
|
||||
{"换一份清单,包含姓名字段模板", "换一份清单+姓名字段"},
|
||||
{"起草系统配置说明文档", "系统配置说明文档撰写"},
|
||||
{"知识库连接失败怎么排查", "知识库连接故障排查询问"},
|
||||
}
|
||||
for _, tc := range testInputs {
|
||||
t.Run(tc.desc, func(t *testing.T) {
|
||||
res, err := rec.Recognize(context.Background(), tc.input, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("recognize %q failed: %v", tc.input, err)
|
||||
}
|
||||
if res.TopIntent == nil {
|
||||
t.Fatalf("recognize %q: top intent is nil", tc.input)
|
||||
}
|
||||
if res.TopIntent.Intent == intent.IntentSecurityForbidden {
|
||||
t.Errorf("input %q: must NOT be SECURITY_FORBIDDEN (false positive); got %s reason=%s", tc.input, res.TopIntent.Intent, res.TopIntent.Reasoning)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntentRecognition_SecurityForbidden_Extended(t *testing.T) {
|
||||
rec := intent.NewRecognizer()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user