diff --git a/.task/archive/fix-svc-pipefail-early-exit-pipes.md b/.task/archive/fix-svc-pipefail-early-exit-pipes.md new file mode 100644 index 00000000..8088c7c5 --- /dev/null +++ b/.task/archive/fix-svc-pipefail-early-exit-pipes.md @@ -0,0 +1,106 @@ +# fix/svc-pipefail-early-exit-pipes 任务卡 + +任务主记录:PR #664 https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Dev/pulls/664;状态:已完成(合入前闭合);owner:weichen + +- 权威仓:ssh://git@git-ssh.u1680832.nyat.app:31411/weichen/Agent-System-Dev.git;目标分支:main;R0(initial_base_sha)= 7e19cf3e182b566bdc48a7e690e44ff11b4ab33c +- 工作区:/home/weichen/projects/Agent-System-Dev-wt-fix-svc-pipefail-early-exit-pipes(worktree);分支:fix/svc-pipefail-early-exit-pipes +- 分支类型:standalone;root_target:main;immediate_target:main;accepting_requirements:false +- 需求分流:verdict=NEW;selected_initiative=无;requirement_sha256=995ae1c10a5bed201669a98755fcd9db243864384184f224d037521c25d156fb;triage_receipt=/tmp/triage-pipefail-class-20261020.json;receipt_sha256=a136d676c51767e6fc53e36b96e6d518cd235ccc9ad4a77a14449ace86136d8f + +- 授权与交付终点:消除本仓受管 shell 脚本在 `set -o pipefail` 下的早退读取器管道隐患,并加守护测试防复发;按现行门禁完成 Review、PR、CI、合入、任务卡闭合与安全回收。不得改动任何命令的语义与退出码契约,不得改 Makefile/CI/子模块/规范文档,不得用 `|| true`、`set +o pipefail` 掩盖状态,不得新增 schema/开关/通用重构。 +- 交付模式:goal-first(目标、范围和交付终点一次确认;范围内连续推进) +- 实施执行:agent-per-unit(配置:AGENT-EXECUTION.md) +- 合入执行:agent(已授权交付范围内,Review 通过、冲突解决且当前 HEAD 的 CI 通过后合入) +- 流程依据:仓库根目录 policy/delivery.md(归档后仍按仓库根目录定位) + +## 目标与非目标 + +目标:`producer | grep -q PAT`、`... | head -N`、`awk ... exit` 这类「读取器先退出」的管道,在 `set -euo pipefail` 下会让写满 64KiB 管道缓冲的生产者收到 SIGPIPE,管道状态变成 141 而非读取器结论,`if` / `if !` / `&&` 的判定被静默翻转。本任务把本仓受管脚本(`scripts/svc.sh`、`scripts/task.sh`、`scripts/e2e.sh`、`scripts/lint-boundaries.sh`、`docker/verify-env.sh`、`tests/load/collaboration/run_pgbench.sh`)中的此类管道改为语义等价且读取器退出码不再经过管道形态,并新增 `tests/test_shell_pipe_hazard.py` 守护测试防止复发。受影响入口:`make task-finish`、`make precheck`、`make svc` 系列与环境校验。 + +非目标:不改变任何命令的行为、退出码与调用顺序;不修 `runtime/` 下的 ignored 陈旧克隆;不改守护测试的检测范围(中段读取器、`awk exit`、`| read`、前缀化 grep 等盲区按 Review 建议保留为非阻塞);不改 Makefile、CI、子模块 pin、共享规范子模块与文档;不新增 schema、运行时开关或通用化重构。 + +验收条件:(1) 受管脚本中不再有末级早退读取器管道,守护测试通过;(2) 变换前后行为等价(含命中/未命中/多行匹配/命令替换取值/退出码),并有对照证据;(3) `make test-governance` 与 `test_svc*.py` 全绿;(4) 独立 Review 无阻塞项。 + +## Agent 实施收据 + +路由与 fail-stop 规则见根目录 AGENT-EXECUTION.md 与 policy/delivery.md;每次尝试一行,原始日志留在源码外收据路径。 + +| 单元 ID | 预期范围 | 配置摘要 | 角色 | 候选顺序 | attempt | CLI/模型 | 终态 | transport | fallback | 源码外收据路径 | +|---|---|---|---|---|---|---|---|---|---|---| +| svc-pipefail-early-exit-pipes | 6 个受管脚本 + `tests/test_shell_pipe_hazard.py` | 本仓 AGENT-EXECUTION.md | implementation | cursor, antigravity, pi-glm, pi, codex | 1 | cursor / auto | success | success (`f87d63f`) | false | `/tmp/svc-pipefail-impl-r1-20261020.json` | +| svc-pipefail-independent-review | `.task/reviews/2026-10-20-svc-pipefail-independent-review.md` | 同上 | review | pi-glm, pi, antigravity, cursor, codex | 1 | pi-glm / zai-coding-cn/glm-5.3-flash | success(PASS,无阻塞项) | success (`2ef67de`) | false | `/tmp/svc-pipefail-review-r1-20261020.json` | +| svc-pipefail-residual-sites | `scripts/svc.sh` | 同上 | repair | cursor, antigravity, pi-glm, pi, codex | 2(r1 failed-unknown:cursor `auto` 模型不可用,零产物不导入) | cursor / auto | success | success (`5fadf85`) | false | `/tmp/svc-pipefail-repair-r2-20261020.json`(r1:`/tmp/svc-pipefail-repair-r1-20261020.json`) | + +## 同步记录 + +| 节点 | sync_base_sha | tested_head_sha | checked_at | +|---|---|---|---| +| 启动 | 7e19cf3e182b566bdc48a7e690e44ff11b4ab33c | 7e19cf3e182b566bdc48a7e690e44ff11b4ab33c | 2026-10-20T14:22:14Z | +| 交付前同步(最终) | 6a7f008ed491e813d7f43fb013cb9542d849ff85 | b9a0309aa4b517d95012b83086a4e02b0d02da85 | 2026-10-20T15:48:20Z | + +## 证据(Evidence) + + +| 时间(UTC) | 命令 | 被测HEAD | exit | 来源 | sha256(16) | +|---|---|---|---|---|---| + +## 结果与验证 + +- 结果:6 个受管脚本中共 41 处早退读取器管道改为语义等价形态——变量入管道改 here-string(`grep -q PAT <<<"$var"`)、命令生产者改进程替换(`grep -q PAT < <(producer)`,二进制 `grep -qa` 亦走进程替换以避免 NUL 截断)、`sed … | head -1` 改 `sed -n '/re/{s//x/p;q;}'`;另新增 `tests/test_shell_pipe_hazard.py`(词法扫描管道末级 + 可失败的合成自检)。未使用 `|| true` / `set +o pipefail` 掩盖状态。 +- 验证:PASS——行为对照 `/tmp/pipefail-demo-evidence-20261020.json`:旧式在 >64KiB 生产者下把命中分支错误地跳过,新式正确命中,小输入语义不变;dsn 提取新旧表达式在命中/未命中/值内含伪 dsn/文件缺失/多行匹配五种输入下逐一 MATCH(`/tmp/dsn-equiv-evidence-20261020.json`);6 个脚本 `bash -n` 全过;最终 HEAD 上守护测试 4 项 OK、`test_svc*.py` 213 项 OK(18 skipped)、`make test-governance` 518 项 OK(18 skipped)。真实故障参照:PR #657 修复前 Backend 组件索引 76386 字节、旧式管道 rc=141。 +- 影响与剩余项:不改变任何命令的行为与退出码调用方式;进程替换后生产者自身的退出码不再参与 `if` 判定(Review 逐处判定为修正旧判定翻转或中性等价)。非阻塞(按 Review 建议保留):守护测试仅覆盖末级读取器,中段 `| head`、`awk … exit`、`| read`、前缀化 `grep` 为已知盲区;`SCRIPTS` 元组硬编码,新增受管脚本需手动登记。 + +## 合入与清理 + +- Review 记录:PASS(独立会话 pi-glm / zai-coding-cn/glm-5.3-flash;base 7e19cf3e182b566bdc48a7e690e44ff11b4ab33c,head 2ef67de19e8bceb0eaa1b4ab1dba4667fc108212;报告 `.task/reviews/2026-10-20-svc-pipefail-independent-review.md`;无阻塞项,4 项非阻塞建议;其点名的 svc.sh 三处残留已由 repair 单元 `5fadf858` 收干净,重放依据:host 行为对照 + dsn 等价对照 + `test_svc*.py`/治理套件) +- 冲突记录:NOT_RUN(一次交付前同步 `make task-sync`,无冲突;业务差异始终限于 6 个脚本、1 个新增测试与任务记录) +- CI 记录:NOT_RUN(当前 HEAD 完整 SHA、全部必需检查、运行链接、执行身份及 GATE 结果) +- 合入记录:NOT_RUN(交付授权范围、PR、被核验 HEAD、合入 SHA、目标分支及内容核验) +- 清理记录:NOT_RUN(授权依据、写者退出确认、worktree/分支/任务卡处置) + + +```json +{ + "summary_schema": 1, + "source": "SELF", + "timestamp": "2026-10-20T14:21:06.568629+00:00", + "command": "make precheck", + "exit_code": 0, + "cwd": "/home/weichen/projects/Agent-System-Dev", + "before_head": "7e19cf3e182b566bdc48a7e690e44ff11b4ab33c", + "after_head": "7e19cf3e182b566bdc48a7e690e44ff11b4ab33c", + "branch": "main", + "target_branch": "main", + "target_head": "7e19cf3e182b566bdc48a7e690e44ff11b4ab33c", + "root_target": "main", + "immediate_target": "main", + "before_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "after_tracked_diff_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "output_tail": [ + "[repo] branch=main head=7e19cf3e18", + "[remote] origin/main=7e19cf3e18 ahead=0 behind=0", + "[tree] clean submodules_drift=0 uninit=0", + "[worktrees] linked=4", + "[recycle] merged un-recycled branches: feature/fast-slow-o2-a01-checkpoint-design", + "[initiatives] local cards (not authoritative): none", + "[hooks] core.hooksPath=scripts/hooks(pre-commit/post-merge 实际接线)", + "[facts] policy/facts.md 6 条事实均在有效期内", + "[required-reading] core/baseline.md, workflows/delivery.md §1, policy/delivery.md", + "[R0] 7e19cf3e18(完整:7e19cf3e182b566bdc48a7e690e44ff11b4ab33c)", + "[worktree] before-sync: recursive component checks passed", + "[verdict] READY — 保留本次 precheck receipt 后建任务工作区:PRECHECK_RECEIPT=/tmp/task-precheck.json make task-start NAME=feature/your-branch" + ], + "output_sha256": "cc58d05246cfa8a11a3e80ab42f15f9909ea2177ddcf2cb8b01700745fdd9921", + "receipt_sha256": "24204950509fc31a6cd58c01eff987d4841dd861e3fc5018df52297722f6b6f9", + "receipt_path": "/home/weichen/.cache/agent-system-dev/evidence/24204950509fc31a6cd58c01eff987d4841dd861e3fc5018df52297722f6b6f9.json" +} +``` +| 同步(记录前对象) | 6a7f008ed491e813d7f43fb013cb9542d849ff85 | ba4917e62bec2a4e149391f4f7535e6e937e4559 | 2026-10-20T15:48:20Z | + +## 闭合记录(合入前折叠) + +- 本卡在 PR #664(https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Dev/pulls/664)合入前闭合并归档,随该 PR 一同合入。 +- 该 PR 的合并提交即本任务的落地记录。 +- 合入后无需回填,也无需单独的闭合 PR。 +| 同步(记录前对象) | 1850d03563875fd848664ab9b348f6858d0fa590 | 3b7123bc5a56bbfefc007167875c044ce3dfe69c | 2026-10-20T16:37:20Z | +| 同步(记录前对象) | 36466b99f2fbc8e389167e01e722ece33965df2b | 657dfdf015f77a08cc9817d1d5acc3b272430015 | 2026-10-20T17:05:29Z | diff --git a/.task/reviews/2026-10-20-svc-pipefail-independent-review.md b/.task/reviews/2026-10-20-svc-pipefail-independent-review.md new file mode 100644 index 00000000..c06e7616 --- /dev/null +++ b/.task/reviews/2026-10-20-svc-pipefail-independent-review.md @@ -0,0 +1,116 @@ +# 独立代码审查:fix/svc-pipefail-early-exit-pipes(svc 管道早退读取器治理) + +- 审查对象:`7e19cf3e182b566bdc48a7e690e44ff11b4ab33c..f87d63ffd8ece163bd7054c80cb51ccd6159f994` +- 涉及文件:scripts/svc.sh、scripts/task.sh、scripts/e2e.sh、scripts/lint-boundaries.sh、docker/verify-env.sh、tests/load/collaboration/run_pgbench.sh、新增 tests/test_shell_pipe_hazard.py(另含任务卡 `.task/fix-svc-pipefail-early-exit-pipes.md`,属 `.task/**` 豁免范围) +- 审查者身份:全新会话独立审查,未参与实现;本会话仅写入本报告,其余只读 +- 参照先例:Dev PR #657(5562015,scripts/task.sh 回收路径 `git ls-files --stage | grep -q '^160000 '` → awk 全量消费,commit 8557ee9/daf0288..8557ee9) + +## 结论:PASS(无阻塞项;4 项非阻塞建议) + +判定标准逐项核对:无语义回归、无判定翻转、守护测试非形同虚设(真实可失败、对已修复形态全部拦截)、无二进制误判。 + +--- + +## 一、逐处语义等价核对(重点 1) + +六个脚本均声明 `set -euo pipefail`(svc.sh:134、task.sh:4、e2e.sh:24、lint-boundaries.sh:4、verify-env.sh:2、run_pgbench.sh:10),隐患前提成立。共 34 处变更,按变换形态归类核对: + +### A. 变量 → here-string(`grep … <<<"$var"`,共 15 处) + +sites:verify-env.sh:34、48;lint-boundaries.sh:76;task.sh:23、26、29、214、219、315、316、1070;svc.sh:1067、1703、3577、4923、4955、5040、5100。 + +原形均为 `echo/printf "$var" | grep …` 或 `printf '%s' "$var" | head -c N`。实测(/tmp/pipefail-review):here-string 会在串尾补 `\n`,`echo`/`printf '%s\n'` 同样补 `\n`;`printf '%s'` 不补但 grep 按行匹配对尾随换行不敏感,且 task.sh:1070 经命令替换取值时尾随 `\n` 被剥除——实测 `$(head -c 120 <<<"$short")` 与 `$(printf '%s' "$short" | head -c 120)` 输出逐字节一致(含 <120 字节与 >120 字节两例)。命中/未命中/多行/退出码全部一致。此处原"生产者"是 shell 内建 echo/printf,不存在可丢失的生产者退出码。**等价,成立。** + +### B. 文件输入 sed:`sed -n 's/^K=//p' f | head -1` → `sed -n '/^K=/{s///p;q;}'`(共 17 处) + +sites:svc.sh:900、906、1067、2432、3320、5449、5506-5509、5676、5939、5977、6086;task.sh:643、644、647、648、649;run_pgbench.sh:186。 + +实测(GNU sed 4.9,/tmp/pipefail-review/kv.txt 多值键/无键/空值用例):多行匹配取首行、未命中输出空、`k=`(空值)输出空——新旧输出一致;`s///` 空正则复用地址正则(POSIX 语义),替换结果与原 `s/^K=//p` 一致。退出码:未命中旧管道(pipefail)=sed 的 0=新式 0;文件不可读旧=2(pipefail 传播 sed)=新式 2。`sed q` 只在匹配后退出,输入是文件而非管道,无 SIGPIPE 面。**等价,成立。** + +### C. 全量消费替代早退:`| head` → `| sed -n 1p` / `sed -n '1,12p'`(3 处) + +- verify-env.sh:41:`find … | sed -n 1p`——sed 无 q 会读尽输入,输出首行,实测与 `head -n 1` 输出一致(含空输入)。 +- run_pgbench.sh:180:`ps … | sed -n '1,12p'`——同上,实测输出一致。 + +### D. 进程替换 `< <(producer)`(6 处) + +svc.sh:1476(find -print -quit)、3943(strings)、4415(pg_psql)、4949(serve -h)、5087(pg_run)、5156/5169(head -c 15)。退出码语义变化见第二节。 + +### E. e2e.sh:211:`grep -o '装载(.*)' f | head -1 || true` → `grep -m1 -o … || true` + +实测:`.*` 贪婪使每行至多产生一个匹配,故 `head -1`(第一个匹配)≡ `-m1 -o`(首个匹配行的全部匹配)——单行/单匹配/整行贪婪匹配三例输出一致。文件输入无管道 SIGPIPE;`|| true` 为原有写法(旧式同样以 `|| true` 收尾),非新增掩盖。**等价,成立。** + +--- + +## 二、退出码语义的真实变化逐处判定(重点 2) + +进程替换后生产者退出码不再参与判定。前提复现:`bash -c 'set -euo pipefail; seq 1 200000 | grep -q 1'` 实测判为 MISS、裸管道状态 141(隐患为真)。「生产者失败但输出恰好命中」实测确实由旧=失败翻转为新=成功(`f(){ echo X; return 3; }`:旧管道判 FAIL(3),新 procsub 判 HIT)。逐处判定: + +| site | 原行为 | 新行为 | 判定 | +|---|---|---|---| +| svc.sh:4949 `serve -h 2>&1 \| grep -Eq config` | `-h` 以非零退出(真实 CLI 常见)时 pipefail 使判定翻为「不识别」→ 假警报 | 仅看 help 文本是否含 `--config`(实测 2×2 矩阵:exit0+命中=静默、exit0+未命中=告警、exit2+命中=旧假警报/新正确静默、exit2+未命中=告警) | **修正**。判定目标本就是「help 文本内容」,忽略 `-h` 退出码语义正确;未发现有害翻转场景 | +| svc.sh:3943 `strings \| grep -q AGENT_SYSTEM_DSN` | 大二进制(串靠前、残余 >64KiB)时 strings 吃 SIGPIPE→141→`&& return 0` 失败→假「不支持」(实测复现:合成用例旧判 NOT supported) | 命中即真支持;strings 半途 I/O 失败且部分输出恰含该串才会假「支持」——部分输出含串即证明产物确实内嵌该串 | **修正**(PR #657 同类缺陷的真实翻版) | +| svc.sh:4415/5087 `pg_psql/pg_run "SELECT 1 …" tA \| grep -q 1` | psql 连接失败(空输出)→ 判「库不存在」→ 转 CREATE → 二次失败报错(误导);「失败但 stdout 恰有 1」对单值 SELECT 不成立 | 失败(空输出)同样判「不存在」转 CREATE,下游可见失败,与旧一致 | **中性偏修正**;无真实「失败但命中」场景 | +| svc.sh:1476 `find … -print -quit \| grep -q .` | 单行输出,无缓冲面;目录不存在时新旧均判空(实测均 status 1) | 同 | **中性(防御性)** | +| svc.sh:5156/5169 `head -c 15 \| grep -qa` | head 只写 15 字节(<64KiB),无 SIGPIPE 面;文件可读性由前置 `[ -f ]` 保证 | 同 | **中性** | +| verify-env.sh:41、run_pgbench.sh:180 | find/ps 输出 >64KiB(大型 playwright 安装、>~700 进程)或 ps 偶发 exit 1 时,旧式在 `set -euo pipefail` 下直接中止(141 传播)| sed 读尽输入,状态恒 0,失败不再中止诊断/汇总块 | **修正**;ps 失败被掩盖属诊断块可接受(已在旧代码 flaky 风险与本次修复间取舍,记录在案) | +| 其余 here-string/sed 文件输入各处 | — | 生产者为内建或 sed 自身,无生产者退出码可丢失 | **等价** | + +结论:所有真实变化方向均为「修正旧判定翻转」或「中性等价」,未发现「生产者失败但输出命中被新写法误判成功且旧写法正确拒绝」的成立场景。 + +## 三、二进制与边界(重点 3) + +svc.sh:5156/5169 `grep -qa '^SQLite format 3$' < <(head -c 15 …)`: + +- 进程替换是 fd 直通,不经过 bash 变量,**无 NUL 截断面**;`-qa` 保留,按字节读取(实测 `head -c 15 real.db | od -c` = `S Q L i t e f o r m a t 3`,15 字节,NUL 是第 16 字节本就不在窗口内)。 +- 无尾随换行的"末行"命中语义新旧一致:实测真 SQLite 头两式均检出、非 SQLite 文件两式均不检出。 +- `head -c 15` 产出 ≤15 字节,新旧均无 SIGPIPE 面。**无回归。** + +## 四、守护测试质量(重点 4) + +tests/test_shell_pipe_hazard.py(311 行):词法扫描器 + 4 个自测。 + +- **真实可失败**:`PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.test_shell_pipe_hazard -v` → 4 tests OK;检测器对合成隐患 `seq 1 200000 | grep -q 1`、`if ! … | grep -q`、`sed|head` 命令替换、`-m1`、续行管道、`sed q`、`grep -qiE || exit` 七种形态全部命中(test_detector_flags_* 断言存在,检测器失效即测试红)。 +- **注释/heredoc/case/引号覆盖**:实测探针 14 项全过——注释行、heredoc 正文、case 模式 `|`、双引号内管道文本均不误报;`<<<`、`< <()`、`sed -n 1p` 安全形态不误报;`grep -q/-m`、`head`、`sed q` 尾级全部命中。 +- **已知盲区(实测确认,当前树未触发恶性形态)**: + 1. 只查「管道末级」:中段早退读取器不查——svc.sh:3804/5482 `grep -oE '"dsn"…' | head -1 | sed -E …`(两处)原样存留且守护通过; + 2. 不建模 awk/read:svc.sh:1058 `ss -ltnp | awk '{print; exit}'` 为尾级早退读取器,未命中; + 3. 不识别前缀化 grep:`producer | LC_ALL=C grep -q x`、`command grep -q` 可绕过(当前六脚本无此写法,实测确认)。 +- **311 行是否失衡**:判定为**可接受、不失衡**。svc.sh 含 heredoc、case 模式、嵌套 `$()`、续行管道与多种引用,逐行正则式简化实现必然双向误报/漏报(其漏报恰是本缺陷类);扫描器自带合成自测可失败,复杂度换来的是零误报地通过当前树。失衡风险的代价是上述盲区,见非阻塞建议。 + +## 五、范围与遗漏(重点 5) + +- 变更范围:`git diff --name-only 7e19cf3..f87d63f` 恰为 7 个目标文件 + 任务卡(`.task/**` 豁免),无越界改动。 +- 残留早退读取器(runtime/ 已确认为 ignored 陈旧克隆:`git check-ignore -v runtime/` 命中 .gitignore:9,且其下 0 个 tracked 文件,未据此判定): + - 受管脚本内(先于本分支存在,git show 7e19cf3 证实):svc.sh:3804、5482(中段 `head -1`)、svc.sh:1058(尾级 `awk … exit`)。现实触发需 >64KiB 的 dsn 匹配行/海量监听套接字,且失败形态是 `$(…)` 赋值非零→`set -e` 中止(响亮失败),非静默翻转——**不阻塞**。 + - 受管清单外(本单元范围外,仅记录):e2e-duties.sh:87(`ps -p | grep -Fq`,单行输出,bounded)、:128(`curl | head -c 160`);tests/integration/rdm-increment/acceptance_scenarios.sh(`set -u` 无 pipefail,echo 内建生产者);tests/ops/lib.sh:82、146(无 pipefail)。 +- 掩盖写法:六脚本无 `set +o pipefail`;本分支新增的 `|| true` 仅 e2e.sh:211 一处且为原有收尾写法的保留,非掩盖新增。PR #657 式「全量消费」替代(awk)未用于本批,改用 here-string/procsub/sed-q,方向一致且更彻底。 + +## 非阻塞建议(不阻塞合入) + +1. **守护测试盲区**:把检测范围扩到中段读取器(`| head`、`| sed q` 出现在任意非首段即报),并建模 `awk '…exit'`、`| read`、前缀化 grep(`LC_ALL=C grep`);或至少在 docstring 声明「仅末级」契约的同时为 svc.sh:3804/5482/1058 立即整改。当前两处 dsn 残留建议改 `sed -n '/"dsn"/{s//…/p;q;}'` 式文件直读。 +2. **SCRIPTS 元组硬编码**:新增受管脚本不会被自动纳入扫描,建议 glob(如 scripts/*.sh + 白名单豁免)。 +3. **守护测试纯词法**:可加一条行为级自检(`bash -c 'set -o pipefail; seq 1 200000 | grep -q 1; test $? -eq 141'`)把 141 前提钉进测试,防 bash 语义漂移。 +4. **run_pgbench/verify-env 的 sed 替代**掩盖了 ps/find 的真实失败状态:诊断块可接受,但若未来该块内命令失败需要可见,应显式记录状态。 + +## 已运行的验证命令与结果 + +| 命令(只读) | 结果 | +|---|---| +| `git log --oneline 7e19cf3..f87d63f` / `git diff --stat` 同范围 | 2 commits;8 文件(7 目标 + 任务卡),+448/−47 | +| `git status --porcelain`(开工时) | 干净;HEAD=f87d63ff(detached) | +| 六脚本 `set -` 声明检查 | 全部 `set -euo pipefail`(svc.sh:134 等) | +| /tmp/pipefail-review 语义实验(sed 新旧式多值/未命中/空值/不可读;here-string vs printf 含 `head -c 120`;SIGPIPE 复现 141;procsub 退出码翻转矩阵;serve -h 2×2 矩阵;`grep -m1 -o` vs `head -1` 贪婪匹配;SQLite 头 15 字节两式对照;find -quit 有/无目录) | 全部如上文引用,行为与判定一致 | +| `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.test_shell_pipe_hazard -v` | 4 tests OK(0.083s) | +| `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest tests.test_svc tests.test_svc_backup` | 34 tests OK(34.3s) | +| 检测器 14 项探针(含中段 head、awk exit、env 前缀 grep、`read`、注释/heredoc/case/引号) | 14/14 符合设计;盲区 3 类确认(见第四节) | +| 22 个 tracked `*.sh` 独立残留扫描(grep 形态匹配) | 残留清单见第五节 | +| `bash -n` 六脚本 | 全部 OK | +| `git check-ignore -v runtime/`;`git ls-files runtime/ \| wc -l` | ignored(.gitignore:9);0 个 tracked 文件 | + +## 未验证项(如实标注) + +- 仅在 GNU sed 4.9 / 当前 bash 验证;macOS BSD sed 的 `s///` 复用地址正则与 `{…q;}` 行为**未验证**。 +- `strings` 大二进制路径为合成复现,未在真实制品上运行(NOT_RUN)。 +- 完整测试套件与 CI 未运行(本单元只读约束内的过滤子集已全绿)。 +- e2e-duties.sh、acceptance_scenarios.sh、tests/ops/lib.sh 的残留未整改(范围外,仅登记)。 diff --git a/docker/verify-env.sh b/docker/verify-env.sh index 483a67fe..69e6fa54 100644 --- a/docker/verify-env.sh +++ b/docker/verify-env.sh @@ -31,20 +31,20 @@ fi id dev >/dev/null 2>&1 || fail_expect "用户 dev 存在" "用户 dev 不存在" sudo_list="$(sudo -l -U dev 2>&1 || true)" -echo "${sudo_list}" | grep -q "NOPASSWD" || fail_expect "sudo -l -U dev 含 NOPASSWD" "${sudo_list}" +grep -q "NOPASSWD" <<<"${sudo_list}" || fail_expect "sudo -l -U dev 含 NOPASSWD" "${sudo_list}" if [ "${PLAYWRIGHT_BROWSERS_PATH:-}" != "/opt/ms-playwright" ]; then fail_expect "PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright" "${PLAYWRIGHT_BROWSERS_PATH:-<未设置>}" fi chromium_path="$( - find /opt/ms-playwright -type f \( -name headless_shell -o -name chrome \) 2>/dev/null | head -n 1 + find /opt/ms-playwright -type f \( -name headless_shell -o -name chrome \) 2>/dev/null | sed -n 1p )" [ -n "${chromium_path}" ] || fail_expect "在 /opt/ms-playwright 下存在 headless_shell 或 chrome" "未找到可执行文件" [ -x "${chromium_path}" ] || fail_expect "chromium 可执行文件具有执行权限" "${chromium_path} 不可执行" chromium_dom="$("${chromium_path}" --headless --no-sandbox --disable-gpu --dump-dom about:blank 2>&1 || true)" -echo "${chromium_dom}" | grep -q ""$OUT/structure-cockpit.log" 2>&1 || { tail -5 "$OUT/structure-cockpit.log"; die "cockpit structure load failed"; } -grep -o '装载(.*)' "$OUT/structure.log" | head -1 || true +grep -m1 -o '装载(.*)' "$OUT/structure.log" || true # 阶段③:重启服务,切片入库挂载到已装载骨架(API)。 start_serve diff --git a/scripts/lint-boundaries.sh b/scripts/lint-boundaries.sh index 8009adc0..88177df3 100755 --- a/scripts/lint-boundaries.sh +++ b/scripts/lint-boundaries.sh @@ -73,7 +73,7 @@ for dir in deps/agent-intent deps/agent-native deps/agent-squad; do deps_out=$(cd "$root/$dir" && GOWORK=off go list -deps ./...) || fail=1 for sib in "${siblings[@]}"; do [ "$sib" = "$mod" ] && continue - if printf '%s\n' "$deps_out" | grep -E -q "^(${sib}|${sib}/)"; then + if grep -E -q "^(${sib}|${sib}/)" <<<"$deps_out"; then echo "boundary: $mod core depends on $sib" >&2 fail=1 fi diff --git a/scripts/svc.sh b/scripts/svc.sh index 75a1d3dc..7741d1b1 100755 --- a/scripts/svc.sh +++ b/scripts/svc.sh @@ -897,13 +897,13 @@ derive_port() { # svc_get KEY:读当前服务 svc.env 的键值(无则空)。 svc_get() { [ -f "$ENV_DIR/svc.env" ] || return 0 - sed -n "s/^$1=//p" "$ENV_DIR/svc.env" | head -1 + sed -n "/^$1=/{s///p;q;}" "$ENV_DIR/svc.env" } # svc_kind NAME:服务类型(svc.env kind=,无登记或无 kind 键 → agent,兼容旧登记)。 svc_kind() { local k="" - [ -f "$RUNTIME/$1/svc.env" ] && k="$(sed -n 's/^kind=//p' "$RUNTIME/$1/svc.env" | head -1)" + [ -f "$RUNTIME/$1/svc.env" ] && k="$(sed -n '/^kind=/{s///p;q;}' "$RUNTIME/$1/svc.env")" [ -n "$k" ] && printf '%s' "$k" || printf agent } @@ -1055,7 +1055,7 @@ proc_stop() { port_owner() { local port="${ENV_ADDR##*:}" if command -v ss >/dev/null 2>&1; then - ss -ltnp 2>/dev/null | awk -v p=":$port" '$4 ~ p"$" {print $0; exit}' + awk -v p=":$port" '$4 ~ p"$" {print $0; exit}' < <(ss -ltnp 2>/dev/null) fi } @@ -1064,7 +1064,7 @@ port_owner_pid() { local line pid line="$(port_owner)" [ -n "$line" ] || return 1 - pid="$(printf '%s' "$line" | sed -n 's/.*pid=\([0-9][0-9]*\).*/\1/p' | head -1)" + pid="$(sed -n '/.*pid=\([0-9][0-9]*\).*/{s//\1/p;q;}' <<<"$line")" [ -n "$pid" ] || return 1 printf '%s' "$pid" } @@ -1473,7 +1473,7 @@ PY iid="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["instance_id"])' "$ENV_DIR/manifest.json")" [ "$CLI_CONFIRM" = "$iid" ] || { c_err "[$ENV_NAME] --migrate-schema 需要 --confirm $iid"; return 1; } backup="$(agent_backup_root)" - if ! find "$backup" -mindepth 1 -maxdepth 1 -type d -print -quit | grep -q .; then + if ! grep -q . < <(find "$backup" -mindepth 1 -maxdepth 1 -type d -print -quit); then c_err "[$ENV_NAME] forward-only 迁移必须先 backup" return 1 fi @@ -1700,7 +1700,7 @@ adjust_args_for_bin() { case "$w" in --auth|--config) flag="${w#--}" - if printf '%s\n' "$out" | grep -Eq -- "(^|[[:space:]])-{1,2}${flag}([[:space:]]|=|$)"; then + if grep -Eq -- "(^|[[:space:]])-{1,2}${flag}([[:space:]]|=|$)" <<<"$out"; then filtered="$filtered $w" else drop_next=1 @@ -2429,7 +2429,7 @@ materialize_release() { fi if [ -f "$mf" ]; then local want have - want="$(sed -n 's/.*"agent-system": *"\([0-9a-f]\{64\}\)".*/\1/p' "$mf" | head -1)" + want="$(sed -n '/.*"agent-system": *"\([0-9a-f]\{64\}\)".*/{s//\1/p;q;}' "$mf")" have="$(sha256sum "$bin" | cut -d' ' -f1)" if [ -n "$want" ] && [ "$want" != "$have" ]; then c_err "[$ENV_NAME] sha256 校验失败:制品与 manifest 不符($bin)——拒绝部署" @@ -3317,7 +3317,7 @@ agent_do_deploy() { fi local mf="$REL_ROOT/versions/$REL_VERSION/manifest.json" want have if [ -f "$mf" ]; then - want="$(sed -n 's/.*"agent-system": *"\([0-9a-f]\{64\}\)".*/\1/p' "$mf" | head -1)" + want="$(sed -n '/.*"agent-system": *"\([0-9a-f]\{64\}\)".*/{s//\1/p;q;}' "$mf")" have="$(sha256sum "$relbin" | cut -d' ' -f1)" if [ -n "$want" ] && [ "$want" != "$have" ]; then c_err "[$ENV_NAME] sha256 校验失败:制品与 manifest 不符($relbin)——拒绝部署" @@ -3574,7 +3574,7 @@ for u in users: "$ENV_BIN" apikey revoke ${seed_dbf[@]+"${seed_dbf[@]}"} --owner "$seed_owner" --name svc-seed >/dev/null 2>&1 \ || c_warn "[$ENV_NAME] 临时播种密钥回收失败:请手动吊销 owner=$seed_owner name=svc-seed" fi - if echo "$resp" | grep -q '"ok": *true'; then + if grep -q '"ok": *true' <<<"$resp"; then local did="" [ -n "${CLI_MBASE:-${SVC_MODEL_BASE:-}}${CLI_MKEY:-${SVC_MODEL_KEY:-}}${CLI_MNAME:-${SVC_MODEL_NAME:-}}" ] && did="主模型" [ -n "${CLI_LBASE:-${SVC_LIGHT_BASE:-}}${CLI_LKEY:-${SVC_LIGHT_KEY:-}}${CLI_LNAME:-${SVC_LIGHT_NAME:-}}" ] && did="${did:+$did、}轻量模型" @@ -3801,7 +3801,7 @@ resolve_db_config() { # conffile 的 dsn 键(二进制自行读文件,连接串不经环境重复传递)。 local cf_dsn="" if [ -n "$p_conf" ] && [ -f "$p_conf" ]; then - cf_dsn="$(grep -oE '"dsn"[[:space:]]*:[[:space:]]*"[^"]*"' "$p_conf" 2>/dev/null | head -1 | sed -E 's/.*"([^"]*)"$/\1/')" + cf_dsn="$(sed -n '/"dsn"[[:space:]]*:[[:space:]]*"[^"]*"/{s//\n&/;s/.*\n"dsn"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p;q;}' "$p_conf" 2>/dev/null)" fi # mock 档与数据库配置共存 = 冲突,拒绝(行为要明确,不隐式降级)。 @@ -3940,7 +3940,7 @@ bin_env_dsn_supported() { # 「支持」(回归教训:test_svc 的 legacy 桩即脚本形态)。 [ "$(head -c 4 "$1" 2>/dev/null)" = $'\x7fELF' ] || return 1 command -v strings >/dev/null 2>&1 || return 1 - strings "$1" 2>/dev/null | grep -q 'AGENT_SYSTEM_DSN' && return 0 + grep -q 'AGENT_SYSTEM_DSN' < <(strings "$1" 2>/dev/null) && return 0 return 1 } @@ -4412,7 +4412,7 @@ pg_do_start() { # 首次启动:建默认库(已存在则跳过)。 local dbname; dbname="$(svc_get dbname)" if [ -n "$dbname" ]; then - if ! pg_psql "SELECT 1 FROM pg_database WHERE datname='$dbname'" tA | grep -q 1; then + if ! grep -q 1 < <(pg_psql "SELECT 1 FROM pg_database WHERE datname='$dbname'" tA); then if pg_psql "CREATE DATABASE \"$dbname\""; then c_ok "[$ENV_NAME] 已建库:$dbname" else @@ -4920,7 +4920,7 @@ EOF [ -n "$target" ] && { c_err "--clear 不接受文件参数"; return 2; } local exec_old new_exec exec_old="$(svc_get exec)" - if printf '%s' "$exec_old" | grep -q -- '--config '; then + if grep -q -- '--config ' <<<"$exec_old"; then new_exec="$(printf '%s' "$exec_old" | sed -E 's/(^| )--config [^ ]+//; s/ +/ /g; s/^ //; s/ $//')" svc_set_key exec "$new_exec" fi @@ -4946,13 +4946,13 @@ EOF c_err "(键与取值以 config explain 为准;先修正文件再切换)" return 2 fi - if ! "$ENV_BIN" serve -h 2>&1 | grep -Eq -- "(^|[[:space:]])-{1,2}config([[:space:]]|=|$)"; then + if ! grep -Eq -- "(^|[[:space:]])-{1,2}config([[:space:]]|=|$)" < <("$ENV_BIN" serve -h 2>&1); then c_warn "[$ENV_NAME] 当前产物不识别 --config(旧版本二进制)——绑定已登记,但重启将忽略该参数" fi fi local exec_old new_exec exec_old="$(svc_get exec)" - if printf '%s' "$exec_old" | grep -q -- '--config '; then + if grep -q -- '--config ' <<<"$exec_old"; then new_exec="$(printf '%s' "$exec_old" | sed -E "s|--config [^ ]+|--config $abs|")" else new_exec="${exec_old:+$exec_old }--config $abs" @@ -5037,7 +5037,7 @@ EOF [ -n "$target" ] && { c_err "--clear 不接受服务名参数"; return 2; } local exec_old new_exec exec_old="$(svc_get exec)" - if printf '%s' "$exec_old" | grep -q -- '--dsn '; then + if grep -q -- '--dsn ' <<<"$exec_old"; then new_exec="$(printf '%s' "$exec_old" | sed -E 's/(^| )--dsn [^ ]+//; s/ +/ /g; s/^ //; s/ $//')" svc_set_key exec "$new_exec" fi @@ -5084,7 +5084,7 @@ EOF # 实例运行中:确保目标库存在(缺则建,幂等)。 st="$(svc_pid_state "$target")" if [ "${st%% *}" = alive ]; then - if ! pg_run "$thost" "$tport" "$tuser" "$tpass" postgres "SELECT 1 FROM pg_database WHERE datname='$tdb'" tA | grep -q 1; then + if ! grep -q 1 < <(pg_run "$thost" "$tport" "$tuser" "$tpass" postgres "SELECT 1 FROM pg_database WHERE datname='$tdb'" tA); then if pg_run "$thost" "$tport" "$tuser" "$tpass" postgres "CREATE DATABASE \"$tdb\"" >/dev/null; then c_ok "[$target] 已建库:$tdb" else @@ -5097,7 +5097,7 @@ EOF dsn="postgresql://$tuser:$tpass@$thost:$tport/$tdb" # 写持久化配置(托管绑定形态):dbref= + db=pg;清除直接 dsn=;剥离 exec 残留 --dsn。 # DSN 不落 exec=——启动时由实例登记重建(实例端口/密码变更自动跟随,密码不重复保存)。 - if printf '%s' "$exec_now" | grep -q -- '--dsn '; then + if grep -q -- '--dsn ' <<<"$exec_now"; then local new_exec new_exec="$(printf '%s' "$exec_now" | sed -E 's/(^| )--dsn [^ ]+//; s/ +/ /g; s/^ //; s/ $//')" svc_set_key exec "$new_exec" @@ -5153,7 +5153,7 @@ agent_backup_snapshot() { # $1=dest:svc.env + history.jsonl + data/(*.db 一 [ -f "$ENV_DATA/$rel" ] || continue case "$rel" in *"'"*|*'"'*) - if head -c 15 "$ENV_DATA/$rel" | grep -qa '^SQLite format 3$'; then + if grep -qa '^SQLite format 3$' < <(head -c 15 "$ENV_DATA/$rel"); then c_err "[$ENV_NAME] 路径含引号,无法走 sqlite 一致性导出:$rel" return 1 fi @@ -5166,7 +5166,7 @@ agent_backup_snapshot() { # $1=dest:svc.env + history.jsonl + data/(*.db 一 rm -f "$dest/data/$rel-wal" "$dest/data/$rel-shm" # .backup 已并入 WAL 内容 else rm -f "$dest/data/$rel.bk" # 失败尝试的半成品不留在快照里 - if head -c 15 "$ENV_DATA/$rel" | grep -qa '^SQLite format 3$'; then + if grep -qa '^SQLite format 3$' < <(head -c 15 "$ENV_DATA/$rel"); then c_err "[$ENV_NAME] SQLite 一致性备份/自检失败:$rel" return 1 fi @@ -5446,7 +5446,7 @@ svc_pid_state() { # NAME → "alive " | "dead" [ -f "$d/svc.pid" ] || { printf dead; return 0; } pid="$(cat "$d/svc.pid" 2>/dev/null || true)" [ -n "$pid" ] || { printf dead; return 0; } - mark="$(sed -n 's/^pidcheck=//p' "$d/svc.env" 2>/dev/null | head -1)" + mark="$(sed -n '/^pidcheck=/{s///p;q;}' "$d/svc.env" 2>/dev/null)" [ -z "$mark" ] && mark="agent-system" if pid_alive_mark "$pid" "$mark"; then printf 'alive %s' "$pid"; else printf dead; fi return 0 @@ -5479,7 +5479,7 @@ db_ref_of() { cf="$(env_kv_of "$name" conffile)" [ -z "$cf" ] && cf="$(printf '%s' "$(env_kv_of "$name" exec)" | sed -n 's/.*--config \([^ ]*\).*/\1/p')" if [ -n "$cf" ] && [ -f "$cf" ]; then - dsn="$(grep -oE '"dsn"[[:space:]]*:[[:space:]]*"[^"]*"' "$cf" 2>/dev/null | head -1 | sed -E 's/.*"([^"]*)"$/\1/')" + dsn="$(sed -n '/"dsn"[[:space:]]*:[[:space:]]*"[^"]*"/{s//\n&/;s/.*\n"dsn"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p;q;}' "$cf" 2>/dev/null)" fi fi [ -n "$dsn" ] || return 0 @@ -5503,10 +5503,10 @@ global_list() { while IFS= read -r name; do names+=("$name"); done < <(registered_names) for name in ${names[@]+"${names[@]}"}; do d="$RUNTIME/$name" - kind="$(sed -n 's/^kind=//p' "$d/svc.env" | head -1)"; [ -n "$kind" ] || kind=agent - addr="$(sed -n 's/^addr=//p' "$d/svc.env" | head -1)" - src="$(sed -n 's/^source=//p' "$d/svc.env" | head -1)" - tags="$(sed -n 's/^tags=//p' "$d/svc.env" | head -1)" + kind="$(sed -n '/^kind=/{s///p;q;}' "$d/svc.env")"; [ -n "$kind" ] || kind=agent + addr="$(sed -n '/^addr=/{s///p;q;}' "$d/svc.env")" + src="$(sed -n '/^source=/{s///p;q;}' "$d/svc.env")" + tags="$(sed -n '/^tags=/{s///p;q;}' "$d/svc.env")" if [ -n "${LIST_TAG:-}" ]; then case ",$tags," in *",${LIST_TAG},"*) ;; *) continue ;; esac fi @@ -5673,7 +5673,7 @@ info_file_mode() { # info 遍历多服务时使用;无则空)。 env_kv_of() { [ -f "$RUNTIME/$1/svc.env" ] || return 0 - sed -n "s/^$2=//p" "$RUNTIME/$1/svc.env" | head -1 + sed -n "/^$2=/{s///p;q;}" "$RUNTIME/$1/svc.env" } # info_snapshot_lines FILE:逐行打印 KEY=VALUE 内容快照(按 CLI_REVEAL 决定掩码)。 @@ -5936,7 +5936,7 @@ cmd_update() { # rdm@ 走独立 RDM 源仓。 local latest_sha="" latest_rel="" latest_rdm="" need_rel=0 need_dev=0 need_rdm=0 e_src for e in ${targets[@]+"${targets[@]}"}; do - e_src="$(sed -n 's/^source=//p' "$RUNTIME/$e/svc.env" | head -1)" + e_src="$(sed -n '/^source=/{s///p;q;}' "$RUNTIME/$e/svc.env")" case "$e_src" in release@*) need_rel=1 ;; local@*|dev@*) need_dev=1 ;; rdm@*) need_rdm=1 ;; esac done local fetch_note="" src_args="" @@ -5974,7 +5974,7 @@ cmd_update() { for e in ${targets[@]+"${targets[@]}"}; do local kind src cur st_running kind="$(svc_kind "$e")" - src="$(sed -n 's/^source=//p' "$RUNTIME/$e/svc.env" | head -1)" + src="$(sed -n '/^source=/{s///p;q;}' "$RUNTIME/$e/svc.env")" to="" latest_disp="" case "$kind" in @@ -6083,7 +6083,7 @@ cmd_update() { # 每个服务独立子进程、清空环境,避免批内 AGENT_SYSTEM_* 串实例。 # 原样带上 plugins=,避免在终端上再次询问,也避免 ${src##*@} 把插件提交当成基座。 local plug plug_args=() - plug="$(sed -n 's/^plugins=//p' "$RUNTIME/$name/svc.env" | head -1)" + plug="$(sed -n '/^plugins=/{s///p;q;}' "$RUNTIME/$name/svc.env")" if [ -n "$plug" ]; then plug_args=(--with "$plug") else diff --git a/scripts/task.sh b/scripts/task.sh index c15a0f11..c88d46ce 100755 --- a/scripts/task.sh +++ b/scripts/task.sh @@ -20,13 +20,13 @@ fetch_target() { } origin_transport_unavailable() { # $1=失败的远端命令 stderr → 仅明确的传输失败(DNS/连接超时/拒绝/重置/无路由,或本地缺失仓库路径)返回 0;认证/授权/权限与未知错误返回 1 local err=$1 url path - if printf '%s\n' "$err" | grep -qiE 'authentication failed|permission denied|unauthori[sz]ed|forbidden|HTTP[^0-9]*40[13]|\b40[13]\b|could not read (username|password)|terminal prompts disabled|invalid credentials|access denied|host key verification failed'; then + if grep -qiE 'authentication failed|permission denied|unauthori[sz]ed|forbidden|HTTP[^0-9]*40[13]|\b40[13]\b|could not read (username|password)|terminal prompts disabled|invalid credentials|access denied|host key verification failed' <<<"$err"; then return 1 fi - if printf '%s\n' "$err" | grep -qiE 'could not resolve (host|hostname|proxy)|name or service not known|temporary failure in name resolution|nodename nor servname|connection timed out|operation timed out|timed out after|connection refused|connection reset|no route to host|network is unreachable|couldn.t connect to server'; then + if grep -qiE 'could not resolve (host|hostname|proxy)|name or service not known|temporary failure in name resolution|nodename nor servname|connection timed out|operation timed out|timed out after|connection refused|connection reset|no route to host|network is unreachable|couldn.t connect to server' <<<"$err"; then return 0 fi - printf '%s\n' "$err" | grep -qF 'does not appear to be a git repository' || return 1 + grep -qF 'does not appear to be a git repository' <<<"$err" || return 1 url=$(git remote get-url origin 2>/dev/null) || return 1 case "$url" in file://*) path=${url#file://} ;; @@ -211,12 +211,12 @@ cmd_precheck() { cmd_start() { need_repo local name="${1:-}"; [ -n "$name" ] || die "usage: task.sh start /" - echo "$name" | grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' || die "branch name must match (feature|fix|hotfix|release)/" + grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' <<<"$name" || die "branch name must match (feature|fix|hotfix|release)/" local kind="${TASK_BRANCH_KIND:-standalone}" initiative="${TASK_INITIATIVE:-}" case "$kind" in standalone|initiative|child) ;; *) die "TASK_BRANCH_KIND must be standalone, initiative or child" ;; esac if [ "$kind" = child ]; then [ -n "$initiative" ] || die "child task requires INITIATIVE=" - echo "$initiative" | grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' || die "invalid parent initiative branch" + grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' <<<"$initiative" || die "invalid parent initiative branch" case "${name#*/}" in "${initiative#*/}"-?*) ;; *) die "child slug must start with parent initiative slug plus '-'" ;; esac fi local slug planned_child=0 @@ -312,8 +312,8 @@ cmd_start() { esac if [ "$kind" != child ] && [ -n "${TASK_PARENT_BRANCH:-}${TASK_PARENT_PR:-}" ]; then [ -n "${TASK_PARENT_BRANCH:-}" ] && [ -n "${TASK_PARENT_PR:-}" ] || die "stacked task requires TASK_PARENT_BRANCH and TASK_PARENT_PR" - echo "$TASK_PARENT_BRANCH" | grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' || die "invalid parent task branch" - echo "$TASK_PARENT_PR" | grep -qE '^https?://[^[:space:]]+/pulls/[0-9]+$' || die "TASK_PARENT_PR must be a parent PR URL" + grep -qE '^(feature|fix|hotfix|release)/[A-Za-z0-9._-]+$' <<<"$TASK_PARENT_BRANCH" || die "invalid parent task branch" + grep -qE '^https?://[^[:space:]]+/pulls/[0-9]+$' <<<"$TASK_PARENT_PR" || die "TASK_PARENT_PR must be a parent PR URL" case "${name#*/}" in "${TASK_PARENT_BRANCH#*/}"-?*) ;; *) die "stacked child slug must start with parent slug plus '-'" ;; esac parent_head=$(git rev-parse --verify "refs/heads/${TASK_PARENT_BRANCH}^{commit}") || die "parent branch missing locally" git merge-base --is-ancestor "$rmain" "$parent_head" || die "parent branch does not include current origin/${TARGET_BRANCH}; sync it first" @@ -640,13 +640,13 @@ cmd_sync() { # 默认闭合方式是 close --self:任务分支在合入前把本分支的卡改为"已完成(合入前闭合)"并归档, # 随本 PR 合入;上述合入后回填只用于遗留卡。 -card_of_branch() { sed -n 's/^- 工作区:.*分支:\([A-Za-z0-9._/-]\+\).*/\1/p' "$1" | head -1; } -card_status() { sed -n 's/^.*;状态:\([^;]*\).*/\1/p' "$1" | head -1; } +card_of_branch() { sed -n '/^- 工作区:.*分支:\([A-Za-z0-9._/-]\+\).*/{s//\1/p;q;}' "$1"; } +card_status() { sed -n '/^.*;状态:\([^;]*\).*/{s//\1/p;q;}' "$1"; } # 两级分支元数据行:'- 分支类型:;initiative_id:…;root_target:…;immediate_target:…;accepting_requirements:…'。 # 旧卡没有该行:按 standalone、root/immediate 均为目标分支、不接收新需求处理。 -card_field() { sed -n "s/^- 分支类型:.*;$2:\([^;]*\).*/\1/p" "$1" 2>/dev/null | head -1; } -card_r0() { sed -n 's/^- 权威仓:.*;R0(initial_base_sha)= \([0-9a-f]\+\)[[:space:]]*$/\1/p' "$1" 2>/dev/null | head -1; } -card_branch_kind() { local v; v=$(sed -n 's/^- 分支类型:\([^;]*\).*/\1/p' "$1" 2>/dev/null | head -1); printf '%s\n' "${v:-standalone}"; } +card_field() { sed -n "/^- 分支类型:.*;$2:\([^;]*\).*/{s//\1/p;q;}" "$1" 2>/dev/null; } +card_r0() { sed -n '/^- 权威仓:.*;R0(initial_base_sha)= \([0-9a-f]\+\)[[:space:]]*$/{s//\1/p;q;}' "$1" 2>/dev/null; } +card_branch_kind() { local v; v=$(sed -n '/^- 分支类型:\([^;]*\).*/{s//\1/p;q;}' "$1" 2>/dev/null); printf '%s\n' "${v:-standalone}"; } card_initiative_id() { card_field "$1" initiative_id; } card_root_target() { local v; v=$(card_field "$1" root_target); printf '%s\n' "${v:-${TARGET_BRANCH}}"; } card_immediate_target() { local v; v=$(card_field "$1" immediate_target); printf '%s\n' "${v:-${TARGET_BRANCH}}"; } @@ -1067,7 +1067,7 @@ EOF -d "{\"head\":\"${headbr}\",\"base\":\"${pr_base}\",\"title\":\"Close merged task cards (${headbr})\",\"body\":\"Auto-generated by task.sh close; see commit message for evidence.\"}" 2>&1) || resp="" pr_url=$(printf '%s' "$resp" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("html_url","") if isinstance(d,dict) else "")' 2>/dev/null || true) [ -n "$pr_url" ] && break - echo "[close] 建 PR 第 ${attempt} 次未成功${resp:+($(printf '%s' "$resp" | head -c 120))},退避重试" >&2 + echo "[close] 建 PR 第 ${attempt} 次未成功${resp:+($(head -c 120 <<<"$resp"))},退避重试" >&2 [ "$attempt" = 4 ] || sleep $((attempt * 3)) done if [ -z "$pr_url" ]; then diff --git a/tests/load/collaboration/run_pgbench.sh b/tests/load/collaboration/run_pgbench.sh index 1be22d43..6cba72b4 100755 --- a/tests/load/collaboration/run_pgbench.sh +++ b/tests/load/collaboration/run_pgbench.sh @@ -177,13 +177,13 @@ echo "[pgbench] results $RESULTS" df -h "$ROOT" pgbench --version echo "--- heavy processes (comm only) ---" - ps -eo pcpu,pmem,comm --sort=-pcpu | head -12 + ps -eo pcpu,pmem,comm --sort=-pcpu | sed -n '1,12p' } >"$RESULTS/host.txt" 2>&1 echo "[pgbench] creating isolated instance $NAME on 127.0.0.1:$PORT" "$ROOT/scripts/svc.sh" db create "$NAME" --port "$PORT" >"$RESULTS/svc-create.txt" 2>&1 OWNED=1 -addr="$(sed -n 's/^addr=//p' "$ROOT/runtime/$NAME/svc.env" | head -1)" +addr="$(sed -n '/^addr=/{s///p;q;}' "$ROOT/runtime/$NAME/svc.env")" [[ "$addr" == "127.0.0.1:$PORT" ]] || { echo "[pgbench] instance addr is $addr, expected 127.0.0.1:$PORT; aborting" >&2 exit 1 diff --git a/tests/test_shell_pipe_hazard.py b/tests/test_shell_pipe_hazard.py new file mode 100644 index 00000000..f7a8c9c6 --- /dev/null +++ b/tests/test_shell_pipe_hazard.py @@ -0,0 +1,311 @@ +"""托管 shell 脚本:管道末级不得是提前退出的读取器(grep -q/-m、head、sed q)。 + +pipefail 下读取器先退出,生产者写满 64 KiB 管道缓冲后收到 SIGPIPE,管道状态报 141, +`if producer | grep -q PAT` 与 `if ! ...` 的判定被静默翻转。改用 here-string、进程替换 +或 sed 打印即退出,读取器的结论才不会丢。 +""" +import re +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +SCRIPTS = ( + "scripts/svc.sh", + "scripts/task.sh", + "scripts/e2e.sh", + "scripts/lint-boundaries.sh", + "docker/verify-env.sh", + "tests/load/collaboration/run_pgbench.sh", +) + +HEREDOC = re.compile(r"<<(-?)\s*(['\"]?)([A-Za-z_][A-Za-z0-9_]*)\2") +SED_QUIT = re.compile(r"(?:^|[\s;{}'\"/$0-9])[qQ][0-9]*\s*(?:$|[\s;}'\"])") +GREP_ARG_OPTS = set("efABCdDX") +BOUNDARY = set(" \t\n;&|()<>") + + +class _Frame: + """一层命令上下文(顶层、$(...)、(...)、<(...))内正在累积的管道。""" + + def __init__(self, line): + self.piped, self.stage, self.start, self.cases = False, [], line, [] + + +def pipeline_tails(src): + """词法扫描 bash 源码,返回每条多级管道末级的 (起始行号, 文本);跳过注释与 heredoc 正文。""" + tails, frames, modes, heredocs = [], [_Frame(1)], ["cmd"], [] + line, word, i, n = 1, "", 0, len(src) + + def emit(s): + for f in frames: + f.stage.append(s) + + def finish(f): + if f.piped: + tails.append((f.start, "".join(f.stage).strip())) + f.piped, f.stage, f.start = False, [], line + + def case_state(): + cases = frames[-1].cases + return cases[-1] if cases else None + + def keyword(w): + cases = frames[-1].cases + if w == "case": + cases.append("in") + elif w == "in" and case_state() == "in": + cases[-1] = "pattern" + elif w == "esac" and cases: + cases.pop() + + def push_cmd(opener): + emit(opener) + modes.append("cmd") + frames.append(_Frame(line)) + + while i < n: + c, nxt = src[i], src[i + 1 : i + 2] + mode = modes[-1] + if mode in ("sq", "ansi"): + if c == "\\" and mode == "ansi": + emit(src[i : i + 2]) + i += 2 + continue + emit(c) + line += c == "\n" + if c == "'": + modes.pop() + i += 1 + continue + if mode in ("dq", "brace"): + if c == "\\": + emit(src[i : i + 2]) + line += nxt == "\n" + i += 2 + elif c == "$" and nxt == "(": + push_cmd("$(") + i += 2 + elif c == "$" and nxt == "{": + emit("${") + modes.append("brace") + i += 2 + elif c == '"': + emit(c) + if mode == "dq": + modes.pop() + else: + modes.append("dq") + i += 1 + elif c == "}" and mode == "brace": + emit(c) + modes.pop() + i += 1 + else: + emit(c) + line += c == "\n" + i += 1 + continue + + if c in BOUNDARY and word: + keyword(word) + word = "" + top = frames[-1] + if c == "\\": + if nxt == "\n": + emit(" ") + line += 1 + else: + emit(src[i : i + 2]) + word += "\0" + i += 2 + elif c == "#" and (i == 0 or src[i - 1] in " \t\n;&|("): + j = src.find("\n", i) + i = n if j < 0 else j + elif c == "'" or (c == "$" and nxt == "'"): + emit(src[i : i + 2] if c == "$" else c) + modes.append("ansi" if c == "$" else "sq") + word += "\0" + i += 2 if c == "$" else 1 + elif c == '"': + emit(c) + modes.append("dq") + word += "\0" + i += 1 + elif c == "$" and nxt == "{": + emit("${") + modes.append("brace") + word += "\0" + i += 2 + elif c == "$" and nxt == "(": + push_cmd("$(") + word += "\0" + i += 2 + elif c == "(": + if case_state() == "pattern": + emit(c) + else: + push_cmd(c) + i += 1 + elif c == ")": + if case_state() == "pattern": + top.cases[-1] = "body" + elif len(frames) > 1: + finish(top) + frames.pop() + modes.pop() + emit(c) + i += 1 + elif c == "|": + if nxt == "|": + finish(top) + emit("||") + i += 2 + elif case_state() == "pattern": + emit(c) + i += 1 + else: + emit(c) + top.piped, top.stage, top.start = True, [], line + i += 2 if nxt == "&" else 1 + elif c == ";": + if src.startswith(";;", i) or src.startswith(";&", i): + if case_state() == "body": + top.cases[-1] = "pattern" + k = 3 if src.startswith(";;&", i) else 2 + else: + k = 1 + finish(top) + emit(src[i : i + k]) + i += k + elif c == "&": + if nxt == "&": + finish(top) + emit("&&") + i += 2 + elif nxt == ">" or (i > 0 and src[i - 1] in "<>"): + emit(c) + i += 1 + else: + finish(top) + emit(c) + i += 1 + elif c == "<" and src.startswith("<<", i): + m = None if src.startswith("<<<", i) else HEREDOC.match(src, i) + if m: + heredocs.append((m.group(3), m.group(1) == "-")) + emit(m.group(0)) + i = m.end() + else: + k = 3 if src.startswith("<<<", i) else 2 + emit(src[i : i + k]) + i += k + elif c == "\n": + line += 1 + i += 1 + for term, dash in heredocs: + while i < n: + j = src.find("\n", i) + body = src[i : n if j < 0 else j] + i = n if j < 0 else j + 1 + line += 1 + if (body.lstrip("\t") if dash else body) == term: + break + heredocs = [] + if not (top.piped and not "".join(top.stage).strip()): + finish(top) + emit("\n") + else: + emit(c) + if c not in BOUNDARY: + word += c + i += 1 + + if len(frames) != 1 or modes != ["cmd"]: + raise ValueError(f"unbalanced shell lexing at EOF: modes={modes}") + finish(frames[0]) + return tails + + +def _grep_exits_early(words): + for w in words[1:]: + w = w.strip("'\"") + if w == "--": + break + if w.startswith("--"): + if w.split("=")[0] in ("--quiet", "--silent", "--max-count"): + return True + elif w.startswith("-") and len(w) > 1: + for ch in w[1:]: + if ch in "qm": + return True + if ch in GREP_ARG_OPTS: + break + return False + + +def early_exit_reader(stage): + """末级命令读到结论后是否会提前关闭 stdin(grep -q/-m、head、带 q 命令的 sed)。""" + words = stage.split() + if not words: + return False + cmd = words[0] + if cmd in ("grep", "egrep", "fgrep"): + return _grep_exits_early(words) + if cmd == "head": + return True + if cmd == "sed": + return bool(SED_QUIT.search(stage.split(None, 1)[1] if len(words) > 1 else "")) + return False + + +def hazards(src): + return [(line, text) for line, text in pipeline_tails(src) if early_exit_reader(text)] + + +class ShellPipeHazardTests(unittest.TestCase): + def test_detector_flags_synthetic_hazard(self): + found = hazards("set -euo pipefail\nseq 1 200000 | grep -q 1\n") + self.assertEqual([line for line, _ in found], [2], found) + + def test_detector_flags_hazard_shapes(self): + for snippet in ( + 'if ! seq 1 200000 | grep -q 1; then echo miss; fi', + 'x="$(sed -n \'s/^k=//p\' "$f" | head -1)"', + 'if ! "$bin" serve -h 2>&1 | grep -Eq -- "(^|[[:space:]])-{1,2}config"; then :; fi', + 'echo "${r:+($(printf \'%s\' "$r" | head -c 120))}"', + 'find . -type f \\\n | grep -m1 x', + "seq 1 200000 | sed -n '/1/{p;q;}'", + 'seq 1 200000 | grep -qiE "a|b" || exit 1', + ): + with self.subTest(snippet=snippet): + self.assertTrue(hazards(snippet + "\n"), snippet) + + def test_detector_accepts_safe_forms(self): + src = "\n".join(( + "set -euo pipefail", + "# seq 1 200000 | grep -q 1", + "grep -q 1 < <(seq 1 200000)", + 'grep -q x <<<"$v"', + "x=\"$(sed -n '/^k=/{s///p;q;}' \"$f\")\"", + "ps -eo comm | sed -n '1,12p'", + "case \"$w\" in --auth|--config) grep -q x <<<\"$w\" ;; esac", + "cat <<'EOF'", + "seq 1 200000 | grep -q 1", + "EOF", + "", + )) + self.assertEqual(hazards(src), []) + + def test_managed_scripts_have_no_early_exit_pipe_tail(self): + for rel in SCRIPTS: + with self.subTest(script=rel): + path = ROOT / rel + self.assertTrue(path.is_file(), f"{rel} missing") + found = hazards(path.read_text(encoding="utf-8")) + self.assertEqual( + found, [], "\n".join(f"{rel}:{line}: | {text}" for line, text in found) + ) + + +if __name__ == "__main__": + unittest.main()