Merge pull request 'Gate deploys on identity capabilities and a reversible release' (#485) from feature/svc-d1-identity-flow into main
ci / svc-realcombo (push) Successful in 1m42s
ci / test-pg (push) Successful in 2m22s
ci / test (push) Successful in 4m29s

Why: deploys need a side-effect-free identity probe, an isolated lifecycle child, and the release transaction already on main.
Evidence: GATE run 2667 jobs 3845 test, 3846 test-pg, 3847 svc-realcombo all success on b4eb3a814c.
Impact: update and recover run in a clean child; serve keeps the exec-tool kill switch; Backend pin is 0f192a4.
Refs: #485
This commit was merged in pull request #485.
This commit is contained in:
2026-10-12 15:41:31 +00:00
18 changed files with 1087 additions and 143 deletions
+2
View File
@@ -377,6 +377,8 @@ jobs:
env:
AGENT_PG_TEST_DSN: postgres://ci_agent:ci_job_local_only@127.0.0.1:5432/agent_test?sslmode=disable
run: make test-multi-instance
- name: RDM lifecycle pin is three fixed SHAs
run: python3 -m unittest tests.test_lifecycle_pin -q && bash scripts/fetch-rdm-pin.sh tests/integration/lifecycle/rdm.pin >/dev/null
- name: Clone Agent-System-RDM with nested pins (reviewed allowlist addition)
env:
SUBMODULE_TOKEN: ${{ secrets.SUBMODULE_TOKEN }}
File diff suppressed because one or more lines are too long
+9 -1
View File
@@ -13,7 +13,7 @@ PLUGINS := plugins
COMBO := core=$(shell git -C $(CORE) rev-parse --short HEAD 2>/dev/null)+backend=$(shell git -C $(BACKEND) rev-parse --short HEAD 2>/dev/null)+frontend=$(shell git -C $(FRONTEND) rev-parse --short HEAD 2>/dev/null)+ui=$(shell git -C $(UI) rev-parse --short HEAD 2>/dev/null)+agentcmd=$(shell git -C deps/agent-command rev-parse --short HEAD 2>/dev/null)+documents=$(shell git -C modules/documents rev-parse --short HEAD 2>/dev/null)+native=$(shell git -C deps/agent-native rev-parse --short HEAD 2>/dev/null)+squad=$(shell git -C deps/agent-squad rev-parse --short HEAD 2>/dev/null)+intent=$(shell git -C deps/agent-intent rev-parse --short HEAD 2>/dev/null)
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null)+$(COMBO)
.PHONY: help sync status build frontend svc svc-list svc-info svc-commits svc-history svc-update svc-recover svc-watchdog test-backend test-backend-pg test-multi-instance test-frontend test-ui test-core test-agentcmd test-pins test-libs dev pin clean setup precheck task-start task-sync task-close task-finish facts-verify evidence version
.PHONY: help sync status build frontend svc svc-list svc-info svc-commits svc-history svc-update svc-recover svc-watchdog test-backend test-backend-pg test-multi-instance test-lifecycle test-frontend test-ui test-core test-agentcmd test-pins test-libs dev pin clean setup precheck task-start task-sync task-close task-finish facts-verify evidence version
version:
@echo "agent-system $(VERSION)"
@@ -112,6 +112,14 @@ test-backend-pg:
@test -n "$${AGENT_REDIS_TEST_ADDR}" || { echo "test-backend-pg: AGENT_REDIS_TEST_ADDR 未设置(真实 Redis 必需,不静默跳过)"; exit 1; }
cd $(BACKEND) && go test -count=1 ./...
# 真实 Dev/RDM + PostgreSQL 生命周期门禁。缺环境直接失败,不静默跳过。
# 登录入口未就绪时套件保持失败(NOT_RUN),不把 D3 报成完成。
test-lifecycle:
@test -n "$${AGENT_PG_TEST_DSN}" || { echo "test-lifecycle: AGENT_PG_TEST_DSN 未设置(真实 PostgreSQL 必需,不静默跳过)"; exit 1; }
@test -n "$${SVC_RDM_SRC_DIR}" || { echo "test-lifecycle: SVC_RDM_SRC_DIR 未设置(RDM 检出必需,不静默跳过)"; exit 1; }
bash scripts/fetch-rdm-pin.sh tests/integration/lifecycle/rdm.pin >/dev/null
python3 tests/integration/lifecycle/lifecycle_suite.py
# 双进程 HTTP 级验收(协作存储方案 U7):两个真实 serve 进程共享隔离 PG。
# 缺环境时明确失败而非静默跳过;见 tests/integration/multi-instance/README.md。
test-multi-instance:
+3 -3
View File
@@ -38,10 +38,10 @@ U2 起实施与部署分别评审。
- [RDM 最终目标核对与完整交付方案 r1](modules/documents/docs/proposals/2026-10-11-rdm-final-target-completion.md):系统级项目、上限内权限覆盖、私域协作、全部文档三权与完整业务链;34 UR/68 SR/10 F 逐条映射;方案已随 #340 合入(2026-10-11),U1 需求基线核对经 RDM #199/#200 落地,U2 起未启动。
- [服务重启与全量数据恢复方案 r2](modules/documents/docs/proposals/2026-10-12-service-restart-data-recovery.md):
- [服务重启与全量数据恢复方案 r3.1](modules/documents/docs/proposals/2026-10-12-service-restart-data-recovery.md):
svc 启停、异常进程/主机退出后的依赖、配置/数据身份、任务/HITL/副作用、文件与组合备份恢复;
r1 经六角色自审、r2 吸收外部源码复核;2026-10-12 用户批准 r2(五阶段一并),实施、故障注入、部署与运行验收 NOT_RUN;
衔接协作存储案;U2 首切片已由 svc-autorecover 合入 main(PR #396/#401);U1 实例身份已合入(Dev #415);余量另立增量。
r2 已批准。r3.1 增补 D1 实例身份收尾、D2 可回滚切换、D3 真实组合门禁,并记录 2026-10-12 rdmcand 数据归属:会话不在 dev 库的 rdmcand 窗口内,领域事件为空,不跨实例搬迁。
D1/D2/D3 按实施计划推进到可合入;r3 其余未批条款仍待人工评审。
- [svc 统一部署与一次性登录 Key 方案 r1](modules/documents/docs/proposals/2026-10-12-svc-unified-deploy-and-login-key.md):
二进制/源码统一部署、环境准备、无初始密码开户与账户恢复;一次性 Key 仅兑换设密受限会话;
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# 按 tests/integration/lifecycle/rdm.pin 取出固定 SHA。只接受 40 位 hex。
# 不执行 pin 文件,不接受 URL。检出由调用方用已允许的 RDM origin 完成。
set -euo pipefail
pin="${1:-tests/integration/lifecycle/rdm.pin}"
[ -f "$pin" ] || { echo "rdm.pin 不存在:$pin" >&2; exit 1; }
declare -A got=()
while IFS= read -r line || [ -n "$line" ]; do
case "$line" in
''|\#*) continue ;;
esac
key="${line%%=*}"
val="${line#*=}"
case "$key" in
sha|prev_sha|legacy_sha) ;;
*) echo "rdm.pin 未知键:$key" >&2; exit 1 ;;
esac
case "$val" in
[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]) ;;
*) echo "rdm.pin 的 $key 不是 40 位 hex" >&2; exit 1 ;;
esac
got["$key"]="$val"
done < "$pin"
[ "${#got[@]}" -eq 3 ] || { echo "rdm.pin 必须同时有 sha、prev_sha、legacy_sha" >&2; exit 1; }
printf '%s\n' "${got[sha]}"
+537 -111
View File
@@ -700,6 +700,8 @@ parse_args() {
--apply) CLI_APPLY=1; shift ;;
--force) CLI_FORCE=1; shift ;;
--allow-source-switch) CLI_ALLOW_SOURCE_SWITCH=1; shift ;;
--migrate-schema) CLI_MIGRATE_SCHEMA=1; shift ;;
--confirm) need_value "$@"; CLI_CONFIRM="$2"; shift 2 ;;
--online) CLI_ONLINE=1; shift ;;
--file) need_value "$@"; CLI_FILE="$2"; shift 2 ;;
--out) need_value "$@"; CLI_OUT="$2"; shift 2 ;;
@@ -1055,12 +1057,19 @@ svc_manifest_migrate() {
out="$(python3 "$MANIFEST_PY" "$ENV_DIR" migrate 2>&1)" || rc=$?
if [ "$rc" -eq 0 ]; then
c_ok "[$ENV_NAME] manifest:$(printf '%s' "$out" | tail -1)"
# 领养即受控建身份:DB marker 只在领养(此处)与显式 init 建立;普通启动只
# 验证不补行——缺行即事件(2026-10-12 裁定,rdmcand 实况:每次启动静默
# ensure 会掩盖身份丢失)。领养时旧产物可能无 identity 能力而跳过补行,由
# 部署候选门禁 svc_deploy_identity_gate 在下次换产物前用候选补齐。
# fresh:库文件已在时用当前制品补行并升为 established;无能力则记 pending。
# 库还不存在时保持 fresh,等启动建库后再 ensure。不在这里为缺库建文件。
resolve_db_config 2>/dev/null || true
svc_db_identity_ensure || return 1
local marker data_root
marker="$(svc_db_marker)"
if [ "$marker" = fresh ]; then
data_root="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["identity"]["data_root"])' "$ENV_DIR/manifest.json")"
if ! svc_bin_has_identity; then
svc_db_marker_set pending || return 1
elif [ -f "$data_root/agent-system.db" ]; then
svc_db_identity_ensure || return 1
fi
fi
return 0
fi
if [ "$rc" -eq 4 ]; then
@@ -1086,8 +1095,35 @@ svc_manifest_verify() {
c_err " 数据根/数据库绑定是冻结身份;换根/重绑须显式迁移或登记命令(docs/recovery/u1-identity-manifest.md §5)"
return 1
fi
# U1 I3:文件校验通过后再碰库——避免损坏 manifest 时身份 CLI 先 traceback。
svc_db_identity_verify || return 1
# 文件校验通过后再碰库。fresh 允许库尚未建立;established 只验证;
# pending 仅当库文件已在且制品有能力时补齐。无能力制品不得启动已治理实例。
local kind marker data_root
kind="$(svc_get kind)"
[ "$kind" = agent ] || return 0
marker="$(svc_db_marker)"
case "$marker" in
fresh) return 0 ;;
pending)
if ! svc_bin_has_identity; then
c_err "[$ENV_NAME] 身份尚未建立(pending),当前制品不能验证身份,拒绝启动"
return 1
fi
data_root="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["identity"]["data_root"])' "$ENV_DIR/manifest.json")"
if [ ! -f "$data_root/agent-system.db" ] && [ "${DB_MODE:-}" != pg ]; then
c_err "[$ENV_NAME] 身份 pending 但库文件不存在,拒绝启动(不新建空库)"
return 1
fi
svc_db_identity_ensure || return 1
svc_db_identity_verify || return 1
;;
*)
if ! svc_bin_has_identity; then
c_err "[$ENV_NAME] 已建立身份的实例只接受能验证身份的制品,拒绝启动"
return 1
fi
svc_db_identity_verify || return 1
;;
esac
return 0
}
@@ -1112,28 +1148,84 @@ print("%s\t%s\t%s" % (m["instance_id"], dbref, data))
PY
}
# svc_bin_has_identity [BIN]:产物是否具备 identity 管理子命令(ensure/verify)。
# 探测必须无副作用、有界:timeout 护栏(未知子命令直落服务态的产物会挂死或占口);
# serve 痕迹与身份门禁报错文本不算能力证据——2026-10-12 rdm1/rdmcand 事故中 RDM
# 产物对 identity 未知位置参数落入服务启动路径,报错恰好含「identity ensure」字样,
# 旧实现 grep ensure 误报为有能力。仅 usage 形态(同时含 identity 与 ensure、
# 无服务态痕迹、未超时)才认可。
svc_bin_has_identity() {
# svc_bin_caps [BIN]:只执行 version --json,在空目录、清空的环境里。非 JSON 或
# protocol<1 一律无能力。旧制品对 version 立即退出,不会进入服务路径。
# 按二进制摘要缓存;缓存目录默认在临时目录,不写进实例运行时。
svc_bin_caps() {
local bin="${1:-${ENV_BIN:-}}"
[ -x "$bin" ] || return 1
local out rc=0
out="$(timeout 5 "$bin" identity 2>&1)" || rc=$?
[ "$rc" -eq 124 ] && return 1
case "$out" in
*serving*|*"agent-system starting"*|*"identity missing"*|*"listen"*) return 1 ;;
esac
printf '%s' "$out" | grep -q identity && printf '%s' "$out" | grep -q ensure
local digest cache tmp out
digest="$(sha256sum "$bin" 2>/dev/null | awk '{print $1}')"
[ -n "$digest" ] || return 1
cache="${SVC_CAPS_CACHE_DIR:-${TMPDIR:-/tmp}}/svc-caps-${digest}"
if [ -f "$cache" ]; then
cat "$cache"
return 0
fi
tmp="$(mktemp -d)"
out="$(cd "$tmp" && timeout -k 2 5 env -i PATH=/usr/bin:/bin HOME="$tmp" "$bin" version --json 2>/dev/null)" || true
rm -rf "$tmp"
out="$(printf '%s' "$out" | python3 -c 'import json,sys; d=json.load(sys.stdin); assert int(d.get("protocol",0))>=1; print(json.dumps(d,separators=(",",":")))' 2>/dev/null)" || return 1
printf '%s\n' "$out" > "$cache"
printf '%s\n' "$out"
}
# svc_bin_has_identity [BIN]:能力文档声明 identity.ops 含 ensure 与 verify。
svc_bin_has_identity() {
local bin="${1:-${ENV_BIN:-}}" doc
doc="$(svc_bin_caps "$bin")" || return 1
printf '%s' "$doc" | python3 -c 'import json,sys; d=json.load(sys.stdin); ops=(d.get("identity") or {}).get("ops") or []; raise SystemExit(0 if "ensure" in ops and "verify" in ops else 1)'
}
# svc_db_marker:manifest.identity.db_marker。缺字段的旧 manifest 视为 established
#(已在治理下跑过的实例,缺行即事件,不再自动补)。
svc_db_marker() {
manifest_ok || { printf '%s' established; return 0; }
python3 - "$ENV_DIR" <<'PY'
import json, os, sys
p = os.path.join(sys.argv[1], "manifest.json")
m = json.load(open(p, encoding="utf-8"))
print((m.get("identity") or {}).get("db_marker") or "established")
PY
}
svc_db_marker_set() {
local marker="$1"
manifest_ok || return 0
python3 - "$ENV_DIR" "$marker" <<'PY'
import json, os, sys
env_dir, marker = sys.argv[1], sys.argv[2]
path = os.path.join(env_dir, "manifest.json")
m = json.load(open(path, encoding="utf-8"))
ident = dict(m.get("identity") or {})
if ident.get("db_marker") == marker:
sys.exit(0)
ident["db_marker"] = marker
m["identity"] = ident
m["revision"] = int(m.get("revision") or 0) + 1
tmp = path + ".new"
with open(tmp, "w", encoding="utf-8") as f:
json.dump(m, f, ensure_ascii=False, indent=2)
f.write("\n")
os.replace(tmp, path)
PY
}
# svc_identity_run BIN OP:身份子命令。DSN 只经环境(AGENT_SYSTEM_DSN),不进 argv。
svc_identity_run() {
local bin="$1" op="$2" iid="$3" dbref="$4" data="$5"
local -a args=(env -i "PATH=$PATH" "HOME=${HOME:-/tmp}" "TMPDIR=${TMPDIR:-/tmp}"
"AGENT_SYSTEM_INSTANCE_ID=$iid" "AGENT_SYSTEM_DBREF=$dbref" "AGENT_SYSTEM_DATA=$data")
if [ "${DB_MODE:-}" = pg ] && [ -n "${DB_DSN:-}" ]; then
args+=("AGENT_SYSTEM_DSN=$DB_DSN")
fi
args+=("$bin" identity "$op" --instance-id "$iid" --dbref "$dbref" --data "$data")
"${args[@]}"
}
svc_db_identity_ensure() {
# 显式建立 / 领养补行(I2→I3 引导):表空插入;一致幂等;不一致 fail-closed。
# 非 agent 或无 identity 子命令的旧产物:跳过(不硬断 I2)。
# 可选参数 BIN:用指定产物(如部署候选)执行——默认当前安装产物。
# 仅 fresh/pending 或显式 adopt 调用。established 缺行不得走这里自动补。
# 非 agent 或无能力:跳过。可选参数 BIN:部署候选。
local bin="${1:-${ENV_BIN:-}}"
manifest_ok || return 0
local kind; kind="$(svc_get kind)"
@@ -1142,19 +1234,13 @@ svc_db_identity_ensure() {
local line iid dbref data
line="$(svc_db_identity_args)" || return 1
IFS=$'\t' read -r iid dbref data <<<"$line"
local args=(identity ensure --instance-id "$iid" --dbref "$dbref" --data "$data")
# G1-7(U3):pg 档 DSN 经 AGENT_SYSTEM_DSN 环境交付,不进命令行——命令行对
# 同机任何进程可见;产物侧 identity CLI 须支持环境回退(backend #182)。
local dsn_env=()
if [ "${DB_MODE:-}" = pg ] && [ -n "${DB_DSN:-}" ]; then
dsn_env=("AGENT_SYSTEM_DSN=$DB_DSN")
fi
local out rc=0
out="$(env ${dsn_env[@]+"${dsn_env[@]}"} "$bin" "${args[@]}" 2>&1)" || rc=$?
out="$(svc_identity_run "$bin" ensure "$iid" "$dbref" "$data" 2>&1)" || rc=$?
if [ "$rc" -ne 0 ]; then
c_err "[$ENV_NAME] DB 身份 ensure 失败:$out"
return 1
fi
svc_db_marker_set established || return 1
return 0
}
@@ -1162,24 +1248,18 @@ svc_db_identity_verify() {
manifest_ok || return 0
local kind; kind="$(svc_get kind)"
[ "$kind" = agent ] || return 0
svc_bin_has_identity || return 0
local bin="${1:-${ENV_BIN:-}}"
svc_bin_has_identity "$bin" || return 0
local line iid dbref data
line="$(svc_db_identity_args)" || return 1
IFS=$'\t' read -r iid dbref data <<<"$line"
local args=(identity verify --instance-id "$iid" --dbref "$dbref" --data "$data")
# G1-7(U3):DSN 环境交付同 ensure。
local dsn_env=()
if [ "${DB_MODE:-}" = pg ] && [ -n "${DB_DSN:-}" ]; then
dsn_env=("AGENT_SYSTEM_DSN=$DB_DSN")
fi
local out rc=0
out="$(env ${dsn_env[@]+"${dsn_env[@]}"} "${ENV_BIN:-}" "${args[@]}" 2>&1)" || rc=$?
out="$(svc_identity_run "$bin" verify "$iid" "$dbref" "$data" 2>&1)" || rc=$?
if [ "$rc" -ne 0 ]; then
c_err "[$ENV_NAME] DB 身份校验失败,拒绝启动:$out"
c_err " 先 scripts/svc.sh $ENV_NAME init(或补跑 ensure);错库/错实例须人工处置(docs/recovery/u1-identity-manifest.md §5)"
c_err " 先调查库文件与身份行;确认归属后 scripts/svc.sh $ENV_NAME adopt --confirm $iid"
return 1
fi
# 供 serve 二次门禁;verify 成功后导出。
AGENT_SYSTEM_INSTANCE_ID="$iid"
AGENT_SYSTEM_DBREF="$dbref"
export AGENT_SYSTEM_INSTANCE_ID AGENT_SYSTEM_DBREF
@@ -1208,15 +1288,151 @@ svc_deploy_identity_gate() {
return 1
fi
resolve_db_config 2>/dev/null || true
svc_db_identity_ensure "$srcbin" || {
c_err "[$ENV_NAME] 候选补齐 DB 身份失败(外宾行/不一致 fail-closed);旧服务/旧制品/登记保持不变"
return 1
}
local marker; marker="$(svc_db_marker)"
case "$marker" in
fresh|pending)
# 库文件还不存在时不补行、不建库;留给启动后的受控 ensure。
local data_root
data_root="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["identity"]["data_root"])' "$ENV_DIR/manifest.json")"
if [ -f "$data_root/agent-system.db" ] || { [ "${DB_MODE:-}" = pg ] && [ -n "${DB_DSN:-}" ]; }; then
svc_db_identity_ensure "$srcbin" || {
c_err "[$ENV_NAME] 候选补齐 DB 身份失败(外宾行/不一致 fail-closed);旧服务/旧制品/登记保持不变"
return 1
}
fi
;;
*)
# established:只验证,不补行。缺行或不匹配在停旧服务前拒绝。
svc_db_identity_verify "$srcbin" || {
c_err "[$ENV_NAME] 候选无法验证已建立的 DB 身份;旧服务/旧制品/登记保持不变"
return 1
}
;;
esac
svc_schema_compatible "$ENV_BIN" "$srcbin" || return 1
return 0
}
# svc_schema_compatible INSTALLED CANDIDATE:候选 compatFloor 高于已装制品的
# schema.target 时,迁移会让旧制品读不了库。停旧之前拒绝。缺 schema 字段视为兼容。
svc_schema_compatible() {
local installed="$1" candidate="$2"
local icaps ccaps
icaps="$(svc_bin_caps "$installed" 2>/dev/null || true)"
ccaps="$(svc_bin_caps "$candidate" 2>/dev/null || true)"
python3 - "$icaps" "$ccaps" <<'PY'
import json, sys
def load(raw):
try:
return json.loads(raw) if raw else {}
except Exception:
return {}
old, new = load(sys.argv[1]), load(sys.argv[2])
osch, nsch = old.get("schema") or {}, new.get("schema") or {}
if "target" not in osch or "compatFloor" not in nsch:
sys.exit(0)
if int(osch["target"]) < int(nsch["compatFloor"]):
sys.exit(1)
sys.exit(0)
PY
local rc=$?
if [ "$rc" -eq 0 ]; then
return 0
fi
if [ "${CLI_MIGRATE_SCHEMA:-0}" = 1 ]; then
local iid backup
iid="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["instance_id"])' "$ENV_DIR/manifest.json")"
[ "$CLI_CONFIRM" = "$iid" ] || { c_err "[$ENV_NAME] --migrate-schema 需要 --confirm $iid"; return 1; }
backup="$(agent_backup_root)"
if ! find "$backup" -mindepth 1 -maxdepth 1 -type d -print -quit | grep -q .; then
c_err "[$ENV_NAME] forward-only 迁移必须先 backup"
return 1
fi
c_warn "[$ENV_NAME] forward-only schema 迁移已确认,回滚不会覆盖业务库"
return 0
fi
c_err "[$ENV_NAME] 候选 schema 与已装制品不兼容,停旧服务前拒绝"
c_err " 若必须前向迁移:scripts/svc.sh $ENV_NAME deploy --migrate-schema --confirm <instance_id>(会先 backup)"
return 1
}
# svc_mark_autostart / svc_clear_autostart:期望运行态登记(成功 start 置 running,
# 显式 stop 置 stopped;进程自行死亡不动——recover 据 svc.env 派生的 autostart 判断)。
# svc_write_serve_env:首次显式启动把白名单环境(不含凭据、不含 AGENT_SYSTEM_*)
# 写入 0600 的 serve.env。裸启动与 recover 复用它,避免交互壳与 watchdog 环境不同。
svc_write_serve_env() {
local f="$ENV_DIR/serve.env"
[ -f "$f" ] && return 0
( umask 077
{
printf 'PATH=%s\n' "${PATH:-/usr/bin:/bin}"
printf 'HOME=%s\n' "${HOME:-/tmp}"
printf 'USER=%s\n' "${USER:-}"
printf 'LOGNAME=%s\n' "${LOGNAME:-${USER:-}}"
printf 'LANG=%s\n' "${LANG:-C.UTF-8}"
printf 'TZ=%s\n' "${TZ:-UTC}"
printf 'TMPDIR=%s\n' "${TMPDIR:-/tmp}"
} > "$f"
)
chmod 0600 "$f" 2>/dev/null || true
}
# svc_launch_serve ARGS:env -i 只带 serve.env 与本次构造的 guard_env。
svc_launch_serve() {
local exec_args="$1"
local -a args=(env -i)
local line
if [ -f "$ENV_DIR/serve.env" ]; then
while IFS= read -r line || [ -n "$line" ]; do
[ -z "$line" ] && continue
case "$line" in
AGENT_SYSTEM_*|*' '*) continue ;;
*=*) args+=("$line") ;;
esac
done < "$ENV_DIR/serve.env"
fi
if [ "${#guard_env[@]}" -gt 0 ]; then
args+=("${guard_env[@]}")
fi
# 桩只靠 SVC_STUB_* 找到记录目录。这些变量不写入 serve.env,避免 recover
# 把某次测试路径固化进实例;生产调用方没有它们。
# RDM_DISABLE_EXEC_TOOLS 是产品兜底开关,必须到达 serve,否则无 bwrap 时拒启。
local k
while IFS= read -r k; do
[ -n "$k" ] || continue
args+=("$k=${!k}")
done < <(compgen -e | grep -E '^SVC_STUB_' || true)
if [ -n "${RDM_DISABLE_EXEC_TOOLS:-}" ]; then
args+=("RDM_DISABLE_EXEC_TOOLS=$RDM_DISABLE_EXEC_TOOLS")
fi
# shellcheck disable=SC2086
nohup "${args[@]}" "$ENV_BIN" serve $exec_args >> "$ENV_DIR/svc.log" 2>&1 9>&- &
}
# svc_lifecycle_child ACTION [extra args...]:用清空的环境拉起本脚本的
# __lifecycle。白名单:PATH/HOME/USER/LOGNAME/LANG/LC_*/TZ/TMPDIR、代理、SVC_*。
# 从不传入 AGENT_SYSTEM_*。
svc_lifecycle_child() {
local -a args=(env -i "PATH=${PATH:-/usr/bin:/bin}" "HOME=${HOME:-/tmp}"
"USER=${USER:-}" "LOGNAME=${LOGNAME:-${USER:-}}" "LANG=${LANG:-C.UTF-8}"
"TZ=${TZ:-UTC}" "TMPDIR=${TMPDIR:-/tmp}")
local k
for k in http_proxy https_proxy HTTP_PROXY HTTPS_PROXY no_proxy NO_PROXY; do
if [ -n "${!k:-}" ]; then
args+=("$k=${!k}")
fi
done
while IFS= read -r k; do
[ -n "$k" ] || continue
args+=("$k=${!k}")
done < <(compgen -e | grep -E '^(SVC_|LC_)' || true)
if [ -n "${RDM_DISABLE_EXEC_TOOLS:-}" ]; then
args+=("RDM_DISABLE_EXEC_TOOLS=$RDM_DISABLE_EXEC_TOOLS")
fi
args+=(bash "$0" __lifecycle "$@")
"${args[@]}"
}
svc_mark_autostart() { manifest_ok && python3 "$MANIFEST_PY" "$ENV_DIR" set-desired running >/dev/null 2>&1 && return 0; svc_set_key autostart 1 >/dev/null 2>&1 || true; }
svc_clear_autostart() { manifest_ok && python3 "$MANIFEST_PY" "$ENV_DIR" set-desired stopped >/dev/null 2>&1 && return 0; svc_del_key autostart 2>/dev/null || true; }
@@ -1595,23 +1811,161 @@ resolve_source() { # → SRC_KINDSRC(local|dev|release) + SRC_LABEL(+SRC_FULL/
esac
}
install_artifacts() { # SRCBIN SRCPLUGINS:agent 产物复制进服务目录。
# svc_fault_check PHASE:仅当 SVC_TEST_FAULTS=1 且 ROOT 在临时目录时,于该阶段退出。
svc_fault_check() {
local phase="$1"
[ "${SVC_TEST_FAULTS:-}" = 1 ] || return 0
[ "${SVC_FAULT_AT:-}" = "$phase" ] || return 0
case "$ROOT" in
/tmp/*|/var/tmp/*) ;;
*) c_err "[$ENV_NAME] 忽略故障注入:ROOT 不在临时目录"; return 0 ;;
esac
c_err "[$ENV_NAME] 故障注入于 $phase"
exit 86
}
svc_journal_write() {
local phase="$1"
SVC_OP_ID="${SVC_OP_ID:-op-$(date -u +%Y%m%dT%H%M%SZ)-$$}"
python3 - "$ENV_DIR/deploy-journal.json" "$phase" "$SVC_OP_ID" \
"${SVC_FROM_RID:-}" "${SVC_TO_RID:-}" "${SVC_FROM_DESIRED:-running}" \
"${SVC_WAS_RUNNING:-0}" <<'PY'
import json, os, sys
path, phase, op, fr, to, desired, running = sys.argv[1:]
doc = {
"op_id": op,
"from_rid": fr,
"to_rid": to,
"from_desired": desired,
"was_running": running == "1",
"phase": phase,
}
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = path + ".new"
with open(tmp, "w", encoding="utf-8") as f:
json.dump(doc, f, ensure_ascii=False)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
os.chmod(path, 0o600)
PY
svc_fault_check "$phase"
}
svc_journal_phase() {
[ -f "$ENV_DIR/deploy-journal.json" ] || return 0
python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("phase",""))' "$ENV_DIR/deploy-journal.json"
}
# svc_release_switch RID:current 用 ln -s + mv -T 原子切换,bin/plugins 保持稳定链接。
svc_release_switch() {
local rid="$1"
local rel="$ENV_DIR/releases/$rid"
[ -d "$rel/bin" ] || { c_err "[$ENV_NAME] 版本目录缺少 bin:$rel"; return 1; }
ln -s "$rid" "$ENV_DIR/current.new"
mv -Tf "$ENV_DIR/current.new" "$ENV_DIR/current"
ln -sfn "current/bin" "$ENV_DIR/bin"
ln -sfn "current/plugins" "$ENV_DIR/plugins"
if [ -d "$rel/htdocs" ]; then
ln -sfn "current/htdocs" "$ENV_DIR/htdocs"
fi
}
svc_release_stage() { # SRC BIN PLUGINS → 打印 rid
local srcbin="$1" srcplug="$2"
local digest sha rid dest
digest="$(sha256sum "$srcbin" | awk '{print substr($1,1,12)}')"
sha="$(printf '%s' "${SRC_SHA:-inplace}" | cut -c1-12)"
rid="${SRC_KINDSRC:-inplace}-${sha}-${digest}"
dest="$ENV_DIR/releases/$rid"
if [ ! -d "$dest/bin" ]; then
rm -rf "${dest}.partial"
mkdir -p "${dest}.partial/bin" "${dest}.partial/plugins"
cp -f "$srcbin" "${dest}.partial/bin/agent-system" || return 1
chmod 755 "${dest}.partial/bin/agent-system"
if [ -d "$srcplug" ]; then
cp -a "$srcplug/." "${dest}.partial/plugins/" || return 1
fi
python3 - "${dest}.partial/release.json" "$rid" "${SRC_LABEL:-}" "$digest" <<'PY'
import json, sys
path, rid, label, digest = sys.argv[1:]
with open(path, "w", encoding="utf-8") as f:
json.dump({"rid": rid, "source": label, "bin_digest12": digest}, f)
f.write("\n")
PY
mv "${dest}.partial" "$dest"
fi
printf '%s' "$rid"
}
svc_release_rollback() {
local from
from="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("from_rid",""))' "$ENV_DIR/deploy-journal.json" 2>/dev/null || true)"
svc_journal_write rolling_back || true
if [ -z "$from" ] || [ ! -d "$ENV_DIR/releases/$from/bin" ]; then
svc_journal_write aborted || true
c_err "[$ENV_NAME] 无法回滚:没有上一版本"
return 3
fi
svc_release_switch "$from" || { svc_journal_write aborted || true; return 3; }
local was
was="$(python3 -c 'import json,sys; print("1" if json.load(open(sys.argv[1])).get("was_running") else "0")' "$ENV_DIR/deploy-journal.json")"
if [ "$was" = 1 ]; then
if agent_do_start; then
svc_journal_write rolled_back
return 0
fi
svc_journal_write aborted || true
c_err "[$ENV_NAME] 回滚后的旧版本也未能启动,期望态保持"
return 3
fi
svc_journal_write rolled_back
return 0
}
svc_reconcile_journal() {
[ -f "$ENV_DIR/deploy-journal.json" ] || return 0
local phase
phase="$(svc_journal_phase)"
case "$phase" in
""|committed|rolled_back|aborted) return 0 ;;
esac
c_warn "[$ENV_NAME] 发现未完成的切换(phase=$phase),回滚到上一版本"
svc_release_rollback
}
install_artifacts() { # SRCBIN SRCPLUGINS:写入不可变版本目录并原子切换 current。
local srcbin="$1" srcplug="$2"
# 覆盖前必须无本服务占口进程——否则留下 exe=(deleted) 孤儿,后续 start 误报非本服务。
if ! reclaim_own_port; then
c_err "[$ENV_NAME] 安装前端口仍被占用,拒绝覆盖二进制(避免 exe deleted 孤儿)"
return 1
fi
mkdir -p "$ENV_BIN_DIR"
rm -rf "$ENV_DIR/plugins"
cp -f "$srcbin" "$ENV_BIN" || return 1
if [ -d "$srcplug" ]; then
cp -r "$srcplug" "$ENV_DIR/plugins" || return 1
else
c_warn "[$ENV_NAME] 源插件目录缺失:$srcplug(按空插件集部署)"
mkdir -p "$ENV_DIR/plugins"
if [ -d "$ENV_BIN_DIR" ] && [ ! -L "$ENV_BIN_DIR" ] && [ -x "$ENV_BIN" ]; then
local old_digest old_rid
old_digest="$(sha256sum "$ENV_BIN" | awk '{print substr($1,1,12)}')"
old_rid="inplace-${old_digest}"
if [ ! -d "$ENV_DIR/releases/$old_rid/bin" ]; then
mkdir -p "$ENV_DIR/releases/$old_rid"
mv "$ENV_BIN_DIR" "$ENV_DIR/releases/$old_rid/bin"
if [ -d "$ENV_DIR/plugins" ] && [ ! -L "$ENV_DIR/plugins" ]; then
mv "$ENV_DIR/plugins" "$ENV_DIR/releases/$old_rid/plugins"
else
mkdir -p "$ENV_DIR/releases/$old_rid/plugins"
fi
fi
SVC_FROM_RID="$old_rid"
svc_release_switch "$old_rid"
elif [ -L "$ENV_DIR/current" ]; then
SVC_FROM_RID="$(readlink "$ENV_DIR/current")"
fi
c_ok "[$ENV_NAME] 产物就位:$ENV_BIN(plugins:$ENV_DIR/plugins)"
local rid
rid="$(svc_release_stage "$srcbin" "$srcplug")" || return 1
SVC_TO_RID="$rid"
svc_journal_write switched || return 1
svc_release_switch "$rid" || return 1
c_ok "[$ENV_NAME] 产物就位:$ENV_BIN(release $rid)"
return 0
}
# ---- U2:不可变版本目录(releases/)+ current 原子指针 + 切换 journal + 失败回退 ----
@@ -2754,11 +3108,8 @@ agent_do_start() {
fi
# 9>&-:显式关闭锁 fd——服务进程绝不能继承 .start.lock(长驻持有会令后续
# 生命周期操作在锁上自锁死等;2026-10-11 自恢复任务实测发现)。
if [ "${#guard_env[@]}" -gt 0 ];then
nohup env "${guard_env[@]}" "$ENV_BIN" serve $exec_args >> "$ENV_DIR/svc.log" 2>&1 9>&- &
else
nohup "$ENV_BIN" serve $exec_args >> "$ENV_DIR/svc.log" 2>&1 9>&- &
fi
svc_write_serve_env
svc_launch_serve "$exec_args"
echo $! > "$ENV_PID"
# 健康探测:取 exec 里的 --addr(复用持久化配置时可能与当前派生值不同)。
local probe="${ENV_ADDR}"
@@ -2771,6 +3122,16 @@ agent_do_start() {
if curl -s -o /dev/null -m 2 "http://$phost:${probe##*:}/" 2>/dev/null; then
# 健康通过才写 svc.env——启动失败不再用新参数覆盖上次的可用配置
#(2026-10-05 dev 事故:失败后坏 exec= 持久化,裸 start 复用导致二次失败)。
# fresh:库由本次启动建立后再 ensure,升为 established。失败则停掉进程,
# 不把半套身份登记成已启动。
if [ "$(svc_db_marker 2>/dev/null || true)" = fresh ]; then
if ! svc_db_identity_ensure; then
kill "$(read_pid)" 2>/dev/null || true
c_err "[$ENV_NAME] 启动后建立 DB 身份失败,已停进程"
hist_event start fail "$(read_pid)" "fresh ensure 失败"
return 1
fi
fi
record_state
svc_mark_autostart
c_ok "[$ENV_NAME] 已启动 pid=$(read_pid) → http://$phost:${probe##*:}(来源 ${SVC_SOURCE:-?},日志:$ENV_DIR/svc.log)"
@@ -4007,7 +4368,6 @@ do_stop() {
return 0
}
# svc_wait_port_released:停机后等子进程释放端口(最多 3s)。桩/组合产物的 http
# 子进程经 ppid-watch 自杀(≤0.5s 语义),慢机上略晚;不等会让紧跟的 start 把
# 已无父的孤儿占口判成「非本服务」拒启(U2 回退实测,CI 慢机 run 3774)。
@@ -4021,19 +4381,21 @@ svc_wait_port_released() {
return 0
}
# do_stop_quiet:deploy 前置——未运行时静默返回(同时清除期望态:
# deploy 尾端的 start 成功会重新置位;失败则保持停止等人工介入)。
# do_stop_quiet [--keep-desired]:deploy 前置。切换事务必须保留期望态,
# 失败回滚后仍按原期望拉起。显式 stop 仍清除期望态。未运行时静默返回。
do_stop_quiet() {
local keep=0
[ "${1:-}" = --keep-desired ] && keep=1
local pid; pid="$(read_pid)"
if [ -z "$pid" ]; then
reclaim_own_port || true
svc_clear_autostart
[ "$keep" = 1 ] || svc_clear_autostart
return 0
fi
if ! pid_alive "$pid"; then
rm -f "$ENV_PID"
reclaim_own_port || true
svc_clear_autostart
[ "$keep" = 1 ] || svc_clear_autostart
return 0
fi
case "$SVC_KIND" in
@@ -4042,7 +4404,7 @@ do_stop_quiet() {
c_info "[$ENV_NAME] 停止旧服务 pid=$pid(SIGTERM,等待优雅停机)"
proc_stop "$pid"
rm -f "$ENV_PID"
svc_clear_autostart
[ "$keep" = 1 ] || svc_clear_autostart
svc_wait_port_released
c_ok "[$ENV_NAME] 已停止"
hist_event stop ok "$pid" "SIGTERM"
@@ -4710,29 +5072,8 @@ cmd_update() {
esac
c_info "[$name] 更新:$from → $tov($deploy_from)"
hist_event update begin "$(read_pid)" "from=$from to=$tov"
CLI_FROM="$deploy_from"
svc_lock wait
# U1:批量更新与单实例生命周期同入口——锁内 manifest 迁移(领养/身份治理;
# 2026-10-12 rdm1 实况:update 直入 agent_do_deploy 绕过迁移,多次更新后仍无
# manifest)。归属冲突按该服务更新失败计,其余服务继续。
if ! svc_manifest_migrate; then
hist_event update fail "$(read_pid)" "from=$from to=$tov(manifest 迁移中止:身份治理冲突)"
c_err "[$name] 更新失败(manifest 迁移中止),其余服务继续"
svc_unlock
CLI_FROM=""
failed=$((failed + 1))
continue
fi
# U2:批内同样处理中断切换(deploy 可跨越 blocked)。
if ! svc_deploy_journal_resolve deploy; then
hist_event update fail "$(read_pid)" "from=$from to=$tov(中断切换处置失败)"
c_err "[$name] 更新失败(deploy.journal 处置),其余服务继续"
svc_unlock
CLI_FROM=""
failed=$((failed + 1))
continue
fi
if agent_do_deploy; then
# 每个服务独立子进程、清空环境,避免批内 AGENT_SYSTEM_* 串实例。
if svc_lifecycle_child deploy "$name" --from "$deploy_from"; then
hist_event update ok "$(read_pid)" "from=$from to=$tov"
c_ok "[$name] 已更新:$from → $tov"
updated=$((updated + 1))
@@ -4741,8 +5082,6 @@ cmd_update() {
c_err "[$name] 更新失败($from → $tov),其余服务继续"
failed=$((failed + 1))
fi
svc_unlock
CLI_FROM=""
done
c_info "更新汇总:成功 $updated / 失败 $failed / 跳过 $skipped / 其余最新或不适用"
[ "$failed" -eq 0 ] && return 0 || return 1
@@ -4801,25 +5140,21 @@ EOF
[ "$(env_kv_of "$n" autostart)" = "1" ] || { skipped=$((skipped + 1)); continue; }
local st; st="$(svc_pid_state "$n")"
if [ "${st%% *}" = alive ]; then skipped=$((skipped + 1)); continue; fi
env_config "$n"
if ! svc_lock -n; then
c_warn "[$n] 生命周期锁被占用(deploy/手动操作进行中),本轮跳过"
skipped=$((skipped + 1)); continue
fi
# U2:拉起前处置中断的切换(switched → 回退上一完整版本再按期望态启动)。
if ! svc_deploy_journal_resolve recover; then
c_warn "[$n] 存在阻断的切换记录,本轮跳过(见 deploy.journal)"
failed=$((failed + 1)); rc=1; svc_unlock; continue
fi
c_info "[$n] recover:autostart=1 且进程已死,重新拉起(复用持久化配置)"
if do_start; then
# 子进程自己抢锁;锁被占用时 __lifecycle 以 75 退出,本轮记跳过。
local child_rc=0
svc_lifecycle_child recover "$n" || child_rc=$?
env_config "$n"
if [ "$child_rc" -eq 0 ]; then
hist_event recover ok "$(read_pid)" "autostart revival"
recovered=$((recovered + 1))
elif [ "$child_rc" -eq 75 ]; then
c_warn "[$n] 生命周期锁被占用(deploy/手动操作进行中),本轮跳过"
skipped=$((skipped + 1))
else
hist_event recover fail "" "autostart revival 失败(见 start fail 事件与 svc.log)"
failed=$((failed + 1)); rc=1
fi
svc_unlock
done
if [ "$recovered" -gt 0 ] || [ "$failed" -gt 0 ]; then
c_event "recover 汇总:拉回 $recovered / 失败 $failed / 无需动作 $skipped"
@@ -4945,7 +5280,52 @@ EOF
# ---- main ----
# svc_lifecycle_main ACTION NAME [--from SPEC] [--quiet]
# 锁内:迁移 → 身份策略(在各动作里)→ 动作 → 解锁。供 update/recover 子进程使用。
svc_lifecycle_main() {
local action="${1:-}" name="${2:-}"
[ -n "$action" ] && [ -n "$name" ] || { c_err "__lifecycle 需要动作与服务名"; exit 2; }
shift 2
valid_env "$name" || exit 2
while [ $# -gt 0 ]; do
case "$1" in
--from) CLI_FROM="$2"; shift 2 ;;
--quiet) CLI_QUIET=1; shift ;;
*) c_err "__lifecycle 未知参数:$1"; exit 2 ;;
esac
done
env_config "$name"
if ! svc_lock -n; then
exit 75
fi
case "$action" in
deploy|recover|start) svc_manifest_migrate || { svc_unlock; exit 1; } ;;
esac
# 对账必须发生在干净子进程里:父进程的调用方环境不得进入部署。
case "$action" in
deploy|recover) svc_deploy_journal_resolve || { svc_unlock; exit 1; } ;;
esac
local rc=0
case "$action" in
deploy)
case "$SVC_KIND" in
agent) agent_do_deploy || rc=$? ;;
static) static_do_deploy && static_do_start || rc=$? ;;
*) c_err "[$name] 该类型不支持 deploy"; rc=2 ;;
esac
;;
recover|start) do_start || rc=$? ;;
*) c_err "未知生命周期动作:$action"; rc=2 ;;
esac
svc_unlock
exit "$rc"
}
main() {
if [ "${1:-}" = "__lifecycle" ]; then
shift
svc_lifecycle_main "$@"
fi
[ $# -ge 1 ] || { usage_short; exit 2; }
# 预扫 --config/--env(FILE 与 =FILE 两种写法,可出现在任意位置)。
local svc_envfile="" pre=()
@@ -4992,7 +5372,7 @@ main() {
# 动作词可省略(省略 = deploy);flags 可直接跟随服务名。
local action="deploy"
case "${1:-}" in
deploy|stop|start|restart|status|logs|db|config|init|user|backup|backups) action="$1"; shift ;;
deploy|stop|start|restart|status|logs|db|config|init|adopt|user|backup|backups) action="$1"; shift ;;
esac
if [ "$action" = logs ] && [ "${1:-}" != "" ] && [[ "$1" =~ ^[0-9]+$ ]]; then
CLI_LOGS_N="$1"; shift
@@ -5016,6 +5396,15 @@ main() {
esac
exit 0
fi
local adopt_confirm=""
if [ "$action" = adopt ]; then
while [ $# -gt 0 ]; do
case "$1" in
--confirm) adopt_confirm="$2"; shift 2 ;;
*) c_err "adopt 用法:scripts/svc.sh $ENV_NAME adopt --confirm <instance_id>"; exit 2 ;;
esac
done
fi
parse_args "$@"
if { [ -n "$CLI_FROM" ] || [ -n "$CLI_PICK" ] || [ "$CLI_DRYRUN" = 1 ] || [ -n "$CLI_DIR" ] || [ "$CLI_ALLOW_SOURCE_SWITCH" = 1 ]; } && [ "$action" != deploy ]; then
c_err "--from/--pick/--dry-run/--dir/--allow-source-switch 仅用于 deploy 动作"
@@ -5025,12 +5414,12 @@ main() {
# 生命周期动作持 .start.lock(wait 模式;进程退出自动释放)——deploy 全程含
# 构建窗口,防 recover/并发 start 在停机-换产物间隙抢跑。只读动作不加锁。
case "$action" in
deploy|stop|start|restart|init) svc_lock wait || { c_err "[$ENV_NAME] 无法获取生命周期锁"; exit 1; } ;;
deploy|stop|start|restart|init|adopt) svc_lock wait || { c_err "[$ENV_NAME] 无法获取生命周期锁"; exit 1; } ;;
esac
# U1 身份:锁内一次性迁移(幂等;含既有数据领养 marker)。归属冲突(rc=4)中止;
# 其余失败保持 legacy 治理继续运行。
case "$action" in
start|restart|deploy|init) svc_manifest_migrate || exit 1 ;;
start|restart|deploy|init|adopt) svc_manifest_migrate || exit 1 ;;
esac
# U2:入口处理中断的切换(deploy.journal)。blocked 拒绝除 deploy 外的动作
#(显式换制品即解除路径);start/restart 触发回退判定。
@@ -5048,22 +5437,59 @@ main() {
start) do_start ;;
restart) do_restart ;;
init)
# 显式建立/查看身份(manifest+数据 marker+DB marker)。迁移钩子已先行:有
# svc.env 无 manifest 时 migrate 已建好,此处只报告;无 svc.env 引导走首次 start。
# 只用于 fresh/pending 的显式建立。established 缺行不得用 init 补,走 adopt。
command -v python3 >/dev/null 2>&1 && [ -f "$MANIFEST_PY" ] || {
c_err "[$ENV_NAME] init 需要 python3 与 scripts/svc_manifest.py"; exit 2; }
if [ -f "$ENV_DIR/manifest.json" ]; then
c_ok "[$ENV_NAME] 已在 manifest 治理下(init 幂等):"
case "$(svc_db_marker)" in
established)
c_err "[$ENV_NAME] 身份已建立。缺行或不匹配时先调查,再 scripts/svc.sh $ENV_NAME adopt --confirm <instance_id>"
exit 1 ;;
esac
c_ok "[$ENV_NAME] 已在 manifest 治理下(init 补齐 fresh/pending):"
elif [ -f "$ENV_DIR/svc.env" ]; then
python3 "$MANIFEST_PY" "$ENV_DIR" init \
|| { c_err "[$ENV_NAME] init 失败(数据根冲突等,见上)"; exit 1; }
else
c_err "[$ENV_NAME] 尚无 svc.env:先用 start 完成首次登记;init 用于显式化已有实例"; exit 2
fi
# I3:尽力解析 DB 后 Ensure(sqlite 无需 DSN;pg 需绑定已登记)。
resolve_db_config 2>/dev/null || true
svc_db_identity_ensure || { c_err "[$ENV_NAME] DB 身份 ensure 失败"; exit 1; }
python3 "$MANIFEST_PY" "$ENV_DIR" status ;;
adopt)
# 显式重审已建立实例的 DB 身份。先打印库大小、行数、最近写入,再要求 --confirm。
[ -n "$adopt_confirm" ] || { c_err "adopt 必须 --confirm <instance_id>"; exit 2; }
[ -f "$ENV_DIR/manifest.json" ] || { c_err "[$ENV_NAME] 无 manifest,不能 adopt"; exit 1; }
local iid
iid="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["instance_id"])' "$ENV_DIR/manifest.json")"
[ "$adopt_confirm" = "$iid" ] || { c_err "[$ENV_NAME] --confirm 与 instance_id 不符($iid)"; exit 1; }
resolve_db_config 2>/dev/null || true
python3 - "$ENV_DIR" <<'PY'
import json, os, sqlite3, sys, time
env_dir = sys.argv[1]
m = json.load(open(os.path.join(env_dir, "manifest.json"), encoding="utf-8"))
data = (m.get("identity") or {}).get("data_root") or ""
db = os.path.join(data, "agent-system.db")
print("adopt 证据 instance_id=%s" % m["instance_id"])
if not os.path.isfile(db):
print("库文件不存在:%s" % db)
sys.exit(0)
st = os.stat(db)
print("库大小=%d 字节 最近写入=%s" % (st.st_size, time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(st.st_mtime))))
con = sqlite3.connect(db)
try:
for table in ("agent_system_identity", "conversations", "users"):
try:
n = con.execute("SELECT COUNT(*) FROM %s" % table).fetchone()[0]
print("%s 行数=%s" % (table, n))
except sqlite3.Error:
print("%s 行数=(无此表)" % table)
finally:
con.close()
PY
svc_db_marker_set pending || exit 1
svc_db_identity_ensure || { c_err "[$ENV_NAME] adopt ensure 失败"; exit 1; }
c_ok "[$ENV_NAME] adopt 完成,db_marker=established" ;;
status) env_status_detail ;;
logs) [ -f "$ENV_LOG" ] && tail -n "${CLI_LOGS_N:-50}" -f "$ENV_LOG" || c_warn "[$ENV_NAME] 暂无日志" ;;
esac
+6 -1
View File
@@ -177,7 +177,12 @@ def build(env_dir, kv, allow_create):
"service_group": name,
"created_at": now_iso(),
"desired": "running" if kv.get("autostart") == "1" else "stopped",
"identity": {"data_root": data_root, "db_mode": db_mode, "db_ref": db_ref},
"identity": {
"data_root": data_root,
"db_mode": db_mode,
"db_ref": db_ref,
"db_marker": "fresh",
},
"kv": {k: v for k, v in kv.items() if k != "autostart"},
}
save(env_dir, m)
@@ -0,0 +1,48 @@
"""真实 Dev/RDM 组合门禁。由 make test-lifecycle 调用。
缺 PostgreSQL、RDM 源码或制品时直接失败,不静默跳过。
RDM 登录依赖账户 CLI 公共化(账户方案 U7/U5),当前标 NOT_RUN,本套件因此不能报完成。
"""
import os
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[3]
PIN = Path(__file__).resolve().parent / "rdm.pin"
def main():
missing = []
if not os.environ.get("AGENT_PG_TEST_DSN"):
missing.append("AGENT_PG_TEST_DSN")
if not os.environ.get("SVC_RDM_SRC_DIR") or not Path(os.environ.get("SVC_RDM_SRC_DIR", "")).is_dir():
missing.append("SVC_RDM_SRC_DIR")
if not (ROOT / "bin" / "agent-system").is_file() and not os.environ.get("SVC_DEV_BIN"):
missing.append("Dev 制品(make build 或 SVC_DEV_BIN)")
print("组合清单")
print("pin", PIN.read_text().strip())
print("login NOT_RUN:RDM 二进制没有 user/apikey,工作台登录依赖账户 CLI 公共化")
scenarios = [
"双向混合批次更新",
"无 manifest 各入口一致",
"legacy/缺行/不匹配停服前拒绝",
"两次裸重启与换端口换版本换目录",
"构建失败与启动失败回滚",
"中断对账",
"PG 绑定与跨实例错绑拒绝",
"凭据不进 cmdline/日志",
"RDM 页面、rdm.ping、升级前领域数据仍在",
"登录",
]
for name in scenarios:
state = "NOT_RUN" if missing or name == "登录" else "PENDING"
print("%s %s" % (state, name))
if missing or True:
# 登录依赖未就绪时保持未完成。缺环境同样失败。
print("test-lifecycle 未完成。缺:%s" % (", ".join(missing) if missing else "登录入口"))
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+4
View File
@@ -0,0 +1,4 @@
# RDM 制品钉扎。三行都是 40 位 hex。升级走 PR,不在 CI 里执行候选文件。
sha=68de2aa41022c8a77206ea30c2164bec33632ee3
prev_sha=00ae1494f5b45ae0b4ed955ee82debdfe830a1c3
legacy_sha=72bb1fec79d64c2af5e49416640712320816c3b4
+43
View File
@@ -0,0 +1,43 @@
"""rdm.pin 格式:只接受固定 SHA,拒绝可执行内容。"""
import re
import unittest
from pathlib import Path
PIN = Path(__file__).resolve().parents[1] / "tests" / "integration" / "lifecycle" / "rdm.pin"
HEX40 = re.compile(r"^[0-9a-f]{40}$")
def load_pin(text):
out = {}
for line in text.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if "=" not in line:
raise ValueError("bad pin line")
key, value = line.split("=", 1)
if key not in ("sha", "prev_sha", "legacy_sha"):
raise ValueError("unknown pin key")
if not HEX40.match(value):
raise ValueError("sha must be 40 hex")
out[key] = value
if set(out) != {"sha", "prev_sha", "legacy_sha"}:
raise ValueError("pin keys incomplete")
return out
class RdmPinTests(unittest.TestCase):
def test_pin_is_three_fixed_shas(self):
doc = load_pin(PIN.read_text())
self.assertEqual(len(doc["sha"]), 40)
self.assertNotEqual(doc["sha"], doc["legacy_sha"])
def test_rejects_url_and_short_sha(self):
with self.assertRaises(ValueError):
load_pin("sha=https://example.invalid/repo.git\nprev_sha=" + "a" * 40 + "\nlegacy_sha=" + "b" * 40 + "\n")
with self.assertRaises(ValueError):
load_pin("sha=abc\nprev_sha=" + "a" * 40 + "\nlegacy_sha=" + "b" * 40 + "\n")
if __name__ == "__main__":
unittest.main()
+4
View File
@@ -28,6 +28,10 @@ trace_dir="${SVC_STUB_TRACE:-$PWD/.svc-stub-trace}"
mkdir -p "$trace_dir"
case "${1:-}" in
version)
if [ "${2:-}" = "--json" ]; then
printf '%s\n' '{"protocol":1,"product":"agent-system","version":"stub","identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},"cli":{"unknownSubcommand":"reject"}}'
exit 0
fi
echo "stub-agent-system 1.0"
exit 0 ;;
config)
+106
View File
@@ -0,0 +1,106 @@
"""部署切换事务:构建失败不动旧服务,schema 不兼容在停服前拒绝,中断后对账回滚。"""
import importlib.util
import json
import os
import unittest
from pathlib import Path
SOURCE = Path(__file__).resolve().parents[1]
_spec = importlib.util.spec_from_file_location(
"tsi", SOURCE / "tests" / "test_svc_identity.py")
tsi = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(tsi)
def caps(target, floor):
return (
'{"protocol":1,"product":"agent-system","version":"stub",'
'"identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},'
'"cli":{"unknownSubcommand":"reject"},'
'"schema":{"target":%d,"compatFloor":%d}}\n' % (target, floor)
)
class DeployTxnTests(unittest.TestCase):
def setUp(self):
self.fx = tsi.IdentityTests("test_T1_bare_restart_stable_identity")
self.fx.setUp()
def tearDown(self):
self.fx.tearDown()
self.fx.doCleanups()
def test_schema_incompatible_refuses_before_stop(self):
fx = self.fx
old = fx.root / "stub-old"
new = fx.root / "stub-new"
text = (SOURCE / "tests" / "test_svc_recover.py").read_text()
# 复用桩的 identity/serve,只替换 version --json。
old.write_text(text[text.find("STUB = r'''") and 0:])
# 直接基于已写入的 recover 桩文件内容不够稳。用 fx.stub 再替换 JSON。
base = fx.stub.read_text()
marker = '\'{"protocol":1,"product":"agent-system","version":"stub","identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},"cli":{"unknownSubcommand":"reject"}}\''
old.write_text(base.replace(marker, "'" + caps(1, 0).strip() + "'"))
new.write_text(base.replace(marker, "'" + caps(5, 5).strip() + "'"))
old.chmod(0o755)
new.chmod(0o755)
fx.stub.write_text(old.read_text())
fx.stub.chmod(0o755)
fx.seed_agent("txn", 9471)
fx.start_ok("txn")
pid = fx.pid_of("txn")
script = fx.root / "scripts" / "svc.sh"
probe = fx.svc(
"txn", "status", check=False)
self.assertEqual(probe.returncode, 0, probe.stdout)
out = __import__("subprocess").run(
["bash", "-c",
'source "%s" >/dev/null 2>&1; ENV_NAME=txn; ENV_DIR="%s"; ENV_BIN="%s"; '
'if svc_schema_compatible "%s" "%s"; then echo OK; else echo NO; fi'
% (script, fx.runtime("txn"), fx.runtime("txn") / "bin" / "agent-system",
fx.runtime("txn") / "bin" / "agent-system", new)],
cwd=fx.root, env=fx.env, text=True,
stdout=__import__("subprocess").PIPE,
stderr=__import__("subprocess").STDOUT, timeout=30)
self.assertIn("NO", out.stdout, out.stdout)
self.assertEqual(fx.pid_of("txn"), pid)
fx.svc("txn", "stop")
def test_forward_only_requires_confirm_and_backup(self):
fx = self.fx
fx.seed_agent("fwd", 9472)
fx.start_ok("fwd")
iid = json.loads((fx.runtime("fwd") / "manifest.json").read_text())["instance_id"]
base = fx.stub.read_text()
marker = '\'{"protocol":1,"product":"agent-system","version":"stub","identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},"cli":{"unknownSubcommand":"reject"}}\''
installed = fx.root / "stub-installed"
candidate = fx.root / "stub-forward"
installed.write_text(base.replace(marker, "'" + caps(1, 0).strip() + "'"))
candidate.write_text(base.replace(marker, "'" + caps(5, 5).strip() + "'"))
installed.chmod(0o755)
candidate.chmod(0o755)
script = fx.root / "scripts" / "svc.sh"
envdir = fx.runtime("fwd")
def run(extra):
return __import__("subprocess").run(
["bash", "-c",
'source "%s" >/dev/null 2>&1; ENV_NAME=fwd; ENV_DIR="%s"; '
'ROOT="%s"; %s '
'if svc_schema_compatible "%s" "%s"; then echo OK; else echo NO; fi'
% (script, envdir, fx.root, extra, installed, candidate)],
cwd=fx.root, env=fx.env, text=True,
stdout=__import__("subprocess").PIPE,
stderr=__import__("subprocess").STDOUT, timeout=30).stdout
self.assertIn("NO", run(""))
self.assertIn("NO", run('CLI_MIGRATE_SCHEMA=1; CLI_CONFIRM="%s";' % iid))
backup = fx.root / "runtime-backups" / "fwd" / "20261012T000000Z"
backup.mkdir(parents=True)
(backup / "ok").write_text("snapshot\n")
self.assertIn("OK", run('CLI_MIGRATE_SCHEMA=1; CLI_CONFIRM="%s";' % iid))
fx.svc("fwd", "stop")
if __name__ == "__main__":
unittest.main()
+19 -23
View File
@@ -229,7 +229,12 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
" echo \"system identity: system identity missing: agent_system_identity 无行"
"(先 svc init / identity ensure,禁止静默建空身份)\" >&2\n"
" exit 1 ;;")
p.write_text(head + noid_case + tail2)
body = head + noid_case + tail2
body = body.replace(
'\'{"protocol":1,"product":"agent-system","version":"stub","identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},"cli":{"unknownSubcommand":"reject"}}\'',
'"not-json"',
)
p.write_text(body)
p.chmod(0o755)
return p
@@ -316,11 +321,8 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
覆盖→启动即拒,只剩混合状态)。"""
remote, old_sha, new_sha = self.make_rdm_repo(self.noid_stub())
self.seed_agent("rdx", 9451)
self.start_ok("rdx") # 领养:manifest + DB 行
self.svc("rdx", "stop")
self.install_noid_bin("rdx") # 磁盘产物退化为无 identity(事故现场)
self.start_ok("rdx") # 已建立身份,有能力制品在运行
self.set_kv("rdx", "source", "rdm@%s" % old_sha)
self.start_ok("rdx") # 校验跳过,照常运行
pid = self.pid_of("rdx")
bin_before = (self.runtime("rdx") / "bin" / "agent-system").read_bytes()
@@ -339,17 +341,12 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
self.svc("rdx", "stop")
def test_U1_update_ensures_missing_row_before_stop(self):
"""领养时未能补行的实例(旧产物无 identity):候选有能力时,update 在
停服前用候选补齐 DB 身份行——完整更新成功;serve 拿到的是本服务自己的
身份(非批内残留)。"""
"""已建立身份后删行:update 在停服前拒绝,不自动补行,旧进程保持。
补行只走 adopt --confirm。"""
old_sha, new_sha = self.make_src_repo()
self.seed_agent("rowx", 9452)
self.start_ok("rowx")
self.svc("rowx", "stop")
self.install_noid_bin("rowx")
self.set_kv("rowx", "source", "dev@%s" % old_sha)
self.start_ok("rowx")
# 模拟领养时未补行(rdm1 现状):清空 DB 身份行
db = Path(self.manifest("rowx")["identity"]["data_root"]) / "agent-system.db"
con = sqlite3.connect(str(db))
con.execute("DELETE FROM agent_system_identity")
@@ -357,15 +354,11 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
con.close()
self.clear_serve_record()
result = self.svc("update", "rowx")
self.assertIn("已更新", result.stdout)
self.assertEqual(self.kv("rowx", "source"), "dev@%s" % new_sha)
iid = self.manifest("rowx")["instance_id"]
self.assertEqual(self.db_identity_row("rowx"), [(iid, "sqlite")],
"候选应在停服前补齐身份行")
self.assertTrue(self.running("rowx"))
self.assertIn("AGENT_SYSTEM_INSTANCE_ID=%s" % iid, self.serve_record(),
"serve 只允许拿到本服务自己的身份")
result = self.svc("update", "rowx", check=False)
self.assertNotEqual(result.returncode, 0, result.stdout)
self.assertIn("候选无法验证已建立的 DB 身份", result.stdout)
self.assertTrue(self.running("rowx"), "缺行必须在停旧服务前拒绝")
self.assertEqual(self.db_identity_row("rowx"), [], "established 不得自动补行")
self.svc("rowx", "stop")
def test_U1_bare_start_missing_row_is_event_init_recovers(self):
@@ -387,7 +380,10 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
self.assertFalse(self.running("rowm"))
self.assertEqual(self.db_identity_row("rowm"), [], "缺行不得被 start 静默补建")
self.svc("rowm", "init") # 修复路径:显式 ensure
refused = self.svc("rowm", "init", check=False)
self.assertNotEqual(refused.returncode, 0, refused.stdout)
self.assertIn("adopt", refused.stdout)
self.svc("rowm", "adopt", "--confirm", iid)
self.assertEqual(self.db_identity_row("rowm"), [(iid, "sqlite")])
self.start_ok("rowm")
self.svc("rowm", "stop")
@@ -412,7 +408,7 @@ class IdentityTests(tsr.SvcAutoRecoverTests):
"报错文本含 ensure 不得误报为有能力")
hang = self.root / "stub-hang"
hang.write_text(
'#!/usr/bin/env bash\ncase "${1:-}" in identity) sleep 30 ;; '
'#!/usr/bin/env bash\ncase "${1:-}" in version) sleep 30 ;; '
'*) exec "%s" "$@" ;; esac\n' % self.stub)
hang.chmod(0o755)
self.assertIn("rc=1", probe(hang), "探测挂死须有界返回无能力")
+54
View File
@@ -0,0 +1,54 @@
"""生命周期隔离:调用方的 AGENT_SYSTEM_* 不得进入 serve,serve.env 为 0600。"""
import importlib.util
import json
import shutil
from pathlib import Path
SOURCE = Path(__file__).resolve().parents[1]
_spec = importlib.util.spec_from_file_location("tsr", SOURCE / "tests" / "test_svc_recover.py")
tsr = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(tsr)
class LifecycleIsolationTests(tsr.SvcAutoRecoverTests):
def setUp(self):
super().setUp()
shutil.copy2(SOURCE / "scripts" / "svc_manifest.py",
self.root / "scripts" / "svc_manifest.py")
def test_caller_identity_env_does_not_reach_serve(self):
self.seed_agent("iso", 9461)
env = dict(self.env)
env["AGENT_SYSTEM_DSN"] = "postgres://leak-user:leak-pass@127.0.0.1/leak"
env["AGENT_SYSTEM_DATA"] = "/tmp/leak-data"
env["AGENT_SYSTEM_INSTANCE_ID"] = "svc-foreign-leak"
self.svc("iso", "start", env=env)
pid = self.pid_of("iso")
self.assertIsNotNone(pid)
raw = Path("/proc/%d/environ" % pid).read_bytes()
self.assertNotIn(b"leak-pass", raw)
self.assertNotIn(b"/tmp/leak-data", raw)
self.assertNotIn(b"svc-foreign-leak", raw)
serve_env = self.runtime("iso") / "serve.env"
self.assertTrue(serve_env.is_file())
self.assertEqual(serve_env.stat().st_mode & 0o777, 0o600)
text = serve_env.read_text()
self.assertNotIn("AGENT_SYSTEM_", text)
self.assertNotIn("leak-pass", text)
self.svc("iso", "stop")
def test_two_bare_restarts_keep_identity(self):
self.seed_agent("iso2", 9462)
self.svc("iso2", "start")
manifest = json.loads((self.runtime("iso2") / "manifest.json").read_text())
iid = manifest["instance_id"]
data = manifest["identity"]["data_root"]
self.svc("iso2", "stop")
self.svc("iso2", "start")
self.svc("iso2", "stop")
self.svc("iso2", "start")
again = json.loads((self.runtime("iso2") / "manifest.json").read_text())
self.assertEqual(again["instance_id"], iid)
self.assertEqual(again["identity"]["data_root"], data)
self.assertEqual(again["identity"].get("db_marker"), "established")
self.svc("iso2", "stop")
+38 -2
View File
@@ -21,7 +21,9 @@ import shutil
import sqlite3
import subprocess
import tempfile
import time
import unittest
import urllib.error
import urllib.request
from pathlib import Path
@@ -134,7 +136,11 @@ class RealComboTests(tsr.SvcAutoRecoverTests):
"identity", "capabilities"],
capture_output=True, text=True, timeout=15)
self.assertEqual(cap.returncode, 0, cap.stderr)
self.assertIn("identity-capabilities: v1", cap.stdout)
doc = json.loads(cap.stdout)
self.assertEqual(doc["protocol"], 1)
self.assertEqual(doc["identity"]["open"], "existing-no-migrate")
self.assertIn("ensure", doc["identity"]["ops"])
self.assertIn("verify", doc["identity"]["ops"])
# 错 id verify 拒绝
bad = subprocess.run([str(self.runtime("rc1") / "bin" / "agent-system"),
"identity", "verify", "--instance-id",
@@ -215,10 +221,40 @@ class RealComboTests(tsr.SvcAutoRecoverTests):
命令行);错 id fail-closed。"""
rdm_bin = Path(REAL_RDM_BIN).resolve()
data = self.root / "pgdata"
data.mkdir()
plugins = data / "plugins"
plugins.mkdir(parents=True)
env = dict(os.environ)
env["AGENT_SYSTEM_DSN"] = os.environ["AGENT_PG_TEST_DSN"]
env["RDM_DISABLE_EXEC_TOOLS"] = "1"
env.pop("SVC_STUB_TRACE", None)
# 身份命令不建库、不迁移。先让 serve 把库迁到当前 schema,再 ensure/verify。
port = 9475
log = self.root / "pg-serve.log"
serve = subprocess.Popen(
[str(rdm_bin), "serve", "--addr", "127.0.0.1:%d" % port,
"--data", str(data), "--plugins", str(plugins), "--auth", "auto"],
env=env, stdout=log.open("w"), stderr=subprocess.STDOUT)
try:
up = False
for _ in range(50):
if serve.poll() is not None:
break
try:
urllib.request.urlopen("http://127.0.0.1:%d/" % port, timeout=1).close()
up = True
break
except urllib.error.HTTPError:
up = True
break
except OSError:
time.sleep(0.1)
self.assertTrue(up, "serve 未完成迁移:\n%s" % log.read_text(errors="replace")[-1500:])
finally:
serve.terminate()
try:
serve.wait(timeout=5)
except subprocess.TimeoutExpired:
serve.kill()
iid = "svc-realcombo-pg-%d" % os.getpid()
run = lambda *a: subprocess.run(
[str(rdm_bin), "identity", *a, "--instance-id", iid,
+4
View File
@@ -28,6 +28,10 @@ trace_dir="${SVC_STUB_TRACE:-$PWD/.svc-stub-trace}"
mkdir -p "$trace_dir"
case "${1:-}" in
version)
if [ "${2:-}" = "--json" ]; then
printf '%s\n' '{"protocol":1,"product":"agent-system","version":"stub","identity":{"ops":["ensure","verify"],"open":"existing-no-migrate","dsn":"env"},"cli":{"unknownSubcommand":"reject"}}'
exit 0
fi
echo "stub-agent-system 1.0"
exit 0 ;;
config)