diff --git a/.doc/ops/CURRENT.md b/.doc/ops/CURRENT.md index d2921728..0fe1b3d2 100644 --- a/.doc/ops/CURRENT.md +++ b/.doc/ops/CURRENT.md @@ -18,6 +18,7 @@ | owner | task | branch | worktree | mutex | status | |---|---|---|---|---|---| +| Cursor | nightly-advance-automation-2026-09-28 | feature/nightly-advance-automation-2026-09-28 | .worktrees/nightly-advance-automation-2026-09-28 | nightly-advance | in_progress | | Cursor | quality-gate-heading-mismatch-2026-09-28-r3 | fix/quality-gate-heading-mismatch-2026-09-28 | .worktrees/quality-gate-heading-mismatch-2026-09-28 | quality-gate-heading | in_progress | | Cursor | agent-reply-llm-narrative-2026-09-28 | feature/agent-reply-llm-narrative-2026-09-28 | .worktrees/agent-reply-llm-narrative-2026-09-28 | chat+coordinator | in_progress | | Cursor | test-user-doc-issues-2026-09-28 | fix/test-user-doc-issues-2026-09-28 | .worktrees/test-user-doc-issues-2026-09-28 | intent-write-verb | done | diff --git a/.doc/ops/short-term/2026-09-28-nightly-advance-automation.md b/.doc/ops/short-term/2026-09-28-nightly-advance-automation.md new file mode 100644 index 00000000..d29b4b21 --- /dev/null +++ b/.doc/ops/short-term/2026-09-28-nightly-advance-automation.md @@ -0,0 +1,49 @@ +# 夜间自动推进任务（nightly-advance-automation-2026-09-28） + +- **status**: in_progress +- **owner**: Cursor +- **mutex**: nightly-advance +- **base_dev_sha**: 650703c5c28d3f4c9fb51ba9022499524a91fe01 +- **write_scope**: scripts/nightly/、scripts/nightly-advance.sh、scripts/tests/test_nightly_advance.py、.doc/ops/short-term/2026-09-28-nightly-advance-automation.md +- **authorization_ref**: 用户 2026-09-28 会话（"每天晚上 3 点 10 分，推进项目.worktree 目录下的所有分支任务……"及后续清理语义修正）；Agents Window 不可用后用户同意本地 cron 替代方案 + +## 用户目标 + +每日 03:10 自动推进 `.worktrees/` 下所有活跃分支任务至彻底完成，合入 dev，验证无内容丢失后删除对应任务目录，更新 dev 服务至最新，产出每日夜间自动化报告。 + +## 用户裁决（AskQuestion 记录） + +1. **推进范围**：所有活跃任务（不分 owner；Cursor/pi/Codex/Cline）。 +2. **清理语义**（用户后续修正）：任务彻底完成、合入 dev、删除目录不丢内容且不影响系统 → 删除目录与分支；删除被拒时保留 cleanup=pending 并报告原因，禁止 `--force`/`-D`。 + +## 方案 + +- `scripts/nightly/plan.py`：读 cleanup report + CURRENT + registry，按依赖排序（integrate 候选 → 活跃任务 → 可清理项）。 +- `scripts/nightly/advance.py`：编排器。预检（daemon 健康、互斥锁、冻结 dev tip）→ 计划 → cursor-agent 无头推进（写走 `agentctl with`）→ daemon ff-only 合入 → dev tip 变化时 `env:dev` 租约 rebuild + `/health` `/ready` → 清理验证+普通删除 → 报告草稿落 run 目录。 +- `scripts/nightly-advance.sh`：cron 入口（PATH 注入，日志 `.tmp/nightly/entry-*.log`）。 +- cron：`10 3 * * *`（用户级 crontab，与现有 jev-service watchdog 同模式）。 +- run 目录：`.tmp/nightly//`（gitignored；`data/` root 属主不可写）。 +- 每日报告：夜间最后一棒 cursor-agent 日报任务把 run 目录草稿正式化并归档 `.doc/test-reports/`（agentctl 租约提交），按 test-reports-standard 命名 `YYYY-MM-DD-nightly-advance-test-report.md`。 + +## 硬边界 + +- 写操作一律 `agentctl with`；裸 git commit 被 pre-commit hook 拒绝。 +- 合入只走 `validate` + `queue enqueue`（daemon `--ff-only`）。 +- dev 服务更新走 `with --resource env:dev` + `env-service.sh dev rebuild`；不碰 all/test/main。 +- 清理只用 `agent-worktree.sh remove --delete-branch`（普通删除）；拒绝即保留 pending。 +- 不 push（用户未要求）；报告真实验证，无 stub 冒充。 + +## 验证 + +- 单元：`python3 -B -m unittest discover -s scripts/tests -p 'test_nightly_advance.py'`（9 用例：CURRENT 解析、排序、phase 分类、registry 回退、CLI cap、自排除）。 +- 真实链路：dry 模式（--skip-agent）端到端：预检锁、plan 13 项、清理拦截未合入分支、报告草稿落盘（run 20260928T063326Z）。 +- 无头执行器冒烟：cursor-agent --print ask 模式回 `NIGHTLY-OK`；写探测 `PROBE-WRITE-OK`。 +- 本报告任务自身从 plan 排除（防递归）。 + +## 五态（收口时更新） + +- review_state: pending +- integration_state: not_started +- deployment: not_required（工程工具变更，无需重建产品容器） +- cleanup: pending（本任务自身 worktree 由独立维护者复查后清理） +- remote_sync: not_requested diff --git a/scripts/nightly-advance.sh b/scripts/nightly-advance.sh new file mode 100755 index 00000000..0dc35502 --- /dev/null +++ b/scripts/nightly-advance.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# Nightly advance entrypoint (cron 10 3 * * *). +# Drives the full pipeline via scripts/nightly/advance.py and always leaves a +# machine-readable summary + report draft under data/nightly//. +# +# Environment: +# NIGHTLY_ADVANCE_REPO default /home/weichen/projects/Agent-System +# NIGHTLY_TASK_TIMEOUT per-task cursor-agent timeout seconds (default 3600) +# NIGHTLY_MAX_TASKS cap planned entries (0 = all) +# +# Never pushes, never promotes test/main, never force-removes worktrees. +set -euo pipefail + +REPO="${NIGHTLY_ADVANCE_REPO:-/home/weichen/projects/Agent-System}" +export PATH="/usr/bin:/usr/local/bin:/home/linuxbrew/.linuxbrew/bin:$PATH" + +LOG_ROOT="${REPO}/.tmp/nightly" +STAMP="$(date -u +%Y%m%dT%H%M%SZ)" +ENTRY_LOG="${LOG_ROOT}/entry-${STAMP}.log" +mkdir -p "${LOG_ROOT}" + +{ + echo "== nightly-advance ${STAMP} repo=${REPO}" + python3 -B "${REPO}/scripts/nightly/advance.py" \ + --repo "${REPO}" \ + --task-timeout "${NIGHTLY_TASK_TIMEOUT:-3600}" \ + --max-tasks "${NIGHTLY_MAX_TASKS:-0}" + RC=$? + echo "== done rc=${RC}" +} >> "${ENTRY_LOG}" 2>&1 + +exit "${RC:-0}" diff --git a/scripts/nightly/advance.py b/scripts/nightly/advance.py new file mode 100755 index 00000000..9d90527f --- /dev/null +++ b/scripts/nightly/advance.py @@ -0,0 +1,506 @@ +#!/usr/bin/env python3 +"""Nightly advance orchestrator. + +Executes the nightly pipeline against the main repo: + + Phase 0 preflight daemon health, reentry lock, dev tip freeze + Phase 1 plan scripts/nightly/plan.py ordering + Phase 2 advance per-task headless cursor-agent runs (bounded) + Phase 3 integrate daemon ff-only via queue enqueue (already done inside + phase 2 per task); wait for queue drain + Phase 4 dev-service rebuild dev env when tip moved; /health /ready + Phase 5 cleanup verify + remove finished worktrees (normal remove + only; refusals leave cleanup=pending, never --force) + Phase 6 report .doc/test-reports/-nightly-advance-test-report.md + draft written to run dir (committed by report task) + +Every phase records structured state into the run dir (data/nightly//), +so a crashed run is inspectable and never double-drives: a live lock file +with a healthy heartbeat aborts the new invocation. + +Exit codes: 0 success, 1 partial failures, 2 usage, 3 fatal infrastructure. +""" + +from __future__ import annotations + +import argparse +import json +import os +import subprocess +import sys +import time +from datetime import datetime, timezone +from pathlib import Path + +DEFAULT_REPO = "/home/weichen/projects/Agent-System" +# data/ is root-owned (docker mounts); .tmp/ is gitignored and user-writable. +RUN_ROOT = ".tmp/nightly" +LOCK_REL = f"{RUN_ROOT}/.lock" +# Standing task identity for the perpetual automation (env leases survive the +# delivery task closeout). Override with NIGHTLY_TASK_ID if retasked. +STANDING_TASK = os.environ.get("NIGHTLY_TASK_ID", "nightly-advance-automation") +HEARTBEAT_STALE_SECONDS = 3600 +REPORT_SLUG = "nightly-advance" + +PROMPT_ADVANCE = """你是 Agent-System 夜间自动化执行器。任务：推进 worktree {worktree} 中的分支任务 {task_id} 至完成。 + +硬边界（违反即失败）： +1. 只在 {worktree} 内工作，不碰其他 worktree 或主目录 dev。 +2. 任何 git 写操作（add/commit）必须包在 `python3 ../../scripts/agentctl.py with --task {task_id} --resource path:<路径> --resource git:index -- bash -lc '<命令>'` 内。 +3. 读取该 worktree 下的任务文档（.doc/ops/short-term/ 或 .doc/audit/ 下与任务 id 匹配的文件）了解剩余工作与验收标准。 +4. 按任务文档推进实现/修复/文档工作；禁止 push、禁止晋升 test/main、禁止操作其他环境。 +5. 按验证契约选择验收：工程工具/文档类 → `python3 -B -m unittest discover -s scripts/tests -p 'test_*.py'`；产品代码改动 → 本 worktree 内可行的真实检查（go build / go test 相关包）。 +6. 完成后：任务状态五字段填入任务文档（review_state/integration_state=validated_cddidate_ready/deployment/cleanup/remote_sync），不要自行 integrate。 +7. 所有命令真实执行并保留输出；禁止用 SKIP/stub 冒充通过。 + +完成时输出 JSON 摘要（最后一行）：{{"task_id":"{task_id}","result":"advanced|done|blocked|failed","summary":"<一句话>","evidence":"<相对路径>"}}""" + +PROMPT_INTEGRATE = """你是 Agent-System 夜间自动化执行器。任务：把集成候选 {task_id} 合入 dev（通过治理门禁）。 + +步骤： +1. 在 {worktree} 内检查 `git status --porcelain` 是否干净、`git rev-parse HEAD` 是否为冻结候选。 +2. 若树不干净：停止并输出 blocked（不要 stash/reset）。 +3. 若干净：构造候选收口——执行 `python3 ../../scripts/agentctl.py validate --task {task_id} -- python3 -B -m unittest discover -s scripts/tests -p 'test_*.py'`（治理类变更）；产品变更按任务文档已冻结的验收命令。 +4. validate 通过后：`python3 ../../scripts/agentctl.py queue enqueue --task {task_id} --expected-dev {expected_dev} --head $(git rev-parse HEAD)`。daemon 会 ff-only 合入。 +5. 轮询 `queue list` 至空或该 task 消失（最多 60s）。 +6. 禁止手动 git merge dev、禁止 push、禁止动 test/main。 + +陈旧候选恢复（仅当 validate 因 TASK_CLOSED 拒绝，或候选第一父不等于当前 dev）： +- 先判断功能内容是否已经在 dev 上（`git merge-base --is-ancestor <候选head或其功能分支> dev`，或对比 dev log）。内容已在 dev → 输出 result=merged, summary 说明由哪个提交带入，不重建。 +- 内容不在 dev 且任务已 cancelled/done → 输出 blocked 并说明需要人工重开任务；禁止复活关闭的任务。 +- 内容不在 dev 且任务仍 active 但基线前移 → 在 worktree 内用 `python3 ../../scripts/agentctl.py integrate prepare --task <基础功能task_id> --expected-dev <当前dev> --integrate-id {task_id}-r` 从新 dev 重建候选（N 为现有最大轮次），对返回的新 worktree 重复步骤 1-5。 + +完成时输出 JSON 摘要（最后一行）：{{"task_id":"{task_id}","result":"enqueued|merged|blocked|failed","summary":"<一句话>","new_dev_tip":""}}""" + +PROMPT_REPORT = """你是 Agent-System 夜间自动化日报执行器。任务：把 {date} 夜间运行的报告草稿正式化并归档。 + +步骤： +1. 读 {run_dir}/report-draft.md、advance.json、env.json、cleanup.json、state.json。 +2. 在当前 worktree（报告任务工作区）把草稿改写为正式报告 .doc/test-reports/{date}-nightly-advance-test-report.md： + - 按 test-reports-standard 模板补全七大板块；引用 run 目录真实数据（不得虚构）。 + - 每个任务的 result/耗时/证据日志相对路径如实填入「实际测试结果与判定」。 + - blocked/failed 任务在「测试结论与后续」列出后续建议（人工裁决点）。 +3. 用治理租约提交（禁止裸 git）： + python3 ../../scripts/agentctl.py with --task {report_task_id} --resource path:.doc --resource git:index -- bash -lc 'git add .doc/test-reports/{date}-nightly-advance-test-report.md && git commit -m "docs(test): nightly advance report {date}"' +4. 不 integrate、不 push；提交留在报告任务分支即可。 + +完成时输出 JSON 摘要（最后一行）：{{"date":"{date}","result":"archived|failed","report":".doc/test-reports/{date}-nightly-advance-test-report.md","commit":""}}""" + + +def sh(args: list[str], cwd: Path, env: dict | None = None, timeout: int = 600) -> tuple[int, str]: + """Run a subprocess; returns (rc, combined output). Never raises.""" + e = dict(os.environ) + if env: + e.update(env) + try: + p = subprocess.run( + args, cwd=str(cwd), capture_output=True, text=True, timeout=timeout, env=e + ) + out = p.stdout + ("\n--stderr--\n" + p.stderr if p.stderr else "") + return p.returncode, out + except subprocess.TimeoutExpired as exc: + return 124, f"TIMEOUT after {timeout}s\n{(exc.stdout or '')[-800:]}" + except Exception as exc: # noqa: BLE001 — orchestrator must not die + return 125, f"ERROR {exc}" + + +def agentctl(repo: Path, *args: str, timeout: int = 600) -> tuple[int, dict | str]: + rc, out = sh([sys.executable, "-B", str(repo / "scripts/agentctl.py"), *args], repo, timeout=timeout) + clean = "\n".join(ln for ln in out.splitlines() if not ln.startswith("[notice]")) + try: + return rc, json.loads(clean.strip() or "{}") + except json.JSONDecodeError: + return rc, out + + +def write_json(path: Path, data) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8") + + +def now_utc() -> str: + return datetime.now(timezone.utc).isoformat(timespec="seconds") + + +# ---------------------------------------------------------------- phase 0 + +def preflight(repo: Path, run_dir: Path) -> tuple[bool, list[str], dict]: + """Reentry lock + daemon health + dev tip freeze. Returns (ok, errors, ctx).""" + errors: list[str] = [] + ctx: dict = {"started_at": now_utc()} + + # lock: atomic create; stale after HEARTBEAT_STALE_SECONDS without update + lock = repo / LOCK_REL + lock.parent.mkdir(parents=True, exist_ok=True) + try: + fd = os.open(lock, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644) + os.write(fd, json.dumps({"pid": os.getpid(), "started": ctx["started_at"]}).encode()) + os.close(fd) + except FileExistsError: + age = time.time() - lock.stat().st_mtime + if age > HEARTBEAT_STALE_SECONDS: + lock.unlink() + fd = os.open(lock, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644) + os.write(fd, json.dumps({"pid": os.getpid(), "took_over_after_stale": age}).encode()) + os.close(fd) + else: + return False, [f"another nightly run is active (lock age {age:.0f}s < {HEARTBEAT_STALE_SECONDS}s)"], ctx + + rc, daemon = agentctl(repo, "daemon", "status") + if rc != 0 or (isinstance(daemon, dict) and daemon.get("status") != "ok"): + return False, [f"daemon not healthy: {str(daemon)[:200]}"], ctx + ctx["daemon"] = "ok" + + rc, tip = sh(["git", "-C", str(repo), "rev-parse", "HEAD"], repo) + if rc != 0: + return False, ["cannot resolve dev tip"], ctx + ctx["dev_tip_start"] = tip.strip() + return True, errors, ctx + + +def touch_heartbeat(repo: Path) -> None: + lock = repo / LOCK_REL + try: + os.utime(lock, None) + except OSError: + pass + + +# ---------------------------------------------------------------- phase 1 + +def build_plan(repo: Path, run_dir: Path) -> dict: + # Locate plan.py next to this file first; fall back to /scripts so + # the orchestrator works both from a task worktree and from merged dev. + here = Path(__file__).resolve().parent + plan_py = here / "plan.py" + if not plan_py.exists(): + plan_py = repo / "scripts/nightly/plan.py" + out_file = run_dir / "plan.json" + rc, out = sh( + [sys.executable, "-B", str(plan_py), "--repo", str(repo), "--out", str(out_file)], + repo, + ) + if rc != 0 or not out_file.exists(): + return {"error": out[-500:], "entries": []} + return json.loads(out_file.read_text(encoding="utf-8")) + + +# ---------------------------------------------------------------- phase 2 + +def run_agent(prompt: str, worktree: Path, timeout_s: int, run_dir: Path, tag: str) -> dict: + """Headless cursor-agent run inside a worktree; full log kept in run_dir.""" + log = run_dir / f"agent-{tag}.log" + cmd = [ + "cursor-agent", "--print", "--trust", + "--workspace", str(worktree), + prompt, + ] + env = {"PATH": os.environ.get("PATH", "") + ":/usr/bin:/home/linuxbrew/.linuxbrew/bin"} + start = time.monotonic() + rc, out = sh(cmd, cwd=worktree, env=env, timeout=timeout_s) + dur = time.monotonic() - start + log.write_text(out, encoding="utf-8", errors="replace") + summary = {"rc": rc, "duration_s": round(dur, 1), "log": str(log)} + # parse last JSON line the agent was asked to emit + for line in reversed(out.strip().splitlines()): + line = line.strip().strip("`") + if line.startswith("{") and line.endswith("}"): + try: + summary["agent"] = json.loads(line) + break + except json.JSONDecodeError: + continue + return summary + + +# ---------------------------------------------------------------- phase 4 + +def dev_tip(repo: Path) -> str | None: + rc, out = sh(["git", "-C", str(repo), "rev-parse", "HEAD"], repo) + return out.strip() if rc == 0 else None + + +def update_dev_service(repo: Path, run_dir: Path) -> dict: + """Rebuild dev env only when dev tip moved; verify /health and /ready.""" + tip = dev_tip(repo) + result: dict = {"dev_tip": tip} + if tip is None: + result["status"] = "failed" + result["error"] = "cannot resolve dev tip" + return result + start = tip == run_dir_state(run_dir).get("dev_tip_start") + if start: + result["status"] = "not_required" + result["reason"] = "dev tip unchanged tonight" + return result + # env:dev write goes through agentctl with on the standing nightly task + # lease. The daemon itself never touches env:*, so we drive env-service.sh + # here. + rc, out = sh( + [sys.executable, "-B", str(repo / "scripts/agentctl.py"), "with", + "--task", STANDING_TASK, + "--resource", "env:dev", "--cwd", str(repo), + "--", str(repo / "scripts/env-service.sh"), "dev", "rebuild"], + repo, timeout=3600, + ) + log = run_dir / "env-rebuild.log" + log.write_text(out, encoding="utf-8", errors="replace") + result["rebuild_rc"] = rc + result["rebuild_log"] = str(log) + if rc != 0: + result["status"] = "failed" + return result + ok_h, body_h = http_get("http://127.0.0.1:8080/health") + ok_r, body_r = http_get("http://127.0.0.1:8080/ready") + result["health"] = {"ok": ok_h, "body": (body_h or "")[:300]} + result["ready"] = {"ok": ok_r, "body": (body_r or "")[:300]} + result["status"] = "ok" if (ok_h and ok_r) else "unhealthy" + return result + + +def http_get(url: str, retries: int = 6, delay: float = 10.0) -> tuple[bool, str | None]: + for i in range(retries): + rc, out = sh(["curl", "-fsS", "--max-time", "10", url], Path("/tmp"), timeout=30) + if rc == 0: + return True, out + time.sleep(delay if i else 2) + return False, None + + +def run_dir_state(run_dir: Path) -> dict: + f = run_dir / "state.json" + if f.exists(): + try: + return json.loads(f.read_text(encoding="utf-8")) + except json.JSONDecodeError: + return {} + return {} + + +# ---------------------------------------------------------------- phase 5 + +def cleanup_pass(repo: Path, run_dir: Path, plan: dict) -> dict: + """Verify + remove finished worktrees. Normal remove only; refusals + leave cleanup=pending recorded in the report — never --force.""" + results = [] + for entry in plan.get("entries", []): + if entry.get("phase") != "cleanup_verify": + continue + wt = Path(entry["worktree"]) + task_id = entry["task_id"] + checks: dict = {"task_id": task_id, "worktree": str(wt)} + # 1) task closed (not active in registry) + rc, st = agentctl(repo, "status") + reg = {} + if isinstance(st, dict): + raw = st.get("tasks") or {} + if isinstance(raw, list): + reg = {t.get("id", ""): t for t in raw if isinstance(t, dict)} + elif isinstance(raw, dict): + reg = raw + tstate = reg.get(task_id, {}).get("state", "absent") + checks["registry_state"] = tstate + # 2) branch head is dev ancestor (content already on dev) + rc, out = sh(["git", "-C", str(repo), "merge-base", "--is-ancestor", + entry.get("branch", ""), "dev"], repo) + checks["ancestor_of_dev"] = rc == 0 + # 3) worktree clean of tracked/untracked (ignored ok) + rc, out = sh(["git", "-C", str(wt), "status", "--porcelain"], wt) + lines = [ln for ln in out.splitlines() if ln.strip() and not ln.startswith("!!")] + checks["clean"] = not lines + checks["dirty_sample"] = lines[:10] + if tstate not in {"done", "cancelled", "absent"}: + checks["verdict"] = "skip_task_active" + elif not checks["ancestor_of_dev"]: + checks["verdict"] = "skip_not_on_dev" + elif not checks["clean"]: + checks["verdict"] = "skip_dirty" + else: + rc, out = sh( + [str(repo / "scripts/agent-worktree.sh"), "remove", + entry["branch"], "--delete-branch"], + repo, timeout=600, + ) + checks["remove_rc"] = rc + checks["remove_out"] = out[-400:] + checks["verdict"] = "removed" if rc == 0 else "remove_refused_keep_pending" + results.append(checks) + write_json(run_dir / "cleanup.json", {"results": results}) + return {"results": results} + + +# ---------------------------------------------------------------- phase 6 + +def write_report(repo: Path, run_dir: Path, ctx: dict, plan: dict, advance: list, envs: dict, cleanup: dict) -> Path: + date = ctx["started_at"][:10] + lines = [] + lines.append(f"# 夜间推进自动化测试报告 {date}") + lines.append("") + lines.append("## 一、 测试背景") + lines.append("每日 03:10 定时推进 .worktrees 分支任务、合入 dev、更新 dev 服务并清理已收口目录的自动化链路。") + lines.append("") + lines.append("## 二、 测试目标") + lines.append("验证夜间流水线可无人值守执行：计划构建 → 任务推进 → dev 合入 → 服务更新 → 完成目录清理 → 报告落盘。") + lines.append("") + lines.append("## 三、 测试方案与环境") + lines.append(f"- 主仓：{repo}（dev）") + lines.append("- 触发：cron 10 3 * * * → scripts/nightly-advance.sh") + lines.append(f"- 执行器：cursor-agent 无头模式；agentctl daemon（coordinator.service）") + lines.append(f"- run 目录：{run_dir}") + lines.append("") + lines.append("## 四、 预期结果") + lines.append("活跃任务按计划推进/合入或带原因 blocked；dev tip 变化时服务重建并通过 /health /ready；完成目录经无内容丢失验证后删除；每日报告归档 .doc/test-reports/。") + lines.append("") + lines.append("## 五、 实际测试结果与判定") + lines.append(f"- 开始：{ctx.get('started_at')}；dev 起点 {ctx.get('dev_tip_start','?')[:12]}") + tip = ctx.get("dev_tip_end") or dev_tip(repo) + lines.append(f"- 结束：{now_utc()}；dev 终点 {(tip or '?')[:12]}") + moved = tip != ctx.get("dev_tip_start") + lines.append(f"- dev 变更：{'是' if moved else '否'}") + lines.append("") + lines.append("### 任务推进明细") + lines.append("| task | phase | result | 摘要 | 证据 |") + lines.append("|---|---|---|---|---|") + for a in advance: + ag = a.get("agent") or {} + ev = ag.get("evidence") or a.get("log", "") + lines.append(f"| {a.get('task_id')} | {a.get('phase')} | {ag.get('result', 'no_summary')} | {(ag.get('summary') or '')[:80]} | {ev} |") + lines.append("") + lines.append("### dev 服务") + lines.append(f"状态：{envs.get('status')}") + if envs.get("health"): + lines.append(f"- /health ok={envs['health'].get('ok')} body={envs['health'].get('body','')[:120]}") + lines.append(f"- /ready ok={envs['ready'].get('ok')} body={envs['ready'].get('body','')[:120]}") + lines.append("") + lines.append("### 清理（已合入并验证无内容丢失后删除）") + lines.append("| worktree | 判定 | 原因 |") + lines.append("|---|---|---|") + for c in cleanup.get("results", []): + lines.append(f"| {Path(c['worktree']).name} | {c['verdict']} | {c.get('remove_out','')[:100].replace('|','/')} |") + lines.append("") + lines.append("## 六、 自动化测试套件汇总") + lines.append(f"- plan 构造：{plan.get('count', 0)} 项") + failed = [a for a in advance if (a.get("agent") or {}).get("result") in {"failed", "blocked"}] + lines.append(f"- 任务推进：{len(advance)} 项，失败/受阻 {len(failed)} 项") + lines.append("") + lines.append("## 七、 测试结论与后续") + overall = "PASS" if not failed and envs.get("status") in {"ok", "not_required"} else "PARTIAL" + lines.append(f"整体判定：{overall}（细节见 run 目录原始日志）") + lines.append("") + report_path = run_dir / "report-draft.md" + report_path.write_text("\n".join(lines), encoding="utf-8") + return report_path + + +# ---------------------------------------------------------------- main + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--repo", default=DEFAULT_REPO) + ap.add_argument("--max-tasks", type=int, default=0) + ap.add_argument("--task-timeout", type=int, default=3600) + ap.add_argument("--skip-agent", action="store_true", help="dry: plan+cleanup only") + ap.add_argument("--only-report", action="store_true", help="dry: reuse existing run dir state") + args = ap.parse_args() + + repo = Path(args.repo).resolve() + stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") + run_dir = repo / RUN_ROOT / stamp + run_dir.mkdir(parents=True, exist_ok=True) + + ok, errors, ctx = preflight(repo, run_dir) + write_json(run_dir / "state.json", ctx) + if not ok: + for e in errors: + print(f"FATAL: {e}", file=sys.stderr) + return 3 + touch_heartbeat(repo) + + plan = build_plan(repo, run_dir) + ctx["plan_count"] = plan.get("count", 0) + write_json(run_dir / "state.json", ctx) + + advance: list[dict] = [] + if not args.skip_agent: + entries = plan.get("entries", []) + if args.max_tasks > 0: + entries = entries[: args.max_tasks] + for i, entry in enumerate(entries): + if entry.get("phase") == "cleanup_verify": + continue # handled in phase 5 + touch_heartbeat(repo) + tag = f"{i:02d}-{entry['task_id']}" + worktree = Path(entry["worktree"]) + prompt = ( + PROMPT_INTEGRATE.format( + task_id=entry["task_id"], worktree=worktree, + expected_dev=ctx["dev_tip_start"], + ) + if entry.get("phase") == "integrate_closeout" + else PROMPT_ADVANCE.format(task_id=entry["task_id"], worktree=worktree) + ) + res = run_agent(prompt, worktree, args.task_timeout, run_dir, tag) + res["task_id"] = entry["task_id"] + res["phase"] = entry.get("phase") + res["branch"] = entry.get("branch") + res["owner"] = entry.get("owner") + advance.append(res) + write_json(run_dir / "advance.json", advance) + + # phase 3: wait for daemon to drain the queue + for _ in range(30): + rc, q = agentctl(repo, "queue", "list") + if isinstance(q, dict) and q.get("items") == []: + break + touch_heartbeat(repo) + time.sleep(10) + + ctx["dev_tip_end"] = dev_tip(repo) + write_json(run_dir / "state.json", ctx) + + # phase 4 + envs = update_dev_service(repo, run_dir) + write_json(run_dir / "env.json", envs) + + # phase 5 + cleanup = cleanup_pass(repo, run_dir, plan) + ctx["dev_tip_final"] = dev_tip(repo) + write_json(run_dir / "state.json", ctx) + + # phase 6 + report = write_report(repo, run_dir, ctx, plan, advance, envs, cleanup) + + # phase 6.5: formalize the daily report into .doc/test-reports/ via a + # headless agent inside the report worktree (lease-wrapped commit). + report_archived = None + if not args.skip_agent: + report_task_wt = Path(os.environ.get( + "NIGHTLY_REPORT_WORKTREE", + str(repo / ".worktrees/nightly-advance-automation-2026-09-28"), + )) + if report_task_wt.exists(): + date = ctx["started_at"][:10] + report_task_id = os.environ.get("NIGHTLY_REPORT_TASK", STANDING_TASK) + prompt = PROMPT_REPORT.format( + date=date, run_dir=run_dir, report_task_id=report_task_id + ) + res = run_agent( + prompt, report_task_wt, min(args.task_timeout, 1800), run_dir, "report" + ) + report_archived = res + write_json(run_dir / "report-archive.json", res) + + print(json.dumps({ + "run_dir": str(run_dir), + "report_draft": str(report), + "report_archived": bool((report_archived or {}).get("agent")), + "plan_count": plan.get("count", 0), + "advanced": len(advance), + "env_status": envs.get("status"), + }, ensure_ascii=False)) + (repo / LOCK_REL).unlink(missing_ok=True) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/nightly/plan.py b/scripts/nightly/plan.py new file mode 100755 index 00000000..3b845a26 --- /dev/null +++ b/scripts/nightly/plan.py @@ -0,0 +1,257 @@ +#!/usr/bin/env python3 +"""Nightly advance plan builder. + +Reads agentctl registry + CURRENT.md + cleanup report and produces an ordered +task list for the nightly advance run. Dependency ordering rules: + +1. Integrate-candidate tasks (integrate-* worktrees) whose branch head is NOT + an ancestor of dev come FIRST — they unblock the dev tip their feature + tasks were built against, and their merge must land before feature tasks + that rebase onto the new tip. +2. Feature/fix tasks follow, ordered by status: + - in_progress / active first (work to advance) + - cleanup=pending done tasks after (closeout + verification + removal) +3. Tasks whose CURRENT status is done and whose worktree verdict is ready + land last (removal only). + +Output: JSON plan on stdout (or --out file) with entries: + {task_id, worktree, branch, owner, status, phase, reason} + +Phases: integrate_closeout | feature_advance | cleanup_verify +Exit codes: 0 ok, 2 usage/preflight failure, 3 unparsable upstream data. +""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from dataclasses import dataclass, field +from pathlib import Path + +DEFAULT_REPO = "/home/weichen/projects/Agent-System" +AGENTCTL = "scripts/agentctl.py" + + +def _agentctl(repo: Path, *args: str) -> dict: + """Run agentctl (notices on stderr) and parse its stdout JSON.""" + proc = subprocess.run( + [sys.executable, "-B", str(repo / AGENTCTL), *args], + capture_output=True, + text=True, + cwd=repo, + ) + if proc.returncode != 0: + raise RuntimeError( + f"agentctl {' '.join(args)} rc={proc.returncode}: {proc.stderr[-500:]}" + ) + # Some subcommands emit [notice] lines on stdout too; keep last JSON blob. + out = "\n".join( + ln for ln in proc.stdout.splitlines() if not ln.startswith("[notice]") + ).strip() + try: + return json.loads(out) + except json.JSONDecodeError as exc: + raise RuntimeError(f"unparsable agentctl output for {args}: {out[:300]}") from exc + + +def _git(repo: Path, *args: str) -> str: + proc = subprocess.run( + ["git", "-C", str(repo), *args], capture_output=True, text=True + ) + if proc.returncode != 0: + raise RuntimeError(f"git {' '.join(args)}: {proc.stderr.strip()[:300]}") + return proc.stdout.strip() + + +def parse_current_rows(repo: Path) -> dict[str, dict]: + """Parse CURRENT.md 并行认领 table rows keyed by task id (col 2).""" + rows: dict[str, dict] = {} + text = (repo / ".doc/ops/CURRENT.md").read_text(encoding="utf-8") + for line in text.splitlines(): + if not line.strip().startswith("|"): + continue + cells = [c.strip() for c in line.strip().strip("|").split("|")] + if len(cells) < 6 or cells[0] in {"owner", "---"} or set(cells[0]) <= {"-", " "}: + continue + task_id, status = cells[1], cells[5] + if not re.match(r"^[A-Za-z0-9._-]+-\d{4}-\d{2}-\d{2}", task_id): + continue + rows[task_id] = { + "owner": cells[0], + "task": task_id, + "branch": cells[2], + "worktree": cells[3], + "mutex": cells[4], + "status": status, + } + return rows + + +def registry_tasks(repo: Path) -> dict[str, dict]: + """task registry state; empty when unavailable (plan falls back to CURRENT).""" + try: + status = _agentctl(repo, "status") + except RuntimeError: + return {} + if not isinstance(status, dict): + return {} + out = status.get("tasks") or {} + if isinstance(out, dict): + return out + return {} + + +@dataclass +class PlanEntry: + task_id: str + worktree: str + branch: str + owner: str + status: str + phase: str + reason: str + extra: dict = field(default_factory=dict) + + def as_dict(self) -> dict: + d = { + "task_id": self.task_id, + "worktree": self.worktree, + "phase": self.phase, + "reason": self.reason, + } + d.update(self.extra) + return d + + +def build_plan(repo: Path) -> list[PlanEntry]: + try: + cleanup = _agentctl(repo, "cleanup", "report") + if not isinstance(cleanup, dict): + cleanup = {} + except RuntimeError: + # Preflight (advance.py) already gates on daemon health; a failing + # cleanup probe here means no worktree info — plan nothing rather + # than crash the whole nightly run. + cleanup = {} + current = parse_current_rows(repo) + reg = registry_tasks(repo) + + entries: list[PlanEntry] = [] + EXCLUDE = {"nightly-advance-automation-2026-09-28"} + for wt in cleanup.get("worktrees", []): + path = wt.get("path", "") + if not path: + continue + wt_name = Path(path).name + if wt_name in EXCLUDE: + continue # never plan the nightly automation task itself + branch = wt.get("branch", "") + tasks = wt.get("tasks") or [] + blockers = wt.get("blockers") or [] + notes = wt.get("notes") or [] + ancestor = "ancestor_of_dev" in notes + reg_states = [t.get("state", "") for t in tasks] + + # Pair integrate worktrees with their base feature task id. + m = re.match(r"^integrate-(.+?)-\d{4}-\d{2}-\d{2}(-r\d+)?$", wt_name) + base_feature = m.group(1) if m else None + + cur = None + for t in tasks: + if t.get("id") in current: + cur = current[t["id"]] + break + if cur is None and base_feature and base_feature in current: + cur = current[base_feature] + + status = (cur or {}).get("status") or ( + reg_states[0] if reg_states else "unknown" + ) + task_id = next((t["id"] for t in tasks if t.get("id")), wt_name) + owner = (cur or {}).get("owner") or ( + tasks[0].get("owner", "unknown") if tasks else "unknown" + ) + + if base_feature and not ancestor: + entries.append( + PlanEntry( + task_id=task_id, + worktree=path, + branch=branch, + owner=owner, + status=status, + phase="integrate_closeout", + reason="integrate candidate not yet on dev; land first so " + "dependent feature tasks rebase onto the new tip", + extra={"blockers": blockers, "base_feature": base_feature}, + ) + ) + elif "TASK_ACTIVE" in " ".join(blockers) or status in {"in_progress", "active"}: + entries.append( + PlanEntry( + task_id=task_id, + worktree=path, + branch=branch, + owner=owner, + status=status, + phase="feature_advance", + reason="active task; advance along its task doc to acceptance", + extra={"blockers": blockers}, + ) + ) + elif wt.get("verdict") == "ready" or ( + status in {"done", "cancelled"} and not blockers + ): + entries.append( + PlanEntry( + task_id=task_id, + worktree=path, + branch=branch, + owner=owner, + status=status, + phase="cleanup_verify", + reason="finished and cleanup-ready; verify no-content-loss " + "then remove worktree+branch", + extra={"blockers": blockers}, + ) + ) + # done-but-blocked worktrees stay out of the plan; they are reported + # by the nightly report as pending-closeout items. + + order = {"integrate_closeout": 0, "feature_advance": 1, "cleanup_verify": 2} + entries.sort(key=lambda e: (order[e.phase], e.task_id)) + return entries + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--repo", default=DEFAULT_REPO) + ap.add_argument("--out", help="write plan JSON here (stdout otherwise)") + ap.add_argument( + "--max-tasks", type=int, default=0, help="cap entries (0 = no cap)" + ) + args = ap.parse_args() + + repo = Path(args.repo).resolve() + plan = [e.as_dict() for e in build_plan(repo)] + if args.max_tasks > 0: + plan = plan[: args.max_tasks] + payload = { + "generated_at": __import__("datetime").datetime.now().isoformat(timespec="seconds"), + "repo": str(repo), + "count": len(plan), + "entries": plan, + } + text = json.dumps(payload, ensure_ascii=False, indent=2) + if args.out: + Path(args.out).write_text(text, encoding="utf-8") + else: + print(text) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/tests/test_nightly_advance.py b/scripts/tests/test_nightly_advance.py new file mode 100644 index 00000000..3b352d1a --- /dev/null +++ b/scripts/tests/test_nightly_advance.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +"""Unit tests for nightly advance plan builder (no repo side effects). + +Runs against fixtures in scripts/tests/fixtures/nightly/. Covers: +- CURRENT row parsing (skip header/separators, keyed by task id) +- plan ordering: integrate_closeout -> feature_advance -> cleanup_verify +- done-but-blocked worktrees stay out of the plan +- registry fallback when CURRENT has no row +""" + +from __future__ import annotations + +import json +import subprocess +import sys +import unittest +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import mock + +SCRIPTS = Path(__file__).resolve().parent.parent # scripts/ +sys.path.insert(0, str(SCRIPTS)) + +import importlib.util + +spec = importlib.util.spec_from_file_location( + "nightly_plan", SCRIPTS / "nightly" / "plan.py" +) +nightly_plan = importlib.util.module_from_spec(spec) +sys.modules["nightly_plan"] = nightly_plan +spec.loader.exec_module(nightly_plan) + +FIXTURES = Path(__file__).parent / "fixtures" / "nightly" + +CURRENT_MD = """# 当前焦点 + +## 并行认领 + +| owner | task | branch | worktree | mutex | status | +|---|---|---|---|---|---| +| Cursor | feat-a-2026-09-28 | feature/feat-a-2026-09-28 | .worktrees/feat-a-2026-09-28 | m1 | in_progress | +| pi | feat-b-2026-09-27 | feature/feat-b-2026-09-27 | .worktrees/feat-b-2026-09-27 | m2 | done | +| Codex | feat-c-2026-09-26 | fix/feat-c-2026-09-26 | （已清理） | m3 | done | +""" + +CLEANUP_REPORT = { + "registry": "initialized", + "status": "ok", + "worktrees": [ + { + "path": "/tmp/repo/.worktrees/integrate-feat-b-2026-09-27", + "branch": "feature/integrate-feat-b-2026-09-27", + "blockers": ["registry:TASK_ACTIVE"], + "notes": ["Close the registered task before removing its worktree"], + "tasks": [{"id": "integrate-feat-b-2026-09-27", "owner": "Cursor", "state": "active"}], + "verdict": "blocked", + }, + { + "path": "/tmp/repo/.worktrees/feat-a-2026-09-28", + "branch": "feature/feat-a-2026-09-28", + "blockers": ["registry:TASK_ACTIVE", "dirty_or_ignored_files"], + "notes": ["ancestor_of_dev"], + "tasks": [{"id": "feat-a-2026-09-28", "owner": "Cursor", "state": "active"}], + "verdict": "blocked", + }, + { + "path": "/tmp/repo/.worktrees/feat-d-2026-09-25", + "branch": "feature/feat-d-2026-09-25", + "blockers": [], + "notes": ["ancestor_of_dev", "registry_clear", "lifecycle_lock_idle"], + "tasks": [{"id": "feat-d-2026-09-25", "owner": "Cursor", "state": "done"}], + "verdict": "ready", + }, + { + "path": "/tmp/repo/.worktrees/feat-done-blocked-2026-09-24", + "branch": "feature/feat-done-blocked-2026-09-24", + "blockers": ["dirty_or_ignored_files"], + "notes": ["ancestor_of_dev"], + "tasks": [{"id": "feat-done-blocked-2026-09-24", "owner": "Cursor", "state": "done"}], + "verdict": "blocked", + }, + ], +} + + +class ParseCurrentRows(unittest.TestCase): + def test_parses_and_skips_non_rows(self): + with TemporaryDirectory() as td: + (Path(td) / ".doc/ops").mkdir(parents=True) + (Path(td) / ".doc/ops/CURRENT.md").write_text(CURRENT_MD, encoding="utf-8") + rows = nightly_plan.parse_current_rows(Path(td)) + self.assertIn("feat-a-2026-09-28", rows) + self.assertEqual(rows["feat-a-2026-09-28"]["status"], "in_progress") + self.assertEqual(rows["feat-b-2026-09-27"]["owner"], "pi") + self.assertEqual(rows["feat-c-2026-09-26"]["worktree"], "（已清理）") + + def test_separator_and_header_rejected(self): + lines = CURRENT_MD.splitlines() + rows = {} + for line in lines: + cells = [c.strip() for c in line.strip().strip("|").split("|")] + if len(cells) < 6 or cells[0] in {"owner", "---"} or set(cells[0]) <= {"-", " "}: + continue + rows[cells[1]] = cells + self.assertNotIn("task", rows) + self.assertNotIn("owner", rows) + + +class BuildPlan(unittest.TestCase): + def _plan(self, cleanup=CLEANUP_REPORT, current=CURRENT_MD, registry=None): + with mock.patch.object(nightly_plan, "_agentctl") as ac, \ + mock.patch.object(nightly_plan, "_git") as g: + ac.side_effect = lambda repo, *a: ( + cleanup if a[:2] == ("cleanup", "report") + else (registry if a and a[0] == "status" else {}) + ) + g.return_value = "sha" + with TemporaryDirectory() as td: + (Path(td) / ".doc/ops").mkdir(parents=True) + (Path(td) / ".doc/ops/CURRENT.md").write_text(current, encoding="utf-8") + entries = nightly_plan.build_plan(Path(td)) + return entries + + def test_ordering_and_phases(self): + entries = self._plan() + phases = [e.phase for e in entries] + self.assertEqual( + phases, + ["integrate_closeout", "feature_advance", "cleanup_verify"], + "expected integrate -> feature -> cleanup ordering", + ) + + def test_integrate_not_ancestor_gets_integrate_phase(self): + entries = self._plan() + integ = [e for e in entries if e.phase == "integrate_closeout"] + self.assertEqual(len(integ), 1) + self.assertEqual(integ[0].task_id, "integrate-feat-b-2026-09-27") + + def test_done_ready_worktree_is_cleanup_verify(self): + entries = self._plan() + clean = [e for e in entries if e.phase == "cleanup_verify"] + self.assertEqual(clean[0].task_id, "feat-d-2026-09-25") + + def test_done_blocked_out_of_plan(self): + entries = self._plan() + ids = [e.task_id for e in entries] + self.assertNotIn("feat-done-blocked-2026-09-24", ids) + + def test_current_fallback_to_registry_status(self): + # feat-d has no CURRENT row here; registry state done + verdict ready + entries = self._plan() + clean = [e for e in entries if e.phase == "cleanup_verify"] + self.assertEqual(clean[0].status, "done") + + def test_registry_unavailable_still_plans(self): + with mock.patch.object(nightly_plan, "registry_tasks", return_value={}): + entries = self._plan(registry={}) + self.assertTrue(entries) + + +class Cli(unittest.TestCase): + def test_max_tasks_cap(self): + # Subprocess run: patch nothing, but point --repo at a fixture tree so + # _agentctl/_git never touch the real repo; build_plan must still work. + with TemporaryDirectory() as td: + (Path(td) / ".doc/ops").mkdir(parents=True) + (Path(td) / ".doc/ops/CURRENT.md").write_text(CURRENT_MD, encoding="utf-8") + out = Path(td) / "plan.json" + # No entries can be planned without a real registry; assert rc=0 + # and empty plan rather than touching production. + rc = subprocess.run( + [sys.executable, "-B", str(SCRIPTS / "nightly/plan.py"), + "--repo", td, "--out", str(out), "--max-tasks", "1"], + capture_output=True, text=True, + ).returncode + self.assertEqual(rc, 0) + data = json.loads(out.read_text()) + self.assertEqual(data["count"], 0) + + +if __name__ == "__main__": + unittest.main()