Why: include the current integration baseline before verification.
Evidence: fetched origin/main=cd61bbb112eea6a210a64e84f2cef00b0d64bfc1.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: identity ensure no longer creates the schema, so the probe has to follow a real serve that applies migrations.
Evidence: SELF python3 -m py_compile tests/test_svc_realcombo.py. GATE run 2663 job 3839 failed on the previous head with identity table missing.
Impact: the PostgreSQL check still passes the DSN only through the environment, after serve has migrated.
Refs: #485
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: env -i dropped RDM_DISABLE_EXEC_TOOLS, so the real RDM binary exited immediately on runners without bwrap, and the capability probe now prints the JSON document.
Evidence: SELF bash -n scripts/svc.sh; python3 -m py_compile tests/test_svc_realcombo.py. GATE run 2656 job 3831 failed on the previous head before this fix.
Impact: serve and the lifecycle child forward that one switch; identity capabilities is checked as protocol 1 JSON.
Refs: #485
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: A missing artifact id aborted the drill before recovery counts, the settle window, and the logical restore.
Evidence: backup-20261012-151951 reached promoted=1 and smoke 200, then StopIteration on an empty id (SELF). bash -n passed.
Impact: K4 stays failed when uploads do not return an id. The rest of the verdict is still written.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: main gained the real-combination CI gate after the previous sync, and this branch still has to carry that gate with the isolated identity calls.
Evidence: SELF python3 -m unittest recovery/identity/u2/deploy/pin/isolation — 100 tests OK (2 skipped); bash -n scripts/svc.sh.
Impact: identity ensure/verify still pass DSN only through the environment; stop waits for the port and can keep the desired state. CI keeps the lifecycle pin step and the svc-realcombo job.
Refs: #485
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The restored postmaster refused to start because the results-directory socket path exceeded the 107-byte Unix socket limit.
Evidence: restore/postgres.log from backup-20261012-150713 says the socket path is too long (SELF). bash -n passed.
Impact: The restored instance listens on /tmp/u7bk-18997, and cleanup removes that directory.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: main already has one release transaction, so this branch keeps that path and only isolates update and recover in a clean child.
Evidence: SELF bash -n scripts/svc.sh; python3 -m unittest identity/recover/deploy/u2/lifecycle pin 92 tests OK (2 skipped), then 66 tests OK after dropping the obsolete journal hook.
Impact: deploy uses releases/ and deploy.journal; the lifecycle child reconciles that journal before deploy or recover. Backend pin is 0f192a4.
Refs: #485
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The checklist had no measured restore path for a physical backup, WAL archive, or pg_dump onto a new target.
Evidence: bash -n, go test in tests/ops/drill_driver, and run_backup_drill.sh --self-check passed locally (SELF). The timed run is recorded separately.
Impact: Isolated u7bk only. Upload responses keep artifactId so the restore check can download the same objects.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: only the deploy invocation carried RDM_DISABLE_EXEC_TOOLS; the
trailing restart ran without it and the serve died on keyless runners
under the bwrap fail-closed policy (run 3765).
Evidence: local suite green; the restart now inherits the hard-disable.
Impact: tests only.
Why: the CI-only serve-exit failure left no server-side evidence in the
assertion output; the tail pinpoints the refuse reason on the runner.
Evidence: local rerun green (diagnostic inert on success).
Impact: test diagnostics only.
Why: an update that stops the old process and overwrites the binary cannot
get the previous version back, and a migration the old binary cannot read
must be refused before that stop. CI also had no fixed RDM SHA to build from.
Evidence: python3 -m unittest tests.test_svc_identity tests.test_svc_recover tests.test_svc_deploy_txn tests.test_lifecycle_pin passed. make test-lifecycle exits 2 when AGENT_PG_TEST_DSN is unset.
Impact: installs land in releases/<rid> and current switches with mv -T.
Deploy keeps the desired state and rolls back a non-terminal journal.
Schema compatFloor above the installed target is refused unless
--migrate-schema --confirm is paired with a backup. test-lifecycle fails
closed without PostgreSQL and an RDM checkout. Login stays NOT_RUN.
Refs: recovery plan r3.1 D2 and D3; Backend feature/d1-identity-open
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: on keyless runners the RDM cache clone and pins materialization
inside svc.sh fetched nested pins over raw SSH URLs (run 2608: host key
verification failed); the suite now injects the same URL-scoped token
header and insteadOf rewrite via environment when SVC_CI_GIT_* is set.
Evidence: local rerun stays green (env unset locally); CI run applies it.
Impact: tests + ci.yaml only; token travels via environment only.
Why: CI runners lack bwrap, so the real RDM binary refuses to start
under the sandbox fail-closed policy (run 2607: serve exits right after
launch); the product provides RDM_DISABLE_EXEC_TOOLS as the top-level
fallback and lifecycle semantics under test are unchanged.
Evidence: local rerun with the env set stays green.
Impact: tests only.
Why: stub suites proved the script logic but not real artifact
behavior (the 2026-10-12 incidents passed all stub tests); the report's
U3 requires real-combination gates, and realcombo dry-runs exposed two
real gaps: fresh deploys closed identity governance one lifecycle late,
and pg-bound identity calls put the DSN on the command line.
Evidence: make test exit 0; local realcombo dry-run 16 OK (rows in the
task card); CI adds svc-realcombo (SQLite, both real binaries, full RDM
track) and a real-PG identity step on test-pg with env-delivered DSN.
Impact: tests/test_svc_realcombo.py (SVC_TEST_REAL gate, explicit
NOT_RUN otherwise); ci.yaml new job + test-pg step (RDM clone via
trusted-base URL-scoped token header, allowlist addition reviewed in
this PR); svc.sh delivers identity DSN via AGENT_SYSTEM_DSN and runs
manifest adoption in the same deploy that first registers svc.env;
modules/backend pinned to main (env fallback, backend #182).
Refs: .task/feature-svc-u3-realcombo.md; user goal directive 2026-10-12
(U3); backend #182; RDM #285.
Why: The first pg-stop window counted an in-flight 200 at shutdown, and five writes on the timeline conversation returned 409 while a mock run held it. That measured the probe, not a proven database failure.
Evidence: go test ./... in tests/ops/drill_driver passed locally (SELF).
Impact: The outage clock starts 2s after stop or kill. RTO is five writes on new conversations. Idempotency replays only the window and retries 409 for up to 2 minutes. The Redis phase no longer uses local outside a function.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: probing identity by running the subcommand still launched old RDM
builds, and ensure on a missing database created an empty one. A batch
update also inherited the caller's AGENT_SYSTEM_* variables.
Evidence: python3 -m unittest tests.test_svc_identity tests.test_svc_recover passed (41 tests, 1 skipped). tests.test_svc_lifecycle_isolation caller-env and restart cases passed.
Impact: capability probes use version --json only. Established instances
refuse a missing identity row until adopt --confirm. update and recover
run each service in a clean environment. serve reads a 0600 serve.env
and does not inherit the caller environment. Backend and documents pins
move to the D1 identity and recovery-plan commits.
Refs: recovery plan r3.1 D1; Backend feature/d1-identity-open; RDM feature/d1-version-json
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=3b9cf82cd6fb53f973f26e2039275f7858b0c654.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: deploy overwrote live artifacts after stopping the old service, so
a failed start or workbench check left instances in a mixed
registration-vs-artifact state with no recovery path (2026-10-12
rdm1/rdmcand); the report's U2 requires restorable updates.
Evidence: make test-governance 250 OK; five svc suites 105 OK (rows in
the task card); new tests cover layout/pidcheck stability, rollback on
start failure with desired-state restore, build failure keeping the old
service running, interrupted-switch journal resolution, and schema-
advance blocking rollback without touching the database.
Impact: agent deploys stage immutable releases/<id>/ and switch the
current symlink atomically; deploy.journal persists the switch and is
resolved at start/restart/deploy/update/recover entries; legacy layouts
are adopted on first deploy and remain readable; release/rdm tracks
verify sha256 before staging.
Refs: .task/feature-svc-u2-release-recovery.md; user goal directive
2026-10-12; Dev #474 (U1 gate); Documents unified-deploy plan r2.
Why: The release checklist cannot mark disconnect recovery until an isolated drill records bounded failures, no SQLite fallback, and idempotent writes.
Evidence: bash -n passed, drill_driver go test passed, unknown arguments exit 2. The timed drill is recorded next.
Impact: New drill entry only. No product change and no release decision.
Refs: docs/release-multi-instance.md C4
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: C4 and C5 need a shared way to start, refuse, and tear down stacks without touching production ports or processes.
Evidence: tests/ops/self_check.sh passed (help, unknown arg, foreign name, production port, busy port, SIGINT cleanup). drill_driver go test passed.
Impact: New tests/ops library and HTTP driver. No drill result and no release-gate change yet.
Refs: docs/release-multi-instance.md C4 C5
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=6a494d3977e6557e7bebe0e5bb1c4a12091296e0.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: A timed window that starts under load average above 16 is not a usable baseline, and creating the instance first only to pause wastes a startup.
Evidence: The rerun at load 24.03 exited 3 before any timed window and removed the instance.
Impact: The pause now happens before initdb. No change to the benchmark itself.
Refs: docs/release-multi-instance.md B6
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The percentile reader also opened the pgbench stdout file, so the first run exited 1 after six successful windows.
Evidence: Parser check on results/pgbench-20261012-131229 select-only-10 gave p50 0.018 ms, matching the reported average. The timed rerun is recorded next.
Impact: Percentile parsing only. The six windows themselves had zero failed transactions.
Refs: docs/release-multi-instance.md B6
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: batch update leaked AGENT_SYSTEM_INSTANCE_ID/DBREF from one service
to the next, and RDM artifacts without the identity CLI were stopped and
overwritten before their startup identity refusal surfaced, leaving
rdm1/rdmcand in a mixed registration-vs-artifact state (2026-10-12).
Evidence: make test-governance 217 OK; tests.test_svc_identity and
tests.test_svc_precheck green; rows recorded in the task card.
Impact: env_config resets identity env per service; capability probe is
timeout-guarded and rejects serve-shaped output; identity ensure moves
to adoption/init (bare start is verify-only); deploy builds and gates
candidates before stopping the old service (refuses identityless
candidates, completes missing identity rows via the candidate); update
runs svc_manifest_migrate like every other lifecycle entry.
Refs: .task/feature-svc-identity-u1.md; user root-cause report 2026-10-12.
Why: make the agent responsible for merging reviewed PRs without another human merge step.
Evidence: SELF governance and generated-card checks are captured in .task/feature-agent-pr-merge.md; CI remains required on the final PR HEAD.
Impact: update project delivery policy, agent entry points, task metadata and closure guidance; preserve platform protection and scoped authorization.
Refs: .task/feature-agent-pr-merge.md; user instruction on 2026-10-12
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=b4b042630a352eca2a399a7a64931c74978bca7f.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: make the six instance-bound user commands discoverable (T5-1) and pin Documents r2/r3.
Evidence: python3 -m unittest tests.test_svc_user_help; existing test_svc_user_binding still green.
Impact: scripts/svc.sh help accounts|user; docs/svc.md topic list; documents gitlink to plan branch tip.
Refs: Documents #87; feature/svc-account-durability-plan
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=c6fefa3435dc5d9524fc35483f1d34fa1a6e0d59.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
# Conflicts:
# tests/test_svc.py
Why: Batch update --all left SVC_PIDCHECK from the prior service in
memory, so pid_alive/reclaim_own_port misclassified the next service's
process as dead/foreign, deleted svc.pid, and failed start on a still-
occupied port (genui/rdm1/rdmcand after successful dev).
Evidence: make test-governance (SELF; receipt /tmp/svc-pidcheck-gov.json);
new test_batch_update_stops_second_service_despite_prior_pidcheck +
SVC_PIDCHECK ratchet in test_svc_precheck.
Impact: env_config clears SVC_PIDCHECK/EXEC/VERSION/SOURCE per service;
batch update correctly stops and restarts each agent. After merge:
restore genui/rdm1/rdmcand and move rdmcand to its own data dir.
Refs: .task/feature-svc-pidcheck-leak.md; complements #456 reclaim_own_port
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: genui update cleared svc.pid without killing the process, then
overwrote the binary and failed start with a false "non-ours" port error.
Evidence: make test-governance (receipt /tmp/svc-reclaim-governance.json)
Impact: reclaim_own_port walks ss listener ancestry; stop/check_port/install
wire it; orphans matching pidcheck are stopped, foreign occupants still refused
Refs: fix/svc-reclaim-own-port
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: the agent stub left python http.server children when tearDown
SIGKILL'd the bash parent, leaking ports 1880x into recover/identity
suites that share the same range.
Evidence: local suite re-run pending after this fix.
Impact: stub HTTP child exits when ppid changes (same as recover stub).
Refs: .task/feature-svc-rdm-ui-gate.md
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: RDM agent deploy left workbench htdocs unserved (prompt-only), so
environments like dev silently offered API-only 404 pages after a source
switch; static first-start also misclassified http.server as dead when
pidcheck was unset.
Evidence: CMD="make test-governance" make evidence (200 ok, 1 skipped; SELF).
Impact: rdm deploy auto-starts <name>web and asserts UI 200; cross-kind
redeploy needs --allow-source-switch; list/info flag missing workbench.
Refs: .task/feature-svc-rdm-ui-gate.md; docs/svc.md §5.4
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: backend U4 password gate returns 403 on business APIs until must_change clears; CLI-created load passwords are terminal.
Evidence: dual smoke failed POST /api/projects 403 after login_keys pin.
Impact: load suite works on schema v26+; EXIT trap disables re-entry and waits for serve.
Refs: server enforcePasswordGate; CreateAccount must_change
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: background subshell PID was recorded instead of serve, so cleanup left orphans that stole ports and broke the next smoke.
Evidence: prior dual evidence failed with bind address already in use + probe 401 against stale instance.
Impact: trap kill now terminates the real serve process.
Refs: tests/load/collaboration/run_load.sh
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=673e687c5c85b9af3ce13ca30845d5e68de5dfa0.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=673e687c5c85b9af3ce13ca30845d5e68de5dfa0.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: Backend login_keys gate blocks Key traffic while
must_change_password is set; U7 failed on PUT settings/user 403.
Evidence: go test -c multi-instance compiles; CI test-pg will GATE.
Impact: loginAlice does session password change alicePass0→alicePass.
Refs: Dev PR #426; Backend #170 pin e7b70c2
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=b431777c9fa9c74f096d7f14fce40d1beba53fe2.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: Dev #429 conflicted on scripts/svc.sh (U1 user vs main backup) and Backend gitlink.
Evidence: conflict hunks resolved; actions include init|user|backup|backups
Impact: PR #429 should become mergeable after push; both U1 binding and agent backup entrypoints preserved.
Refs: Dev pulls/429; main backup #417; Backend #171
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: task-sync hit a submodule conflict; pin must track Backend #170
HEAD after rebase onto main (copyOrder fix).
Evidence: backend 0c79618; origin/main merge completed in worktree.
Impact: modules/backend gitlink; sync with main.
Refs: Backend #170; Dev #426
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=ae465d24441ab349c961a55e9b04f8c4ccaf6de8.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
weichen
