Why: The drill, handbook, and Documents pin need a combined test result before review.
Evidence: make test exit 0 via make evidence (SELF). CI on the pull request remains the GATE.
Impact: Local suite passed, including worktree, Python, frontend, UI, and agent-command checks.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The drill report and storage plan r2.2 are on Documents main, and this repo still pointed at the previous pin.
Evidence: Documents PR https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Documents/pulls/90 merged as f76cce9 (SELF API merge, no CI statuses on that repo).
Impact: The gitlink moves from 3cc6438 to f76cce9. Second-instance writes stay closed.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The measured backup, PITR, and disconnect steps were only in the drill scripts, and the checklist still said those rows were not run.
Evidence: Results pgbench-20261012-131948, fault-20261012-142734, and backup-20261012-153441 (SELF). K4 stays failed.
Impact: Second-instance write traffic stays closed. The handbook records the upload panic and the short socket path.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The timed drill finished with a verdict for every criterion, including the upload failure.
Evidence: make evidence exit 1 because K4 failed; results tests/ops/results/backup-20261012-153441 (SELF). K1-K3 and K5-K7 passed. RPO 64s, RTO 4s.
Impact: Physical backup, WAL replay, and pg_dump restore are measured. Artifact upload panics when S3 is unset, so K4 is FAIL and product code was not changed.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The backup drill again refused to create an instance because the 1-minute load average was 18.23.
Evidence: make evidence exit 3, receipt on the task card (SELF).
Impact: No PostgreSQL instance was created.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The restored server promoted and the smoke check passed, then the drill stopped while reading artifact ids.
Evidence: make evidence exit 1, results tests/ops/results/backup-20261012-151951 (SELF). rto_s=3, smoke login/projects/conversation 200.
Impact: The instance was cleaned up. The next run records a full verdict, including a failed upload check.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: A missing artifact id aborted the drill before recovery counts, the settle window, and the logical restore.
Evidence: backup-20261012-151951 reached promoted=1 and smoke 200, then StopIteration on an empty id (SELF). bash -n passed.
Impact: K4 stays failed when uploads do not return an id. The rest of the verdict is still written.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: pg_ctl could not start the restored server, so the drill exited before a verdict.
Evidence: make evidence exit 1, results tests/ops/results/backup-20261012-150713 (SELF). The log says the Unix socket path is too long.
Impact: Cleanup removed the instance. The rerun uses a short socket path.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The restored postmaster refused to start because the results-directory socket path exceeded the 107-byte Unix socket limit.
Evidence: restore/postgres.log from backup-20261012-150713 says the socket path is too long (SELF). bash -n passed.
Impact: The restored instance listens on /tmp/u7bk-18997, and cleanup removes that directory.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The backup drill refused to create an instance because the 1-minute load average was 17.72.
Evidence: make evidence exit 3, receipt on the task card (SELF).
Impact: No PostgreSQL instance was created. The rerun waits until load is at or below 16.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: C4 evidence is recorded, and the next measured run is the backup and PITR drill.
Evidence: The increment line is the only change.
Impact: No runtime change.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The checklist had no measured restore path for a physical backup, WAL archive, or pg_dump onto a new target.
Evidence: bash -n, go test in tests/ops/drill_driver, and run_backup_drill.sh --self-check passed locally (SELF). The timed run is recorded separately.
Impact: Isolated u7bk only. Upload responses keep artifactId so the restore check can download the same objects.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The corrected probe measured bounded failures, sub-second recovery without restarting the apps, and Redis shared limits across both instances.
Evidence: make evidence exit 0, results tests/ops/results/fault-20261012-142734 (SELF). F1-F5 and G1-G3 passed. G4 and G5 are the expected findings.
Impact: C4 is SELF on this host. Second-instance write traffic stays closed.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The corrected drill refused to create an instance because the 1-minute load average was 16.98.
Evidence: make evidence exit 3, receipt on the task card (SELF).
Impact: No PostgreSQL instance was created. The rerun waits until load is at or below 16.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: A second evidence run found port 18991 still held by the in-flight drill and exited before creating an instance.
Evidence: make evidence exit 2, receipt on the task card (SELF).
Impact: Nothing was created. The in-flight drill was stopped so the corrected probe can run.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The first pg-stop window counted an in-flight 200 at shutdown, and five writes on the timeline conversation returned 409 while a mock run held it. That measured the probe, not a proven database failure.
Evidence: go test ./... in tests/ops/drill_driver passed locally (SELF).
Impact: The outage clock starts 2s after stop or kill. RTO is five writes on new conversations. Idempotency replays only the window and retries 409 for up to 2 minutes. The Redis phase no longer uses local outside a function.
Refs: feature/u7-ops-drills
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The release checklist cannot mark disconnect recovery until an isolated drill records bounded failures, no SQLite fallback, and idempotent writes.
Evidence: bash -n passed, drill_driver go test passed, unknown arguments exit 2. The timed drill is recorded next.
Impact: New drill entry only. No product change and no release decision.
Refs: docs/release-multi-instance.md C4
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: C4 and C5 need a shared way to start, refuse, and tear down stacks without touching production ports or processes.
Evidence: tests/ops/self_check.sh passed (help, unknown arg, foreign name, production port, busy port, SIGINT cleanup). drill_driver go test passed.
Impact: New tests/ops library and HTTP driver. No drill result and no release-gate change yet.
Refs: docs/release-multi-instance.md C4 C5
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The database-only baseline is usable only with the command, object SHA, and full output kept on the task card.
Evidence: CMD=tests/load/collaboration/run_pgbench.sh make evidence exit 0 at b2e129a (SELF). Six windows, zero failed transactions. An earlier parse failure and a load-gate pause stay on the card.
Impact: Evidence only. Checklist B6 is not flipped until the drill report and checklist edit land. Second-instance writes stay closed.
Refs: docs/release-multi-instance.md B6
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: A timed window that starts under load average above 16 is not a usable baseline, and creating the instance first only to pause wastes a startup.
Evidence: The rerun at load 24.03 exited 3 before any timed window and removed the instance.
Impact: The pause now happens before initdb. No change to the benchmark itself.
Refs: docs/release-multi-instance.md B6
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: The percentile reader also opened the pgbench stdout file, so the first run exited 1 after six successful windows.
Evidence: Parser check on results/pgbench-20261012-131229 select-only-10 gave p50 0.018 ms, matching the reported average. The timed rerun is recorded next.
Impact: Percentile parsing only. The six windows themselves had zero failed transactions.
Refs: docs/release-multi-instance.md B6
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: Continuous delivery needs the approved scope, stop conditions, and B6/C4/C5 acceptance on the task card before any drill runs.
Evidence: Precheck receipt imported at task-start (SELF, R0 8ebf502146); plan approved by the user on 2026-10-12.
Impact: Records authorization only. No drill, product, or release-gate change.
Refs: docs/release-multi-instance.md B6/C4/C5
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: establish the authorized task workspace and its initial baseline.
Evidence: preserved precheck receipt in .task/feature-u7-ops-drills.md; R0=8ebf5021468a1301ccbe2fe0568508e16d8bb708.
Impact: task metadata only; the feature workspace starts at 8ebf502146.
Refs: .task/feature-u7-ops-drills.md
Why: branches merged into main had left their task cards open; backfill status and archive to prevent bookkeeping debt.
Evidence: feature-pin-backend-read-consumption.md(PR#470) — merges verified via git log --merges against origin/main.
Impact: .task/*.md moved to .task/archive/ with status backfilled; no code or submodule change.
Refs: task.sh close (card-autoclose-r1)
Why: include the current integration baseline before verification.
Evidence: fetched origin/main=ac1de9c4c44a74b6a20a76ef3d948b492db84c39.
Impact: integrate upstream changes without rewriting task history.
Refs: task-sync
Why: Archived card still said merge/cleanup NOT_RUN, and the green
make-test retry landed on the feature branch after PR #468 merged.
Evidence: merge 1325824 CI push jobs success; receipt from 30a5421.
Impact: Close record matches merged facts; SELF capacity still not a release.
Refs: Dev #468; Documents #83#86
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: resolve archive card conflict after u7 card already landed on main.
Evidence: took main .task/archive/feature-u7-capacity-wrap-measure.md; kept audit-combo archive.
Impact: #449 only archives remaining audit-combo debt if needed.
Refs: Dev #449
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: PR head changed after resolving documents/frontend sync merge.
Evidence: CMD=make test make evidence (SELF, exit 0).
Impact: PR #468 head carries current combo verification.
Refs: Dev #468
Co-authored-by: Cursor <cursoragent@cursor.com>
Why: establish the authorized task workspace and its initial baseline.
Evidence: preserved precheck receipt in .task/feature-pin-backend-read-consumption.md; R0=1dece88ab0eb6bbeab0ddfb4c7c57c2a7752d248.
Impact: task metadata only; the feature workspace starts at 1dece88ab0.
Refs: .task/feature-pin-backend-read-consumption.md
Why: Restore mergeability for PR #468 after main advanced; keep
Documents pin that includes capacity report r2.1 (3cc6438).
Evidence: documents contains 2026-10-12-u7-capacity-window-report.md.
Impact: PR can merge onto current main.
Refs: Dev #468; Documents #86/#87
Co-authored-by: Cursor <cursoragent@cursor.com>
weichenandCursor