U1+U2: team PG tier, atomic migrations, connection budget, real-database CI gate (pin #137/#61) #325

Merged
weichen merged 13 commits from feature/collab-storage-pg-tier into main 2026-10-11 04:13:50 +00:00
Owner

Implements the approved collaboration plan's first increment U1+U2 (user directive 2026-10-11: start the transformation per the plan).

  • backend PR #137 (pin 2497508): team/local deployment tiers with fail-closed readiness (KTD1), DDL+ledger single-transaction migrations with PG advisory-lock and SQLite immediate-transaction serialization plus unknown-schema refusal (KTD8), configurable PG connection budget with statement_timeout and sanitized diagnostics (KTD12), tier/pool/atomicity/race/old-schema tests, and fixes to three real-PG test defects the new gate exposed.
  • documents PR #61 (pin 3e2d8c9): team-storage runbook (U1 manifest, config contract, migration mapping, capacity contract, isolation scheme, KTD14 copy protection).
  • Dev: make test-backend-pg target (fails fast when AGENT_PG_TEST_DSN/AGENT_REDIS_TEST_ADDR missing) and a required CI test-pg job (apt PostgreSQL 16 + pgvector + Redis; no silent skip).

Evidence (SELF, via evidence tool in the task card): make test PASS; real-service gate make test-backend-pg green twice against isolated PostgreSQL 18.6+pgvector and Redis 8.10.2; make build PASS. CI test-pg first run happens on this PR.

Task card: .task/feature-collab-storage-pg-tier.md

Implements the approved collaboration plan's first increment U1+U2 (user directive 2026-10-11: start the transformation per the plan). - backend PR #137 (pin 2497508): team/local deployment tiers with fail-closed readiness (KTD1), DDL+ledger single-transaction migrations with PG advisory-lock and SQLite immediate-transaction serialization plus unknown-schema refusal (KTD8), configurable PG connection budget with statement_timeout and sanitized diagnostics (KTD12), tier/pool/atomicity/race/old-schema tests, and fixes to three real-PG test defects the new gate exposed. - documents PR #61 (pin 3e2d8c9): team-storage runbook (U1 manifest, config contract, migration mapping, capacity contract, isolation scheme, KTD14 copy protection). - Dev: make test-backend-pg target (fails fast when AGENT_PG_TEST_DSN/AGENT_REDIS_TEST_ADDR missing) and a required CI test-pg job (apt PostgreSQL 16 + pgvector + Redis; no silent skip). Evidence (SELF, via evidence tool in the task card): make test PASS; real-service gate make test-backend-pg green twice against isolated PostgreSQL 18.6+pgvector and Redis 8.10.2; make build PASS. CI test-pg first run happens on this PR. Task card: .task/feature-collab-storage-pg-tier.md
weichen added 6 commits 2026-10-11 03:17:41 +00:00
Why: establish the authorized task workspace and its initial baseline.

Evidence: preserved precheck receipt in .task/feature-collab-storage-pg-tier.md; R0=7743f53c5dfe9ed8dfd0663d05e3be15544d8942.

Impact: task metadata only; the feature workspace starts at 7743f53c5d.

Refs: .task/feature-collab-storage-pg-tier.md
Why: the approved collaboration plan U2 requires CI to run the backend suite against real PostgreSQL+pgvector and Redis and fail when the services are missing, instead of silently skipping the tier.

Evidence: local gate green twice against PostgreSQL 18.6+pgvector and Redis 8.10.2 (isolated svc.sh instance); the target fails fast when AGENT_PG_TEST_DSN/AGENT_REDIS_TEST_ADDR are unset.

Impact: new make target test-backend-pg (env-guarded) and a required test-pg job (apt postgresql-16-pgvector + redis-server, ephemeral per-run services); existing test job unchanged.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U2; docs/guides/team-storage.md section 5.
Why: the parent must pin the merged component SHAs so the combined build and CI gate run against the delivered U2 code and U1 runbook.

Evidence: backend origin-main merge commit of #137 and documents origin-main merge commit of #61 (fetched post-merge, replacing a stale pre-merge pin attempt).

Impact: gitlink updates only.

Refs: backend PR #137; documents PR #61.
Why: include the current integration baseline before verification.

Evidence: fetched origin/main=2afcfd5d6db29e76ae89b34798b8f34f030a3a69.

Impact: integrate upstream changes without rewriting task history.

Refs: task-sync
Why: bind the task validation checkpoint to the latest integration baseline.

Evidence: SELF recursive component checks; sync_base=2afcfd5d6db29e76ae89b34798b8f34f030a3a69; candidate=94d8158fba12f213d1ed9206018fe9bc7a00c318.

Impact: task metadata only.

Refs: .task/feature-collab-storage-pg-tier.md
Record the U1+U2 delivery state and evidence in the task card
ci / test-pg (pull_request) Failing after 33s
ci / test (pull_request) Successful in 2m22s
5d71f8a73e
Why: the card is the single record for stage review; results, stage verdicts and the three evidence runs must be in it before the PR.

Evidence: make test / test-backend-pg (real PG+Redis) / make build all exit 0, written by the evidence tool.

Impact: task metadata only.

Refs: backend PR #137; documents PR #61.
weichen added 1 commit 2026-10-11 03:18:26 +00:00
Fill the task record PR link and authorization basis
ci / test (pull_request) Successful in 2m21s
ci / test-pg (pull_request) Failing after 33s
2d2c792316
Why: card and PR description are one document; the PR link and the user's authorization basis must be on the branch before merge.

Evidence: Dev PR #325 created via Gitea API; user directive 2026-10-11 quoted in the card.

Impact: task metadata only.

Refs: PR #325.
weichen added 1 commit 2026-10-11 03:34:43 +00:00
Run the CI PG setup as postgres via runuser/su (root container, no sudo)
ci / test-pg (pull_request) Failing after 32s
ci / test (pull_request) Failing after 1m41s
b41b136607
Why: the runner container executes as root without sudo; the previous $SUDO -u postgres expanded to a bare '-u' and failed with exit 127 on the first test-pg run (job 2280 log).

Evidence: job 2280 log lines '/var/run/act/workflow/3: line 6: -u: command not found', exitcode 127; postgresql service itself started fine as root.

Impact: test-pg job setup steps only; also prints pinned pgvector/pg_trgm extension versions for acceptance records.

Refs: PR #325; docs/guides/team-storage.md section 5.
weichen added 1 commit 2026-10-11 03:43:08 +00:00
Fix the CI test-pg setup: single bracket test and the vector extension name
ci / test-pg (pull_request) Failing after 53s
ci / test (pull_request) Successful in 2m22s
53defb4bb4
Why: the runner-container guard had a doubled '[[' from a string replacement (exit '[: too many arguments'), and pgvector's extension is named 'vector' (control file vector.control) - CREATE EXTENSION pgvector failed with 'extension not available' (jobs 2285/2286 logs).

Evidence: job 2286 log 'ERROR: extension pgvector is not available ... pgvector.control: No such file'; package postgresql-16-pgvector 0.6.0 installed fine.

Impact: test-pg job only; repins backend to the #138 merge (30s SQLite busy timeout for serialized migrators).

Refs: backend PR #138; Dev PR #325.
weichen added 1 commit 2026-10-11 03:48:18 +00:00
Install house-style Python deps in the test-pg job too
ci / test (pull_request) Successful in 2m15s
ci / test-pg (pull_request) Successful in 53s
7134f83fde
Why: the job runs the full backend suite against real services; internal/runtime lint tests require python-docx (job 2290: ModuleNotFoundError 'docx') which the plain test job already installs.

Evidence: job 2290 log; all PG/Redis-dependent packages green in the same run (store 2.6s, pgbus, runmgr, ratelimit, server).

Impact: test-pg job setup only.

Refs: Dev PR #325.
weichen added 2 commits 2026-10-11 04:04:35 +00:00
Why: include the current integration baseline before verification.

Evidence: fetched origin/main=5771d4f1e5b692bce95eb64cc30dc9931079ad12.

Impact: integrate upstream changes without rewriting task history.

Refs: task-sync
Record the sync checkpoint
ci / test (pull_request) Failing after 1m44s
ci / test-pg (pull_request) Failing after 1m11s
20f74220d2
Why: bind the task validation checkpoint to the latest integration baseline.

Evidence: SELF recursive component checks; sync_base=5771d4f1e5b692bce95eb64cc30dc9931079ad12; candidate=f4d46afdd2fe1035ef325b5c3198a50ebbaae813.

Impact: task metadata only.

Refs: .task/feature-collab-storage-pg-tier.md
weichen added 1 commit 2026-10-11 04:09:09 +00:00
Repin backend to the migration busy-retry fix (PR #139)
ci / test-pg (pull_request) Successful in 52s
ci / test (pull_request) Successful in 2m16s
bb82924454
Why: the racing-migrator test failed deterministically on slow CI disks from the connection-open phase; PR #139 adds a bounded Begin retry and pre-materializes the racing test database.

Evidence: backend PR #139 merged (CI logs jobs 2285/2293 cited there); local store suite green.

Impact: gitlink only.

Refs: PR #325.
weichen merged commit 62d7c6e27c into main 2026-10-11 04:13:50 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: weichen/Agent-System-Dev#325