Commit Graph
23 Commits
Author SHA1 Message Date
weichenandCursor 69ea7dcc14 Open identity commands on an existing database only
Why: identity ensure/verify went through the normal store open, which
creates a database and runs migrations. A deploy gate that calls the
candidate before stopping the old service could migrate production
ahead of the switch, and a missing database was turned into an empty one.
Evidence: go test ./pkg/identity/ ./internal/store/ ./cmd/agent-system/ -run 'TestCapabilities|TestEnsure|TestVerify|TestDSN|TestUsage|TestSystemIdentity|TestOpenIdentity|TestUnknownSubcommand' passed in this worktree.
Impact: identity CLI refuses a missing file or a missing agent_system_identity table, does not migrate, reads DSN from the environment or a 0600 file, and version --json plus identity capabilities emit one capability document. Unknown top-level commands exit 2.
Refs: recovery plan r3.1 D1; Dev feature/svc-d1-identity-flow

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 14:04:41 +00:00
weichen 5623000d51 Publicize identity CLI as pkg/identity for downstream reuse
ci / vet-and-test (pull_request) Successful in 43s
Why: runIdentity lived in cmd/agent-system package main, so downstream
combination binaries (Agent-System-RDM) cannot reuse it and their unknown
subcommands fall into the serve path; the 2026-10-12 rdm1/rdmcand batch
update failures traced back to that capability gap.
Evidence: go test ./pkg/identity/... ok; CGO_ENABLED=0 go build ok;
behavioral probes (usage rc=2, capabilities v1, ensure/verify roundtrip)
recorded in the PR.
Impact: identity semantics unchanged for the base binary; adds
identity capabilities (v1) and usage-on-unknown contract shared by all
products; no storage changes.
Refs: Dev PR #474 (svc.sh candidate gate); user root-cause report
2026-10-12 U1 (shared identity interface).
2026-10-12 13:10:13 +00:00
weichenandCursor e7b70c24f8 Renumber agent_system_identity to v27 after login_keys took v26
ci / vet-and-test (pull_request) Successful in 42s
Why: main #171 claimed migration v26; rebase conflict required
bumping the U1 I3 table and completing copyOrder for both new tables.
Evidence: go test ./internal/store/ ./cmd/storage-migrate/ -run
'SystemIdentity|CopyOrder'
Impact: migrate v27; copyOrder += login_keys + agent_system_identity.
Refs: Backend PR #170; conflict with main 1d9e8f4

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:04:16 +00:00
weichenandCursor dcd5445d61 Include agent_system_identity in storage-migrate copyOrder
Why: TestCopyOrderCoversAllFixtureTables fails when new schema tables
are omitted from the migrate tool inventory.
Evidence: go test ./cmd/storage-migrate/ -run CopyOrder
Impact: copyOrder gains agent_system_identity (no FK deps).
Refs: U1 I3; Backend PR #170

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:03:45 +00:00
weichenandCursor e046bbd15c Add agent_system_identity marker with ensure/verify CLI
Why: U1 I3 needs a DB-side instance identity anchor matching
manifest.instance_id so wrong/missing DB blocks startup (I4).
Evidence: go test ./internal/store/ ./internal/app/; CLI smoke
ensure/verify on SQLite temp dir.
Impact: migration v26; store Ensure/Verify; identity subcommand;
serve checks AGENT_SYSTEM_INSTANCE_ID when set.
Refs: docs/recovery/u1-identity-manifest.md §5.2; Dev feature/recovery-u1-db-identity

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:03:45 +00:00
weichenandCursor 1ba7225e83 Add one-time login keys, restricted sessions, and OpenExisting
ci / vet-and-test (pull_request) Failing after 56s
Why: passwordless activation/recovery must redeem only into a
set-password session; account ops must not create empty stores.
Evidence: SELF go test store/server login-key and OpenExisting cases.
Impact: auth middleware gates; CLI user create defaults to activate key.
Refs: docs/svc-deploy/u4-one-time-login-key.md

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:44:16 +00:00
weichen 5d5cef9fcd Add model identity and in-database vector top-k via pgvector (U6 slice 3)
ci / vet-and-test (pull_request) Successful in 56s
Why: plan U6 vector slice (KTD11 correctness step) - same-dimension vectors from different models must never mix, and PG tiers should run filtered exact top-k in the database instead of scanning every row in-process.

Evidence: full suite green on SQLite and real PostgreSQL 18.6 with pgvector (35 packages each tier) - pgvector_test covers model-identity and dimension isolation, update/delete atomicity (BLOB and vec_pg swap together), shared top-5 with score monotonicity, Recall@10 = exact-scan parity, deny-all predicate leakage; storage-migrate tests green after excluding the derived vec_pg column.

Impact: v25 migration adds model/content_hash columns and (PG, best-effort DO block) the pgvector extension plus vec_pg column - absence degrades to in-process scan; UpsertKnowledgeVectorExt dual-writes BLOB (cross-tier authority) and vec_pg; SearchKnowledgeVectorsModel/SharedModel filter by model identity and use SQL exact top-k (ORDER BY vec_pg <=> q) with automatic degradation; embedders expose Model() and the rag pipeline threads it through indexing and both retrieve paths; storage-migrate excludes vec_pg as a derived column.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U6 slice 3, KTD11; R9.
2026-10-11 16:31:12 +00:00
weichen f5eb27df86 Add the surface action pipeline: ledger, guards, dispatch, submit (U5)
ci / vet-and-test (pull_request) Successful in 1m10s
Why: The approved U5 scope needs the action half of the loop: an
idempotent ledger (migration v24), a single-choke SubmitAction (plugin
activation, current-revision, action whitelist, params-hash
idempotency), dispatch through the command bus so DangerHigh lands on
the existing inline confirm card with no Confirmed passthrough (the
endpoint does not even bind the field), the button-only
generative-ui.submit storing form data in the plugin KV ledger, and
catalog discipline rejecting unknown action ids at present time.

Evidence: go build/vet clean; full go test ./... green; surfacesvc/
genui/store suites also green under AGENT_PG_TEST_DSN; red matrix
covers disabled/stale-revision/cross-owner/unknown-action/idempotent
same-and-different-params/danger-high-await-confirm and the endpoint
forged-confirmed case.

Impact: Additive migration v24 + new endpoint/command; existing
endpoints and command semantics untouched (B-class whitelist starts
empty per the approved scope).

Refs: U5 action-scope r2 (approved, Documents #71); UI #46/#47;
Frontend #56; Dev task feature/generative-ui-u5
2026-10-11 14:24:31 +00:00
weichen 603f1c0fa4 Add execution ownership leases with epoch fencing and persistent run commands (U5)
ci / vet-and-test (pull_request) Successful in 57s
Why: plan U5/KTD4+KTD5 - before a second instance may serve writes, conversation execution must have a single valid owner (DB lease, epoch increments on takeover), stale owners must not settle (in-transaction fence check), and cancel/HITL must reach the executing instance via a persistent command inbox.

Evidence: full suite green twice-relevant on SQLite and real PostgreSQL 18.6 (35 packages each tier) - lease tests (claim/contention/re-claim/renew fencing/expiry takeover/monotonic epoch, command inbox idempotent enqueue, atomic concurrent claim) and multi-instance tests (same-conversation contention rejected, stale-owner settle fenced with run unchanged, cross-instance cancel consumed exactly once by the owner instance, non-owner does not consume).

Impact: v23 migration (conversation_executions + run_commands); ClaimConversation stacks a DB lease on PG tiers (15min, release on conversation release); settle path fenced (fenced runs emit run.fenced, no terminal write); StartCommandConsumer polls the inbox for commands on locally-owned executions (cancel routes to local cascade or DB-level close-out; HITL revalidates gate and rejects expired explicitly); server cancel/HITL fall back to command enqueue with 202 when the run is not local; app wires instance identity (host:addr on PG tiers) and starts the consumer.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U5 KTD4/KTD5; R4/R10.
2026-10-11 12:25:31 +00:00
weichen 1d5672e4fe Cover the surface tables in storage-migrate's copy list
ci / vet-and-test (pull_request) Successful in 40s
Why: the copy-order coverage guard caught that backend PR #143's new tables (surfaces, surface_revisions, plugin_activation) were absent from the migration list - they would have been silently skipped on real migrations.

Evidence: TestCopyOrderCoversAllFixtureTables failed on the merged main and passes with the addition; full migrator suite green on SQLite and real PostgreSQL.

Impact: copyOrder list only; ordered after their referenced owners/projects.

Refs: backend PR #143; Dev U4 task.
2026-10-11 08:32:45 +00:00
weichen 093b86fe7e Add the storage-migrate tool: inventory, backup, and identity-governed migration (U4)
ci / vet-and-test (pull_request) Successful in 38s
Why: plan U4/R7 - the SQLite-to-PostgreSQL path needs a managed entry that never replays SQLite dumps as PG SQL, fixes migration identity against a frozen snapshot, refuses foreign or non-empty targets, verifies per-table counts and ordered checksums, and keeps secrets out of reports (KTD14).

Evidence: integration tests against real PostgreSQL green twice (end-to-end with bool/NULL/Unicode/BLOB/sequence assertions, idempotent rerun, interrupted-table resume, foreign-data and wrong-snapshot refusals, unknown-owner block); SQLite unit tests for inventory/backup/coverage; full backend suite green on both tiers.

Impact: new cmd/storage-migrate binary (make build now emits bin/storage-migrate); store gains a DB() accessor documented for ops tooling; no server or runtime behavior change.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U4 and section 6; docs/guides/team-storage.md section 10.
2026-10-11 08:30:01 +00:00
weichen 8633711b1d Add the team PG tier, atomic migrations and the connection budget (U2)
ci / vet-and-test (pull_request) Successful in 38s
Why: the approved collaboration plan U2 (KTD1/KTD8/KTD12) requires an explicit fail-closed team deployment tier, DDL-and-ledger-atomic migrations serialized across migrators, and a configurable PG connection budget with sanitized effective values.

Evidence: real PostgreSQL 18.6 + pgvector + Redis 8.10.2 local gate - full backend suite green twice with AGENT_PG_TEST_DSN/AGENT_REDIS_TEST_ADDR (isolated svc.sh instance collabtest:8897, redis 6399); SQLite suite green without env.

Impact: config tier+pool keys and validation; store PoolOptions + statement_timeout DSN param + single-transaction migration with PG advisory lock and SQLite _txlock=immediate + unknown-schema-version refusal; app tier guard fail-closed (team requires DSN+auth, no SQLite fallback); CLI flags and explain rows; tests for tier/pool parsing, atomicity fault-injection, racing migrators, old-schema upgrade; fixed pre-existing real-PG test defects (blind &search_path= appends in runmgr/pgbus helpers, pg_trgm per-schema install race, exhausted seq-conflict retries).

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U2, KTD1/KTD8/KTD12; R2/R3/R7.
2026-10-11 03:10:55 +00:00
weichen 01fd3b6fd4 Add the S0 task workspace overlay: frozen base, private upper, CAS apply
ci / vet-and-test (pull_request) Successful in 43s
Why: The sandbox tier wrote the session directory directly, so a buggy
or runaway script could leave the workspace half-mutated and nothing
bounded what a timeout had already written. The plan's first landing
shape (section 4.4) swaps the session bind for a frozen-base overlay:
each exec freezes the session dir (hardlink clone + sha256 baseline),
runs on a private upper, and only a clean exit promotes the change set
back via CAS (baseline hash match -> atomic replace/remove; externally
changed targets double-store as *.conflict-* instead of clobbering).
Timeouts and kills discard the upper entirely - half-written state
never lands.

Evidence: go test ./... green including bwrap integration roundtrips
(TestSandboxWorkspaceOverlayRoundtrip: add/modify/delete through a real
overlayfs upper with whiteout handling and ws cleanup;
TestSandboxWorkspaceOverlayTimeoutDiscards: partial writes discarded on
timeout) and unit anchors for freeze manifest (.agentx excluded), CAS
add/modify, conflict double-store, and the change summary.

Impact: Opt-in via --exec-workspace overlay / AGENT_SYSTEM_EXEC_WORKSPACE
(only with --exec-backend sandbox; fail-closed on unknown modes). The
model sees a one-line change summary appended to the tool output, so
discards and conflict files are visible. Default behavior is unchanged.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md section
4.4; Dev task feature/agent-s0-workspace-overlay.
2026-10-11 02:04:05 +00:00
weichen 4df81ef1da Enable parallel tool execution, fix delegate truncation, sandbox deps layer
ci / vet-and-test (pull_request) Successful in 43s
Why: Real runs were fully serial: the direct path never set the kernel's
parallel tool mode, delegate_task handed the parent only the child's
160-char summary (the full text was dropped), MiniMax streamed its
reasoning inside content <think> blocks, and the EXT-16 sandbox hid
~/.local so python-docx/openpyxl/pptx could not even import. All four
were measured defects (F1/F2/F3/F5) in the delegation/routing proposal.

Why: Serial-only mutual exclusion is not enough for order-sensitive
tools - the card-stream test failed with out-of-order appends - so the
ordering moved into the kernel (SerialTools partition, Agent-Native
#44); process events pair by CallID (#42) and tool results write back
through the injected call context (#43).

Evidence: go test ./... green, including new regression anchors:
TestDelegateToolReturnsFullText, TestToolEventsPairByCallID,
TestToolExecModeAndSerialSet, TestOpenLLMClientReasoningSplitAuto,
TestSandboxArgvDepsLayer, TestExecEnvJoinsPythonPath.

Impact: Direct runs now execute independent same-turn calls
(searches, reads, delegate_task, review_findings) concurrently by
default (runtime.parallelTools=false restores sequential); card flow,
plans, workflows, exec primitives and file edits keep model-declared
order. delegate_task returns the capped full child text. MiniMax-*
models get reasoning_split=true so thinking streams via
reasoning_content. --exec-deps-dir/AGENT_SYSTEM_EXEC_DEPS_DIR mounts a
read-only dependency layer into the sandbox and merges PYTHONPATH.
Bumps deps/agent-native to c348441.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md (E1,
F1-F5); Agent-Native #42/#43/#44; Dev task feature/agent-e1-parallel-fixes.
2026-10-10 09:02:10 +00:00
weichen 4badecbf57 Add the EXT-16 exec backend interface: direct status quo and bwrap sandbox
ci / vet-and-test (pull_request) Successful in 41s
Why: Q2 ruled exec tools stay disabled for now but code execution must
return later; the recovery path treats model-generated code as
untrusted. This abstracts the interpreter launch point in
exec_interpreter.go into an ExecBackend port (direct = byte-identical
status quo; sandbox = bwrap isolation) so downstream products can
re-enable execution behind isolation without further base surgery.

Evidence: go build/vet clean; go test ./... 32/32 packages green;
sandbox escape negatives all blocked locally (bwrap 0.11.1): home
secret unreadable (tmpfs /home), sibling session unreadable
(session-only bind), network blocked (--unshare-net), 400-process
spawn storm capped by ulimit -u 256 (fork: retry), 60s sleep killed at
3s timeout, orphans die with the namespace; session-dir write visible
on host; direct backend regression (exit=7 / exit=timeout texts)
unchanged.

Impact: run_shell/run_python route through the backend; workflow
ScriptCommand and the squad kernel ExecConfig stay direct (follow-up
scope); config key execBackend + env AGENT_SYSTEM_EXEC_BACKEND + CLI
--exec-backend; extension.Options.ExecBackend (SurfaceVersion
2.6.0-experimental); DisableExecTools remains the top-priority
fail-closed switch.

Refs: proposal 2026-10-09-ext-g1-b1-b3-exec-backend.md section 4
(EXT-16, G2-8 prototype); Backend #110 (DisableExecTools).
2026-10-09 17:01:32 +00:00
weichen 68c1a7272a Keep tracked-change and hyperlink text in DOCX and add an exec hard switch
ci / vet-and-test (pull_request) Successful in 43s
Why: The DOCX parser only read w:r runs that were direct children of w:p, so
text inside w:ins, w:hyperlink and w:fldSimple was dropped with no warning,
violating the no-silent-degradation rule. Separately, runtime.exec is a
user-writable setting, so multi-user deployments could not reliably turn the
exec primitives off.
Evidence: SELF - red tests TestParseDocxTrackedChangesAndHyperlinks and
TestExecHardDisabledOverridesSettings failed before and pass after; backend
go test ./... (32 packages) passes; a mock-mode binary started with
--disable-exec-tools hides run_shell/run_python/register_artifact from
/api/tools even after the user sets runtime.exec=true.
Impact: DOCX paragraphs are decoded from the token stream at any depth;
deleted/moved-from text, field codes and mc:Fallback are excluded and
revised documents carry an explicit warning. New Options.DisableExecTools
(SurfaceVersion 2.5.0-experimental), CLI --disable-exec-tools, config key
disableExecTools and env AGENT_SYSTEM_DISABLE_EXEC_TOOLS; zero value keeps
current behavior.
Refs: Agent-System-RDM docs/proposals/2026-10-09-V22-review-and-implementation-plan.md (G0-7, B-5)
2026-10-09 10:36:02 +00:00
weichenandCursor 8eda35f365 Refuse open listeners and cross-site writes without auth
Why: The embedded tier starts without auth, and nothing checked Origin or Host, so a public bind or a foreign page could call write APIs.
Evidence: go test ./internal/config ./internal/server covering listen policy, origin rejection, and response headers.
Impact: Non-loopback without auth exits before opening the database. HTML artifacts get a sandbox CSP. Reverse proxies must list their public host in allowedHosts.
Refs: feature/sillytavern-review
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-04 17:28:46 +00:00
weichenandCursor 44aa764b56 Recover interrupted runs and commit terminal state once
ci / vet-and-test (pull_request) Successful in 43s
Why: A hard kill left runs in pending, running, or waiting_hitl, a busy send stored a message before the 409, finishRun could mark a run completed without an assistant message, and a full SSE buffer dropped events without closing the stream.
Evidence: GOWORK=off go vet ./... and GOWORK=off go test -count=1 ./... passed, including store, runmgr, bus, and server recovery tests.
Impact: Migration v14 adds runs.instance_id and messages.request_id. Startup reconciliation and shutdown share one seal. Turn and workflow sends claim a slot before writing. Duplicate request ids return the existing message. SettleRun updates the run and appends the assistant message in one transaction. A full subscriber buffer closes the channel so SSE can replay from the cursor.
Refs: Dev feature/pi-durable-borrow
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-03 14:37:49 +00:00
weichen 9a545b1e69 Add admin-issued accounts with forced first-login password change
Why: the user ruled out self-registration (2026-09-30): accounts are
created only by admins who set an initial password; first login (and
first login after any admin reset) must force a password change.

Evidence:
- migration v10 auth_accounts (users password/must_change/is_admin +
  sessions table; upgrade test walks v1<->v10 both directions).
- argon2id hash/verify with PHC encoding; store tests cover hash
  roundtrip, duplicate-owner conflict, must-change lifecycle, session
  keep/except semantics.
- e2e TestAdminAccountFlow: static login page reachable without auth,
  wrong password 401, login sets HttpOnly cookie + mustChangePassword,
  admin creates user (409 on duplicate), non-admin create 403, forced
  change clears flag and old password dies, admin reset revokes sessions
  and re-arms forced change, API-key channel untouched, 5-fail backoff
  429, logout kills session. Full go test ./... green.

Impact: middleware now accepts Bearer key OR session cookie and exempts
static + /api/auth/login (also fixes the latent case where the login
page itself could 401). Embedded no-auth mode unchanged.

Refs: proposal 2026-09-30-ui-redesign-and-auth.md section 5 (admin-issued
variant, user decision 2026-09-30)
2026-09-30 11:15:22 +00:00
weichen 66b4bbf55a Add the startup configuration pipe with file and env layers plus explain
Why: startup had only defaults and CLI flags; dev/test/prod operation needs
the standard baseline (explicit file, environment, explicit-only CLI) and a
read-only diagnostic.
Evidence: internal/config unit tests cover precedence, unset-CLI no-override,
presence-based file semantics, unknown-key/missing-file errors, env bool
parsing, explain source labelling and DSN redaction; full suite 12 packages
pass; live run of config explain with envs/prod.json shows correct layer
attribution and redaction.
Impact: serve gains --config and AGENT_SYSTEM_* env overrides (behavior
unchanged when neither is used); new config explain subcommand; envs/
examples and PROJECT.md configuration section.
Refs: documents docs/proposals/2026-09-30-configuration-surface.md; standards/configuration.md
2026-09-30 04:19:11 +00:00
weichen 8ae5b2334a Fix apikey subcommand flag parsing (flags must follow the subcommand) 2026-09-30 02:39:50 +00:00
weichen 02e2036fc6 Add dual-tier storage (embedded SQLite + optional PostgreSQL)
Details (original message):
Add dual-tier storage (embedded SQLite + optional PostgreSQL), tenant owner scoping, API key auth, versioned dual-dialect migrations, integrity hardening, and batched event persistence
2026-09-30 02:39:50 +00:00
weichen 0d74ec94dd Implement the backend: store/bus/runmgr core
Details (original message):
Implement the backend: store/bus/runmgr core, mock and LLM runners (agent-native client), HITL three-branch routing, subagent runs, builtin and declarative HTTP tools, ingest parsers (md/txt/docx/pdf-text), knowledge fulltext, layered settings, profiles, plugin host, REST+SSE API, single-binary entrypoint; e2e and unit tests.
2026-09-29 17:53:24 +00:00