Commit Graph
13 Commits
Author SHA1 Message Date
weichen c170939bb7 Implement the U1 identity manifest layer with startup verification and tests
Why: approved design (2026-10-12) - persistent manifest becomes the startup authority, closing the param-rebuild drift, non-atomic per-key writes, and missing-config-creates-empty-system recovery gaps (plan r2 U1 / KTD1-KTD2).

Evidence: SELF - python3 tests/test_svc_identity.py 21 tests OK (1 skip = real-PG identity NOT_RUN by design); make test-governance 169 OK incl. existing svc db-binding and autorecover suites; receipts appended via make evidence.

Impact: scripts/svc.sh gains manifest-aware set/del/record_state/autostart plus lock-held migrate, pre-launch identity verify and an init action; new scripts/svc_manifest.py (atomic temp-fsync-rename, identity anchors, derived svc.env); unmigrated instances keep the exact legacy path (regressions green).

Refs: docs/recovery/u1-identity-manifest.md; plan r2 section 7 U1
2026-10-12 02:50:32 +00:00
weichen c43ccd7481 Draft the U1 identity manifest contract, data ownership inventory and card
Why: U1 starts per the approved recovery plan r2 - a reviewed design contract must exist before touching scripts/svc.sh, and R3 needs an enumerable data-ownership baseline.

Evidence: SELF - doc-check receipt via make evidence. First run FAILED (exit 1: proposal r2 not present in this worktree because it sits unmerged on feature/service-restart-recovery-plan); references reworded to cite the path with a pending-merge note and the check re-run PASS (exit 0). Both receipts are in the card; grounding facts verified at baseline 89174a0.

Impact: docs/recovery/ additions and task card only; no behavior change; implementation gated on design review.

Refs: modules/documents/docs/proposals/2026-10-12-service-restart-data-recovery.md r2 U1
2026-10-12 01:39:20 +00:00
weichen c514bc4f4d Merge authoritative main into the task branch
Why: include the current integration baseline before verification.

Evidence: fetched origin/main=3de5791e0841e9c7c40c3dc236be6d95b3969827.

Impact: integrate upstream changes without rewriting task history.

Refs: task-sync
2026-10-11 21:56:13 +00:00
weichen 419380187d Add an rdm deploy source to svc (downstream composite binary)
Why: RDM capabilities (two-trees-one-net, rdm.* trace-audit command face)
only exist in the downstream Agent-System-RDM composite binary; svc's three
existing sources all produce the Dev binary, so none of it could be deployed
or lifecycle-managed. This adds --from rdm[@<sha>] with the same isolated
source-repo semantics as the dev source.

Evidence: make test-governance 138/138 (two new stub cases: deploy E2E and
commits/update track); real deployment rdm@2aef3fc recorded in the task card
(pins materialized, pure-Go build, web workbench to htdocs, serve health OK,
rdm.ping/rdm.inspect.capabilities committed, update --check reports latest);
make test full suite exit=0. All segments written by make evidence.

Impact: scripts/svc.sh (rdm source resolution/materialization/deploy/update/
commits, probe timeouts, rdm seed skip, ELF env-DSN evidence fallback),
docs/svc.md 5.4, tests/test_svc.py. RDM-side CLI alignment landed as RDM
PR #228; no base submodules touched.

Refs: task card .task/feature-svc-rdm-source.md; RDM PR #228/#229.
2026-10-11 20:46:03 +00:00
weichen eda942523c Add svc auto-recovery: autostart desired-state, recover, watchdog
Why: svc services are bare nohup'd processes with persisted registration
but no supervision -- a crash, an OOM kill, or a reboot left them dead
until manual restart (2026-10-11 outage: dev service lost, runtime/ also
wiped by hand, nothing brought services back).

Evidence: tests/test_svc_recover.py 10 cases green (start records
autostart, stop clears it incl. dead-process stop, SIGKILL then recover
revives with persisted exec=, lock-held skips, port-stolen fails with
recover fail event, watchdog unit text via --print); make test-governance
146/146 OK locally; real-machine watchdog install + kill -9 revival to be
recorded on the task card.

Impact: svc.sh gains autostart key in svc.env, `recover [name]
[--quiet]` (postgres-first, per-service .start.lock, recover history
events), `watchdog install|remove|status` (systemd user timer, cron
fallback hint), lifecycle lock across start/stop/restart/deploy/update;
spawned services no longer inherit the lock fd (9>&-), and lock release
no longer clobbers script stderr. Makefile adds svc-recover/svc-watchdog;
docs/svc.md documents semantics and boundaries.

Refs: task card .task/feature-svc-autorecover.md; docs/svc.md 6.5/6.6;
parallel planning card feature/service-restart-recovery-plan (same
problem space, plan-stage).
2026-10-11 20:20:27 +00:00
weichen 3b8fef7fad Recover svc.sh database config across restarts and refuse on invalid binding
Why: the exec= reuse path stripped --dsn into the process environment (G1-7)
and record_state then persisted a DSN-less exec=, while the dbref fallback only
existed in the explicit-rebuild branch - the second bare restart after a
rebind silently dropped the database (SQLite fallback on loopback). Auth
preflight also matched --dsn on the command line only, so env-delivered DSN
with auth=auto was misjudged as unauthenticated.
Evidence: bash -n OK; make test-governance 115 tests OK (new tests/test_svc.py
11 cases; 10/11 fail against the pre-fix script); make test green; make build
OK; isolated real PostgreSQL 18.6 + real binary validation 27/27 (consecutive
restarts/deploy keep binding, psql-verified database identity, DB-down restart
fails without SQLite fallback or config loss, no credential leakage) - all
recorded via make evidence in the task card.
Impact: svc.env gains db=/dbref=/dsn= recovery keys (atomic 0600 writes);
start/restart/deploy/update/db share one resolver; invalid bindings, config
conflicts, mock coexistence and env-DSN-incapable binaries now refuse to
start instead of silently downgrading; legacy exec= DSN migrates idempotently;
docs/svc.md and script help updated.
Refs: task card .task/feature-svc-db-config-recover.md; user-approved plan
(2026-10-11 session message); AGENTS.md evidence gates.
2026-10-11 09:16:54 +00:00
weichenandCursor f8e06090e6 Pin the SillyTavern borrow across the linked repositories
Why: The research record, HTML cleaning, listen guard, secret store, and context controls land together, and the parent pointers have to name those commits.
Evidence: Child commits are core bd63223, agent-native 3a110a7, ui d76a8e6, frontend 36b92ed, documents f3f823e, backend 42cca8f. svc.sh forwards the listen switches and info prints them.
Impact: A reverse proxy or a LAN hostname needs allowedHosts or an unauthenticated host check returns 403. Startup moves raw API keys into the secret table after a settings snapshot.
Refs: .task/feature-sillytavern-review.md
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-04 18:21:17 +00:00
weichen ecfed7be31 Add service config-file switching with schema validation to svc.sh
Why: beyond database binding, services needed a way to switch the JSON config
file they run with and have the new config take effect promptly; hand-editing
exec= risks restarting a service into an invalid config.

Evidence: in the task worktree with a real agent binary -- a config with an
unknown key is rejected by the binary's strict schema check before any state
change; a valid file binds via --config in exec= + conffile= (survives
record_state rewrites), restarts the running service, and the config's dsn
verifiably drives the process (wrong-password variant failed PG auth in the
log; correct variant connected and created 21 tables); editing the file and
re-running 'config <same file>' re-validates and restarts (pid changes);
'--clear' removes both the flag and conffile; db and config coexist in exec=;
view mode shows masked JSON content plus 'config explain' effective values
with masked dsn; list DB column also resolves dsns that live only inside the
config file. Full log in the task card.

Impact: new service-level action 'svc.sh <name> config [file | --clear]
[--no-restart]' for agent services; config-file content snapshots (masked,
--reveal to show) in info; a config-file line in status; conffile persisted
in svc.env. No agent binary changes.

Refs: task card .task/feature-svc-db-bind.md (same doc as PR #182);
docs/svc.md sections 2/4.5/4.6/6/10.1
2026-10-04 13:42:51 +00:00
weichen 843495e0d0 Add per-service database binding and fix single-dash flag detection in svc.sh
Why: info showed services and postgres instances but no service-to-database
correspondence, and switching an agent service to another database required
redeploying with a hand-built DSN. While wiring this up, adjust_args_for_bin
was found matching Go flag help with '--dsn' although Go prints '-dsn', so
deploy-built --dsn/--auth args were silently dropped on current binaries.

Evidence: full E2E in the task worktree with a real agent binary and a real
PostgreSQL 18 instance -- 't-app db bpg' persisted exec=/dbref, auto-restarted
(dbref survives record_state rewrites), and the agent created 21 tables in the
bound database; 'db bpg --db t2' auto-created and switched databases (21
tables in t2); '--clear' returned the service to embedded SQLite; --no-restart
and error paths behave; list gains a DB column, status a database line, info a
reverse bound-services line; legacy deployments without dbref resolve by DSN
host:port; single-dash detection proven with positive and negative cases;
regressions list/db dsn/db ls/info/history/--help pass; make test exit 0.
Full log in the task card.

Impact: new service-level action 'svc.sh <name> db [pg [--db X] | --clear]
[--no-restart]' for agent services (bind/switch/unbind/view), DB column in
list, binding lines in status/info, dbref persisted in svc.env, auto database
creation when the instance runs, and the adjust_args_for_bin fix restoring
deploy --dsn/--auth on current binaries. No agent binary changes.

Refs: task card .task/feature-svc-db-bind.md (same doc as the PR); docs/svc.md
sections 4.5/6.1/6.2/10.1
2026-10-04 13:30:50 +00:00
weichen 106ef07ef0 Add read-only svc.sh info report for services, databases, and config snapshots
Why: svc.sh could show a service table (list), per-instance DB names one at a
time (db ls), and per-service status, but had no single view answering "what
is running, which databases exist, and what is configured" -- the questions
asked when inspecting or handing over a machine.

Evidence: exercised in the task worktree runtime -- empty-state report; fake
agent/static/postgres registrations proving masking (exec DSN password,
password=, SVC_TAVILY_KEYS -> ***, non-secret values verbatim; --reveal shows
plaintext); a real postgres instance (db create r-pg --port 18901, start,
extra orders db, one backup) whose info section lists inner databases with
sizes, pgdata size and backup stats; regressions list/history/--help/help
lifecycle/make svc-info; info stdout carries the full report (stderr 0 bytes)
so it can be redirected as a snapshot. Full log in the task card.

Impact: new read-only `scripts/svc.sh info [name] [--tag T] [--reveal]`
command (also `make svc-info`): service table + running summary, postgres
instances incl. inner databases, per-service svc.env content snapshots plus
global .svc.env / --config / exported SVC_* vars, and recent events. Secrets
are masked by default. No deploy/lifecycle/db-ops behavior changes; `info`
becomes a reserved service name; docs/svc.md gains section 6.2 (former
6.2-6.4 shifted to 6.3-6.5).

Refs: task card .task/feature-svc-info.md (same doc as the PR); docs/svc.md
2026-10-04 12:42:49 +00:00
weichen a42fff69e4 Deploy svc updates from an isolated remote source clone
Why: svc.sh update skipped services with a local@<sha> source whenever the
main checkout was dirty, and bare deploy built the current workspace. The
owner requires updates/deploys to be independent of the working tree
state, defaulting to the latest remote commit built outside the checkout.

Evidence: bash -n passes; commits dev lists from the source clone; bare
deploy --dry-run plans dev@<origin/main>; end-to-end "svc.sh update
svc-selftest" with a dirty tree updated a local@d64742c service to
dev@0be23f9 (built in runtime/.worktrees from
~/.cache/agent-system-svc/src, started healthy), then stopped and purged.
Full make test output recorded on the task card.

Impact: deploy now defaults to dev@<origin/main> resolved in the
script-owned clone (SVC_SRC_DIR, default ~/.cache/agent-system-svc/src);
--from local stays the only workspace-dependent source; update no longer
pulls the main checkout - local@/dev@ services redeploy from remote
latest, ahead/diverged/missing-on-remote still decline to act.
docs/svc.md and in-script help updated to match.

Refs: .task/svc-remote-source.md
2026-09-30 18:39:25 +00:00
weichen ac8db54ecb Add svc.sh update command: auto-detect repo advances and refresh services
Why:
After the repository moves forward, updating a running service meant
manually figuring out the new commit, rebuilding and redeploying by hand;
the user asked for the tool to detect and update automatically.

Evidence:
- "update [name...] [--check] [--all] [--no-fetch]": fetches origin,
  compares each service's registered source against the latest (Dev repo
  origin/main for local@/dev@ sources, newest Releases version for
  release@), prints a per-service table (current / latest / state), and
  without --check refreshes outdated running services by reusing the full
  deploy pipeline (stop → build/verify → start → health check; persisted
  exec= config and tags survive; update events appended to history).
- local@ services: auto git pull --ff-only origin main + make sync before
  rebuilding, guarded three ways (current branch == main, clean worktree,
  ff-only) — dirty trees or non-main branches are skipped with a warning,
  never touching uncommitted work.
- dev@ sources go straight to the remote main commit via an isolated
  worktree; release@ sources deploy the newest Releases version with
  sha256 verification; postgres and static --dir services are reported as
  not applicable; sources that diverged from or lead the remote are never
  force-updated.
- Verified end to end with five test services: dev@73b2802 → dev@ffda6d3
  (real isolated build, HTTP 200, tag/port preserved, worktree recycled),
  non-main-branch skip, stopped-service warning vs --all, release
  sha256-verified update, --check read-only (svc.env checksums unchanged),
  unregistered name exits 2.
- Fixed en route: a latent bug from the v2 refactor — materialize_dev_commit
  leaked submodule/make stdout into the captured worktree path, breaking
  every real dev@sha deploy since (cp: File name too long); build output
  now goes to stderr. Also fixed an unbound-variable crash in the update
  loop (local statement self-reference under set -u).

Impact:
New read-mostly operational command; no behavior change to existing
commands except dev@sha deploys now actually working end to end.
Makefile gains svc-update; --help and docs/svc.md document the command.

Refs: .task/feature-svc-auto-update.md(任务卡=PR 描述)
2026-09-30 15:22:33 +00:00
weichen 2c7f55bfe8 Document svc.sh comprehensively and layer its help system
Why:
svc.sh grew into a three-kind service manager with database operations,
but its only documentation was a flat --help wall of text; the user asked
for a complete usage document and a better help experience.

Evidence:
- docs/svc.md: 14-section manual (prerequisites, quick start, concepts
  and directory layout, per-command reference for service/global/db
  commands, deploy sources, database operations handbook, configuration
  layering, registry fields and event types, safety boundaries, exit
  codes, troubleshooting table, recipe index) — every claim cross-checked
  against the implementation from the previous two rounds.
- Help layering: no-args and -h print a one-screen overview; --help
  renders the header comment reorganized into 7 labeled sections; the new
  "help [topic]" command deep-dives eight topics (model, deploy,
  lifecycle, db, dbops, config, examples, safety); usage errors now show
  the short overview plus a pointer to --help / help / docs/svc.md.
  All topics verified to render; unknown topic exits 2.
- No runtime behavior changed: end-to-end db lifecycle regression
  (create → start → backup → status → stop → rm --purge) passes after the
  refactor.
- Makefile help and README's documentation section point to docs/svc.md.

Impact:
Documentation-only plus display-layer changes; command semantics,
exit codes and file formats are untouched. Maintenance note: behavior
changes must now update both --help and docs/svc.md.

Refs: .task/feature-svc-docs-help.md(任务卡=PR 描述)
2026-09-30 14:29:39 +00:00