Compare commits

...
308 Commits
Author SHA1 Message Date
weichen 5d5fc9891c Merge pull request 'Enable private member work in shared projects' (#195) from feature/project-member-access into main
ci / vet-and-test (push) Successful in 45s
2026-10-13 11:52:27 +00:00
weichen d26f3cf2e8 Protect private project event replay and live delivery
ci / vet-and-test (pull_request) Successful in 49s
Why:
Legacy project streams replayed upload filenames and parser metadata to newly
authorized members even though the underlying cards remained owner-only.

Evidence:
SELF real HTTP red regressions exposed legacy replay, spoofed sharing claims,
private-history pagination, and live delivery after membership removal. The
affected packages, uncached full Backend suite, isolated PostgreSQL Store suite,
and agent-system build pass after applying the event boundary.

Impact:
Registered lifecycle streams resolve trusted resource ownership and require an
internal publication proof for cross-owner delivery. Event and proof commit in
one transaction; v30 adds an additive SQLite/PostgreSQL sidecar without granting
access to historical events. Seen and visible cursors remain separate, revoked
members leave live streams, and unregistered base behavior remains compatible.
The storage migration tool preserves the new proof rows.

Refs:
project-member-access-implementation-plan.md Task 1 / R5; authorized legacy SSE
privacy follow-up; project-member-composed-legacy-project-history.json
2026-10-13 11:12:47 +00:00
weichen d2f96e79bb Admit private member uploads and preserve masked metadata edits
Why:
Nested unchanged credential masks left empty parents that looked like managed
subtree overwrites. Shared members also hit an owner-only upload guard, while
upload ingestion cards could disclose private filenames and parser metadata.

Evidence:
SELF independent nested-mask probe, HTTP nested/mixed writeback, member upload,
project card privacy and authorization-change regressions failed before each fix
and passed after it. Affected settings/server suite passed; exact final-object
build and independent replay remain delivery gates owned by the task coordinator.

Impact:
Prune only parents emptied by ignored masks; explicit empty/real managed writes
stay rejected. Registered uploads check live work and trusted project existence,
keep originals owned by the actor, and recheck before persistence. Suppress only
upload project cards when no private stream exists; explicit structure cards and
unregistered upload behavior remain available.

Refs:
Dev project-member-access plan r1 Task 1/R5; independent nested mask P2;
2026-10-13 composed member-upload and private project-stream acceptance findings
2026-10-13 10:23:37 +00:00
weichen c4435d1415 Protect shared project metadata credentials and managed settings
Why:
Legacy project metadata can contain credentials outside the settings scope table.
Sharing raw projection settings exposed them, and generic project PATCH bypassed
the system-managed path guard already enforced by project settings endpoints.

Evidence:
SELF member Host regression failed before the fix. The fixed 4f72aaf server
baseline overlay reproduced HTTP metadata leakage and managed PATCH returning 200.
SELF shared Host-to-HTTP, immutable mask, dotted/schema/fallback, public mutation
and atomic managed-write regressions passed; uncached full suite, isolated PG
Store suite and static build passed. Independent delta replay remains a gate.

Impact:
Registered lifecycle metadata returns detached credential-masked views while
preserving service/configuration keys and persisted data. Generic project PATCH
shares the existing managed validator before rename or settings mutation; allowed
metadata key shapes and raw owner-only management lookups remain available.

Refs:
Dev project-member-access plan r1 Task 1 and R5; approved hard secret boundary;
2026-10-13 shared project metadata review follow-up
2026-10-13 09:46:20 +00:00
weichen 4f72aaf3d8 Recheck resumed project work and isolate publication replacement
Why:
Membership could be revoked while a tool waited for confirmation or a queued
HITL reply waited for consumption. Publication replacement also crossed an
empty project scope or another plugin's source namespace for the same Ref.

Evidence:
SELF replay of all four independent review reproductions failed before the
fix and passed unchanged afterward. Committed regressions cover direct and
queued HITL, yes/session confirmation, direct artifact writes, protected vectors,
exact empty project scopes, literal wildcard IDs and unregistered compatibility.
SELF uncached go test ./..., isolated PostgreSQL Store tests and static build passed.

Impact:
Recheck trusted live owner/project authorization when HITL resumes and at write
boundaries. Registered publishers replace only exact owner/project/ref/plugin
rows and vectors; ordinary unregistered publication semantics are preserved.

Refs:
Dev project-member-access implementation plan r1 Task 1; independent Backend
review findings against b9bea5bee0
2026-10-13 09:04:06 +00:00
weichen b9bea5bee0 Filter private knowledge before applying shared result limits
Why:
Higher-ranked private rows could exhaust SQL candidate limits and hide an
otherwise authorized published slice from a member's current-project search.

Evidence:
SELF TestAuthorizedKnowledgeFiltersBeforeTopK failed with no shared hits before
the change and passed after it. SELF go test ./... and the isolated PostgreSQL
Store suite passed; independent replay and current-HEAD CI remain owner gates.

Impact:
Scan the resource-authorized shared fulltext path until enough allowed results
are found. Preserve ordinary legacy retrieval candidate limits and semantics.

Refs:
Dev project-member-access plan r1 Task 1; R5 published knowledge availability
2026-10-13 08:47:19 +00:00
weichen 7cf8f02399 Add project lifecycle delegation and private member work
Why:
Shared project reads previously doubled as write permission, project mutations
bypassed the domain lifecycle, and coarse retrieval shared private resources.

Evidence:
SELF red-before-implementation logs cover projection, lifecycle, private work,
resource filtering, canonical retrieval scope, publication source and namespace.
SELF go test ./..., isolated PostgreSQL go test ./internal/store -count=1,
and CGO_ENABLED=0 go build ./cmd/agent-system passed; task owner owns replay/CI.

Impact:
Add optional actor-scoped projection/lifecycle capabilities; registered providers
control public lifecycle and live work. Preserve owned conversation history and
ordinary unregistered behavior. Authorize knowledge by stored ID/source, map
current host scopes through trusted service metadata, and filter artifact lists.

Refs:
Dev project-member-access implementation plan r1 Task 1; approved design r0.1;
2026-10-13 trusted publication Source hardening ruling
2026-10-13 08:43:02 +00:00
weichen 15566f9a33 Merge pull request 'Bump embedded core pin to TS mirror fix 5773e74' (#194) from chore/bump-core-pin into main
ci / vet-and-test (push) Successful in 51s
2026-10-13 07:12:20 +00:00
weichen 4e285bdd8d Bump embedded core pin to TS mirror fix 5773e74
ci / vet-and-test (pull_request) Successful in 1m3s
Why: 三层 pin 全等——Core 权威 TS 镜像补受管 scope 后,Backend 嵌套 pin 同步。
Evidence: go build 通过;组合校验随 Dev CI。
Impact: gitlink 一枚。
2026-10-13 07:07:47 +00:00
weichen 31a6b326c6 Merge pull request 'Add account name passthrough and EXT-19 account search' (#191) from feature/rdm-increment-account-search into main
ci / vet-and-test (push) Successful in 49s
2026-10-13 06:26:51 +00:00
weichen baaafe99d3 Merge origin/main into feature/rdm-increment-account-search
ci / vet-and-test (pull_request) Successful in 47s
2026-10-13 06:21:41 +00:00
weichen cfb8308895 Merge pull request 'Close the account lifecycle leftovers: atomicity, bootstrap lock, redeem guard, credential delivery' (#192) from feature/svc-account-hardening into main
ci / vet-and-test (push) Successful in 47s
2026-10-13 06:07:33 +00:00
weichen bc1e0a36f7 Close the account lifecycle leftovers: atomicity, bootstrap lock, redeem guard, credential delivery
ci / vet-and-test (pull_request) Successful in 51s
Why: The login recovery contract leftovers and KTD12/KTD14 require:
concurrent same-name account creation must not overwrite the winner,
first-admin bootstrap must serialize across processes, the redeem
endpoint backoff must be shared across instances, revoked API keys
must close established SSE streams, and CLI credentials need an
env-DSN path plus a non-interactive 0600 file delivery mode.

Evidence: go test ./... green; new tests cover 16-way concurrent
create (1 win/15 conflict, winner credentials intact), 8-way
concurrent bootstrap (single admin, refuse takeover after), redeem
guard shared across two store handles on one DB, SSE stream closed
after key revocation within the recheck window, and HTTP 429 after
the failure cap (SELF worktree run; CI is the GATE, PG via test-pg).

Impact: Migration v29 (redeem_attempts); CreateAccount becomes a
single-row INSERT with unique-violation conflict; BootstrapFirstAdmin
advisory-locks (PG) / rechecks (SQLite) and commits admin+key+audit in
one transaction; SSE revalidation covers API-key streams; redeem
backoff moves to the shared DB; CLI gains AGENT_SYSTEM_DSN fallback
and --out file delivery for create/reset-password/issue-login-key/
bootstrap-admin.

Refs: docs/svc-deploy/2026-10-13-login-recovery-contract.md leftovers;
proposal r2 KTD12/KTD14/R5/R11.
2026-10-13 06:05:45 +00:00
weichen fe977c8e52 Record account search surface entries
ci / vet-and-test (pull_request) Successful in 50s
Why: AccountSearcher 属有意的公开契约扩展,表面快照随行。
Evidence: go test ./... 全绿(TestSurfaceSnapshot 通过)。
2026-10-13 06:05:10 +00:00
weichen e3b6cca8bc Add account name passthrough and EXT-19 account search
Why: P3 人员选择器需要按账号或姓名查找成员并区分同名——EXT-19 目录此前只有 owner+有效态;扩展申请四要素记入 Dev 任务卡(授权管理读面下发姓名,联系方式仍不下发;停用能力经 facts F6 核实为未实现,Valid 维持存在性口径)。
Evidence: go test ./... 全绿;TestInprocHostSearchAccounts 覆盖姓名/账号子串、大小写不敏感、空 query 首页、ListAccounts 姓名回归;facts F5/F6 登记随 Dev PR。
Impact: AccountEntry 加 Name 可选字段;新增可选接口 AccountSearcher 与 inprocHost 实现;既有消费者不受影响(字段可选、接口可选)。
Refs: 提案 r1 P3;Dev 任务卡 feature/rdm-increment-settings-duties
2026-10-13 06:04:26 +00:00
weichen 741da4a0b6 Merge pull request 'Seed the mock startup option into the system settings scope' (#190) from fix/mock-startup-system-scope into main
ci / vet-and-test (push) Successful in 1m13s
2026-10-13 04:46:44 +00:00
weichen 0af1ebc915 Seed the mock startup option into the system settings scope
ci / vet-and-test (pull_request) Successful in 48s
Why: Since the system-managed settings change (#188), runtime.* writes
at the user scope are rejected, so the mock startup flag failed with
"system-managed settings require admin" and the serve process never
became healthy - the Dev multi-instance integration regression.

Evidence: go build ok; internal/app and internal/settings suites green
(SELF); the Dev composite CI on the follow-up PR re-runs
make test-multi-instance against this fix (GATE).

Impact: The mock startup flag now seeds runtime.mock via MergeSystem
(trusted operator input, effective for every tenant through the
Effective merge chain); no HTTP surface changes.

Refs: Dev PR #508; backend #188.
2026-10-13 04:45:12 +00:00
weichen 349a47de6f Merge pull request 'Close invalid sessions SSE streams at heartbeat recheck' (#189) from fix/sse-session-invalidation into main
ci / vet-and-test (pull_request) Successful in 48s
ci / vet-and-test (push) Successful in 1m10s
2026-10-13 04:35:26 +00:00
weichen 8b73afeef6 Close invalid sessions' SSE streams at heartbeat recheck
ci / vet-and-test (pull_request) Successful in 55s
Why: The 2026-10-13 login recovery contract (Dev docs/svc-deploy/
2026-10-13-login-recovery-contract.md, plan item 6.7) requires that
established SSE connections stop receiving private events once a
password write destroys their session.

Evidence: go test ./... green; TestSSESessionInvalidationClosesStream
streams over a real HTTP server, invalidates the session via
AdminResetUserPassword, and asserts the stream terminates with the
session-expired marker within the recheck window (SELF, worktree run).

Impact: Session-authenticated event streams revalidate at a 15s
heartbeat cadence (package var, shortened in tests) and terminate on
invalidation; API-key streams are unchanged (key revocation is separate).
EventSource reconnects are rejected by the auth middleware.

Refs: Dev PR #504; login recovery contract section 6.7.
2026-10-13 04:33:17 +00:00
weichen 76c75ce50c Merge pull request 'Enforce system-managed settings scope with admin system settings API' (#188) from feature/rdm-increment-settings-duties into main
ci / vet-and-test (push) Successful in 58s
2026-10-13 04:29:25 +00:00
weichen 496c784213 Merge origin/main into feature/rdm-increment-settings-duties
ci / vet-and-test (pull_request) Successful in 1m2s
2026-10-13 04:22:36 +00:00
weichen 386afcd4a6 Point core pin at merged main 0f94e4b0
ci / vet-and-test (pull_request) Successful in 42s
Why: Core PR #30 合入后按 pin 级联惯例把嵌套 pin 对准组件 main 合入提交。
Evidence: go test ./internal/settings/ ./internal/server/ 全绿。
Impact: 仅 gitlink 指针调整(同一能力内容,9150e6f 为其祖先)。
2026-10-13 04:20:07 +00:00
weichen 716fe9541c Enforce system-managed settings scope with admin system settings API
Why: P2——模型、密钥、Tavily、遥测与运行基础设施属系统受管项;user/project 层须整笔拒绝(含嵌套、点路径与子树覆盖),生效链剥除存量覆盖,管理员经新系统面统一配置。
Evidence: go test ./... 全绿;settings 单测覆盖嵌套/点路径/祖先标量拒绝、系统层生效、读面剥除;server e2e 覆盖 400/系统面读写/掩码;认证档 admin 门测试(普通 403、admin 200、密钥不泄漏)。
Impact: settings.go 受管标记与 mergeScope 拦截、Effective 增系统层、server 新增 GET/PUT /api/settings/system(无认证档本机即管理员)、SecretLookup 系统行回退、插件 schema 受管注入;测试基建 mock seed 与三处用例迁到系统层;svc.sh 播种改走系统面(仅 admin,无管理员时明确跳过)。Core gitlink 升至 9150e6f。
Refs: modules/documents/docs/proposals/2026-10-13-rdm-settings-members-duties-perms-increment.md P2
2026-10-13 04:05:15 +00:00
weichen f103540d60 Merge pull request 'Harden login-key recovery flow with auth-version transactions' (#187) from feature/svc-login-recovery-flow into main
ci / vet-and-test (push) Successful in 46s
2026-10-13 04:03:05 +00:00
weichen 467718a519 Harden login-key recovery flow with auth-version transactions
ci / vet-and-test (pull_request) Successful in 52s
Why: The login recovery contract (2026-10-13) requires admin-issued
one-time keys, in-transaction revalidation of restricted sessions, a
login race guard, and audited password writes; the previous
implementation validated sessions outside the change-password
transaction and let stale password verifications create sessions.

Evidence: go test ./... green (SQLite); store tests switch to
openTestStore so the CI test-pg job covers real PostgreSQL, including
100-way concurrent redeem single-winner and auth-version race cases
(SELF, local run).

Impact: New migration v28 (users/sessions.auth_version,
account_events). New admin endpoints POST/GET
/api/admin/users/{owner}/login-keys and DELETE
.../login-keys/{id}. Login binds session creation to the verified
auth version and creates restricted sessions for must-change accounts;
redeem gains per-IP backoff; cookies gain Secure under TLS; active
change-password and admin/CLI resets become single transactions that
bump auth_version, revoke keys and sessions, and write audit events.

Refs: modules/documents/docs/proposals/2026-10-12-svc-unified-deploy-and-login-key.md r2; Dev task feature/svc-login-recovery-flow
2026-10-13 03:36:38 +00:00
weichen dc736e339d Merge pull request 'Fall back to local artifacts when the blob store is a typed nil' (#186) from fix/u7-k4-artifact-upload into main
ci / vet-and-test (push) Successful in 43s
Why: an unset object-storage endpoint passed a typed nil into the store and artifact upload panicked.
Evidence: GATE vet-and-test run 2783 job 4031 success on 46073d30a5. Independent review approved with no blockers.
Impact: unconfigured stores fall back to the local artifacts directory. Configured S3 is unchanged.
Refs: #186
2026-10-13 01:53:02 +00:00
weichenandCursor 46073d30a5 Fall back to local artifacts when the blob store is a typed nil
ci / vet-and-test (pull_request) Successful in 46s
Why: an unset S3 client passed through the Store interface is not a nil interface, so uploads panic instead of using the local artifact directory.
Evidence: TestIsNil and TestFinishOpenTypedNilS3UsesLocalArtifacts failed before this change and pass after it. The PostgreSQL case skips unless AGENT_PG_TEST_DSN is set.
Impact: only an unset blob store falls back; a constructed S3 or local store is unchanged.
Refs: internal/blobstore/blobstore.go internal/store/store.go
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-13 01:22:57 +00:00
weichen c6fb13016a Merge pull request 'Open existing databases without migrating' (#184) from feature/d1-identity-open into main
ci / vet-and-test (push) Successful in 47s
Why: identity probes must open an existing database without creating or migrating it, and DSN must stay out of argv.
Evidence: GATE ci / vet-and-test success on 0f192a4813, run 2625 job 3782.
Impact: version --json carries schema target and compat floor; unknown top-level commands exit 2.
Refs: #184
2026-10-12 15:05:04 +00:00
weichenandCursor 0f192a4813 Merge origin/main into feature/d1-identity-open
ci / vet-and-test (pull_request) Successful in 57s
Why: main gained AGENT_SYSTEM_DSN fallback in #182 after this branch
was cut. Both changes touch identity DSN parsing.
Evidence: go test ./pkg/identity/ ./internal/store/ ./cmd/agent-system/ for the identity, schema, and unknown-subcommand cases passed after the resolution.
Impact: keeps open-existing identity mode and the 0600 dsn file. Environment
DSN still wins when --dsn is absent, via ResolveDSN.
Refs: Backend #182; recovery plan r3.1 D1

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 14:55:56 +00:00
weichenandCursor 5c26a86331 Publish schema target and compatibility floor
ci / vet-and-test (pull_request) Successful in 47s
Why: a rollback can only be safe when the candidate says which schema it
will migrate to and whether older binaries can still read that schema.
A new migration that forgets to register that fact must fail the test.
Evidence: go test ./internal/store/ ./pkg/identity/ -run 'TestMigrationCompat|TestCapabilities' passed.
Impact: version --json and identity capabilities include schema.target and
schema.compatFloor. Every migration version is listed in
migrationBreaksOldReaders. Current versions do not break old readers, so
the floor stays 0.
Refs: recovery plan r3.1 D2

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 14:11:07 +00:00
weichen 41e7eea0c3 Merge pull request 'Fall back to AGENT_SYSTEM_DSN for identity CLI DSN delivery' (#182) from feature/identity-dsn-env into main
ci / vet-and-test (push) Successful in 41s
2026-10-12 14:05:30 +00:00
weichenandCursor 69ea7dcc14 Open identity commands on an existing database only
Why: identity ensure/verify went through the normal store open, which
creates a database and runs migrations. A deploy gate that calls the
candidate before stopping the old service could migrate production
ahead of the switch, and a missing database was turned into an empty one.
Evidence: go test ./pkg/identity/ ./internal/store/ ./cmd/agent-system/ -run 'TestCapabilities|TestEnsure|TestVerify|TestDSN|TestUsage|TestSystemIdentity|TestOpenIdentity|TestUnknownSubcommand' passed in this worktree.
Impact: identity CLI refuses a missing file or a missing agent_system_identity table, does not migrate, reads DSN from the environment or a 0600 file, and version --json plus identity capabilities emit one capability document. Unknown top-level commands exit 2.
Refs: recovery plan r3.1 D1; Dev feature/svc-d1-identity-flow

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 14:04:41 +00:00
weichen 58b4c640d5 Fall back to AGENT_SYSTEM_DSN for identity CLI DSN delivery
ci / vet-and-test (pull_request) Successful in 44s
Why: svc.sh delivers database credentials via environment per G1-7
(command lines are visible to every process on the host), but identity
ensure/verify only accepted --dsn on the command line for pg-bound
instances.
Evidence: go test ./pkg/identity/... ok; build ok.
Impact: --dsn remains highest priority, then AGENT_SYSTEM_DSN, then
embedded SQLite; no storage changes.
Refs: Dev U3 (real-combination CI gates); G1-7 credential discipline.
2026-10-12 14:04:07 +00:00
weichen 0191be52cd Merge pull request 'Add EXT-19 account lister host capability' (#181) from feature/ext19-account-lister into main
ci / vet-and-test (push) Successful in 43s
2026-10-12 13:47:26 +00:00
weichen bff6820e5f Add EXT-19 account lister host capability
ci / vet-and-test (pull_request) Successful in 42s
Why: 方案 r1 U3/KTD2——RDM 成员管理需最小账号目录(owner+有效态+分页);此前无公开能力,下游只能盲加成员字符串。
Evidence: 红(AccountLister 未定义编译红);绿:surface 棘轮 2.11.0 + TestInprocHostListAccounts(排序/游标/limit 钳制/真实 store);go test ./... 全绿。
Impact: 公开面 AccountEntry{Owner,Valid}+AccountLister(游标分页,limit 1-200 缺省 50);inprocHost 实现(users 表,无姓名/联系方式下发);既有 EXT-8b 版本断言放宽为下限。
Refs: 方案 r1 U3/KTD2;Dev#477/#180 identity CLI
2026-10-12 13:46:15 +00:00
weichen f718d36d23 Add red test for EXT-19 account lister surface
Why: 方案 r1 U3/KTD2——最小账号目录(owner+有效态+分页)是 RDM 成员管理的公开前置。
Evidence: go vet 编译红(AccountLister/AccountEntry/SurfaceVersion 2.11.0 未定义)。
Impact: 仅测试。
Refs: 方案 r1 U3/KTD2;Dev#477 identity CLI
2026-10-12 13:43:24 +00:00
weichen acabb85e2a Merge pull request 'Publicize identity CLI as pkg/identity for downstream reuse' (#180) from feature/identity-cli-public into main
ci / vet-and-test (push) Successful in 43s
2026-10-12 13:11:27 +00:00
weichen 5623000d51 Publicize identity CLI as pkg/identity for downstream reuse
ci / vet-and-test (pull_request) Successful in 43s
Why: runIdentity lived in cmd/agent-system package main, so downstream
combination binaries (Agent-System-RDM) cannot reuse it and their unknown
subcommands fall into the serve path; the 2026-10-12 rdm1/rdmcand batch
update failures traced back to that capability gap.
Evidence: go test ./pkg/identity/... ok; CGO_ENABLED=0 go build ok;
behavioral probes (usage rc=2, capabilities v1, ensure/verify roundtrip)
recorded in the PR.
Impact: identity semantics unchanged for the base binary; adds
identity capabilities (v1) and usage-on-unknown contract shared by all
products; no storage changes.
Refs: Dev PR #474 (svc.sh candidate gate); user root-cause report
2026-10-12 U1 (shared identity interface).
2026-10-12 13:10:13 +00:00
weichen d1f46c5646 Merge pull request 'Map background card failures to CardFailed with L1 body' (#177) from feature/fast-slow-display-u1 into main
ci / vet-and-test (push) Successful in 49s
Reviewed-on: https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Backend/pulls/177
2026-10-12 12:16:45 +00:00
weichen 830fea3d3b Merge pull request 'Consult resource-level authorizer on all shared read paths' (#179) from feature/ext8-read-consumption into main
ci / vet-and-test (push) Successful in 47s
2026-10-12 12:09:51 +00:00
weichen 19d4e1ef0d Consult resource-level authorizer on all shared read paths
ci / vet-and-test (pull_request) Successful in 46s
Why: 方案 r1 U9 第二切片——EXT-8b 契约落地后,读面(下载/清单/建会话/SSE 订阅/检索过滤/动作矩阵 read 回落)仍走 V1 owner×project 布尔,资源粒度与受信六要素未消费。
Evidence: 红测(SetResourceAuthorizerV2 未定义编译红,head 见任务收据);绿测三件:六要素查询断言(download=artifact×ID×Owner=alice×ProjectID、list/read=project×pid)、V2 拒绝压过 V1 放行(404 不泄露)、V2 缺席回落 V1(现行兼容);go test ./... 全绿。
Impact: Server 增 ProjectAuthzV2 槽与 SetResourceAuthorizerV2;authorizeSharedResource(六要素受信构造、字段缺失/出错一律 false)替换六处读路径的 V1 直调(artifactContent/getArtifact/sharedArtifactGet=listArtifacts/createConversation/SSE subscribe/检索 allow-fn/authorizeProjectAction read 回落);app.go 组合根注入 authzSlot.AuthorizeResource。V2 未注册时全部回落 V1,RDM 现网行为不变。
Refs: Backend#178;方案 r1 U9/KTD2
2026-10-12 12:08:14 +00:00
weichen 430caa6296 Add red tests for EXT-8b read-path consumption
Why: U9 第二切片——server 读面须按资源级六要素咨询 V2(红:SetResourceAuthorizerV2 未定义)。
Evidence: go vet 编译红(槽位/消费缺失);断言下载/list/建会话三路径查询要素、V2 拒绝压过 V1、V2 缺席回落 V1。
Impact: 仅测试。
Refs: Backend#178;方案 r1 U9
2026-10-12 12:04:43 +00:00
weichen bbe977351d Merge pull request 'Add EXT-8b resource-level authorization contract' (#178) from feature/ext8-resource-authz-v2 into main
ci / vet-and-test (push) Successful in 45s
2026-10-12 11:45:35 +00:00
weichen 0270b40197 Add EXT-8b resource-level authorization contract
ci / vet-and-test (pull_request) Successful in 45s
Why: 方案 r1 U9 首切片——V1(owner×project 布尔)无法表达资源粒度判定,读面(下载/列表/检索/模型上下文)需要执行端按资源咨询下游。
Evidence: 红:resource_authz_v2_test.go 编译红(ResourceAuthzQuery/V2 未定义);绿:pkg/extension surface 棘轮(2.10.0)+ internal/app 槽位测试(V2 咨询/V1 回落/字段缺失拒/error fail-closed/首注册者生效/compiled 拒绝);go test ./... 全绿(收据 /tmp/be-v2-test.txt)。
Impact: 公开面新增 ResourceAuthzQuery/ResourceAuthorizerV2/Registry.RegisterResourceAuthorizerV2;projectAuthz 增 V2 槽(AuthorizeResource:V2 优先、V1 兼容回落、受信字段缺失=拒);compiledRegistry fail-closed 拒绝声明式注册;读路径消费接线(server 5 处 authorizeSharedProject 升级)留待下一切片。
Refs: 方案 r1 U9/KTD2;RDM EXT-8 现有消费者(hostcaps/projectAuthz)
2026-10-12 11:44:00 +00:00
weichenandCursor 4d48abaa8c Map background card failures to CardFailed with L1 body
ci / vet-and-test (pull_request) Successful in 55s
Why: Stop showing raw task text as progress and stop lighting the
done lamp on failed background runs (U1 whitelist).
Evidence: go test ./internal/runmgr/ -count=1
Impact: create/promote/finish card bodies; material queue status text.
Refs: fast-slow-channel-display-u1-plan W2; Core ba289e7

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 07:43:31 +00:00
weichen 5e8ada727c Wait on CardDone and materialActive release in lifecycle test (#176)
ci / vet-and-test (push) Successful in 1m16s
2026-10-12 06:32:43 +00:00
weichenandCursor c963347e52 Wait on CardDone and materialActive release in lifecycle test
ci / vet-and-test (pull_request) Successful in 46s
Why: TestMaterialIntakeAttachmentOnlyLifecycle flaked when waiting on
systemTexts; deliverNotices runs before defer materialKeyRelease, so
done-dedup still saw the in-memory key and returned pending.
Evidence: go test ./internal/runmgr/ -run TestMaterialIntakeAttachmentOnlyLifecycle -count=20
Impact: Aligns the wait with materialExisting gates; no production change.
Refs: fix/material-intake-lifecycle-flake

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 06:31:43 +00:00
weichen dcc180dcad Merge pull request 'Add agent_system_identity DB marker (U1 I3)' (#170) from feature/recovery-u1-db-identity into main
ci / vet-and-test (push) Successful in 48s
Reviewed-on: https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Backend/pulls/170
2026-10-12 05:35:11 +00:00
weichenandCursor e7b70c24f8 Renumber agent_system_identity to v27 after login_keys took v26
ci / vet-and-test (pull_request) Successful in 42s
Why: main #171 claimed migration v26; rebase conflict required
bumping the U1 I3 table and completing copyOrder for both new tables.
Evidence: go test ./internal/store/ ./cmd/storage-migrate/ -run
'SystemIdentity|CopyOrder'
Impact: migrate v27; copyOrder += login_keys + agent_system_identity.
Refs: Backend PR #170; conflict with main 1d9e8f4

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:04:16 +00:00
weichenandCursor dcd5445d61 Include agent_system_identity in storage-migrate copyOrder
Why: TestCopyOrderCoversAllFixtureTables fails when new schema tables
are omitted from the migrate tool inventory.
Evidence: go test ./cmd/storage-migrate/ -run CopyOrder
Impact: copyOrder gains agent_system_identity (no FK deps).
Refs: U1 I3; Backend PR #170

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:03:45 +00:00
weichenandCursor e046bbd15c Add agent_system_identity marker with ensure/verify CLI
Why: U1 I3 needs a DB-side instance identity anchor matching
manifest.instance_id so wrong/missing DB blocks startup (I4).
Evidence: go test ./internal/store/ ./internal/app/; CLI smoke
ensure/verify on SQLite temp dir.
Impact: migration v26; store Ensure/Verify; identity subcommand;
serve checks AGENT_SYSTEM_INSTANCE_ID when set.
Refs: docs/recovery/u1-identity-manifest.md §5.2; Dev feature/recovery-u1-db-identity

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 05:03:45 +00:00
weichen 1d9e8f4a79 Merge pull request 'Add one-time login keys, restricted sessions, and OpenExisting' (#171) from feature/svc-deploy-u4-login-key into main
ci / vet-and-test (push) Failing after 43s
Reviewed-on: https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Backend/pulls/171
2026-10-12 05:00:47 +00:00
weichen 52d75fd931 Merge pull request 'Flatten surface action data into command params' (#174) from fix/surface-action-flatten-data into main
ci / vet-and-test (push) Successful in 47s
2026-10-12 04:16:41 +00:00
weichenandCursor ed443ed3f6 Flatten surface action data into command params
ci / vet-and-test (pull_request) Successful in 47s
Why: system.intent ParamsSchema requires top-level text, but
SubmitAction only nested values under data, so button intent always
rejected missing text during prod acceptance.
Evidence: go test ./internal/surfacesvc/ -run TestSubmitActionFlattensDataIntoParams
Impact: B-class intent and other schema-required params work via actions;
submit still receives nested data.
Refs: GenUI prod deploy acceptance; fix track

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 04:16:34 +00:00
weichen c7b30d2109 Merge pull request 'Guard nil confirm card in surface action dispatcher' (#173) from fix/surface-action-nil-card into main
ci / vet-and-test (push) Successful in 43s
2026-10-12 04:08:58 +00:00
weichenandCursor 4352f368cb Guard nil confirm card in surface action dispatcher
ci / vet-and-test (pull_request) Successful in 44s
Why: Committed button actions with no HITL card panicked on
resp.Card.ID during prod acceptance (nil pointer).
Evidence: go test ./internal/app/ ./internal/surfacesvc/; prod panic stack
Impact: B-class help/intent and C-class submit return without crash.
Refs: GenUI prod deploy acceptance

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 04:08:47 +00:00
weichen cb386023d0 Merge pull request 'Keep auto-present reminders synchronous to avoid run races' (#172) from feature/genui-auto-trigger-optin into main
ci / vet-and-test (push) Failing after 58s
2026-10-12 03:57:11 +00:00
weichenandCursor 9bd0aae9d8 Keep auto-present reminders synchronous to avoid run races
ci / vet-and-test (pull_request) Successful in 56s
Why: Async notice-run delivery raced with test teardown and in-flight
conversation runs (database closed / material lifecycle flakes).
Evidence: go test ./internal/runmgr/ -run Material
Impact: Reminders append as system messages only; material path still
inlines into startNoticeRun.
Refs: Dev PR #425 CI test-pg failure

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:56:59 +00:00
weichenandCursor 47463a9623 Align admin account e2e with must-change password gate
ci / vet-and-test (pull_request) Failing after 1m25s
Why: U4 middleware restricts must_change sessions to me/password/logout;
the legacy e2e expected admin create before first password change.
Evidence: go test ./internal/store/ ./internal/server/ ./internal/app/
Impact: TestAdminAccountFlow matches set-password + reloginRequired.
Refs: docs/svc-deploy/u4-one-time-login-key.md §5–6

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:54:59 +00:00
weichen b6ea1a5910 Merge pull request 'Drop runmgr import of genui to satisfy package boundaries' (#169) from feature/genui-auto-trigger-optin into main
ci / vet-and-test (push) Successful in 1m1s
2026-10-12 03:48:28 +00:00
weichenandCursor 41df7ae56f Drop runmgr import of genui to satisfy package boundaries
ci / vet-and-test (pull_request) Successful in 1m10s
Why: lint-boundaries forbids backend/runmgr -> backend/genui; keep the
auto trigger as an injected interface with local reminder text.
Evidence: bash scripts/lint-boundaries.sh
Impact: No behavior change; wiring still via app.SetGenUIAuto(genui.NewAuto).
Refs: architecture boundaries

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:48:10 +00:00
weichen 9b69117bea Merge pull request 'Add Generative UI auto-present opt-in with budget and cooldown' (#168) from feature/genui-auto-trigger-optin into main
ci / vet-and-test (push) Failing after 1m4s
2026-10-12 03:45:03 +00:00
weichenandCursor 6ff7fa9bd7 Add Generative UI auto-present opt-in with budget and cooldown
ci / vet-and-test (pull_request) Successful in 1m4s
Why: Known-boundary follow-up: owner opt-in auto page suggestions on
semantic task events without self-trigger from surface revisions.
Evidence: go test ./internal/genui/ ./internal/runmgr/ ./internal/settings/ ./internal/app/
Impact: Default-off settings key; material/plan/todo/document hooks nudge present.
Refs: docs/proposals/2026-10-12-generative-ui-auto-trigger.md

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:44:47 +00:00
weichenandCursor 1ba7225e83 Add one-time login keys, restricted sessions, and OpenExisting
ci / vet-and-test (pull_request) Failing after 56s
Why: passwordless activation/recovery must redeem only into a
set-password session; account ops must not create empty stores.
Evidence: SELF go test store/server login-key and OpenExisting cases.
Impact: auth middleware gates; CLI user create defaults to activate key.
Refs: docs/svc-deploy/u4-one-time-login-key.md

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:44:16 +00:00
weichen d1b5a94d15 Merge pull request 'Whitelist B-class surface actions for help and intent' (#167) from feature/genui-bclass-help-intent into main
ci / vet-and-test (push) Successful in 1m3s
2026-10-12 03:23:57 +00:00
weichen ba619747c5 Merge authoritative main into the B-class branch
ci / vet-and-test (pull_request) Successful in 1m12s
2026-10-12 03:23:43 +00:00
weichenandCursor 5e0077dc11 Whitelist B-class surface actions for help and intent
ci / vet-and-test (pull_request) Successful in 48s
Why: Generative UI U5 left the B-class action allowlist empty; owner
authorized first entries builtin.help and system.intent.
Evidence: go test ./internal/genui/ ./internal/surfacesvc/ ./internal/server/
Impact: Surface buttons may bind help/intent; button-entrance intent
matches only and does not nest-dispatch write commands.
Refs: docs/proposals/2026-10-12-generative-ui-bclass-help-intent.md

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-12 03:18:57 +00:00
weichen b160515905 Merge pull request 'Unwrap item-style spec wrappers, pair bare table cells, let subagents present pages' (#166) from feature/genui-spec-repair into main
ci / vet-and-test (push) Successful in 55s
Reviewed-on: https://git-home.u1680832.nyat.app:30319/weichen/Agent-System-Backend/pulls/166
2026-10-12 03:09:35 +00:00
weichen 9ec9f9a010 Merge pull request 'Harden cross-instance HITL/cancel command paths and add disk-full rejection tests (U7)' (#164) from feature/u7-load-fault-release into main
ci / vet-and-test (push) Successful in 59s
2026-10-12 03:09:03 +00:00
weichen 58598df095 Merge remote-tracking branch 'origin/main' into feature/u7-load-fault-release
ci / vet-and-test (pull_request) Successful in 51s
2026-10-12 03:07:33 +00:00
weichen 3d850a282f Unwrap item-style spec wrappers, pair bare table cells, let subagents present pages
ci / vet-and-test (pull_request) Successful in 1m4s
Why: MiniMax-M3 emits XML-RPC-style {"item": ...} single-key wrappers on
array fields (kv/cells/items), bare scalar table cells, and page
submission delegated to the general sub-agent had no generative-ui
tool - observed live on dev (events seq 1395/1396/2171) where every
present with kv/table/list nodes failed invalid-spec and the delegated
retry reported the tool missing.

Evidence: red test reproducing the exact production error
(.nodes.1.kv of type [][2]string) goes green after the repair; full
genui/prompts/runtime/runmgr suites pass (SELF); mismatched-length
cells stay untouched by a dedicated guard test.

Impact: generative-ui pages with kv/table/list nodes save successfully
again; the general sub-agent can submit pages (tool resolves only when
the plugin exposes it, so absent plugins are unaffected).

Refs: dev task card feature/genui-spec-repair; dev conversation
激活验证-1012.
2026-10-12 03:03:26 +00:00
weichen a80d3c3359 Merge pull request 'Declare admin-zone sections from compiled plugins (surface 2.9.0, WP3-I3)' (#165) from feature/adminzone-plugin-sections into main
ci / vet-and-test (push) Successful in 46s
2026-10-12 02:39:56 +00:00
weichen 1719b5f668 Merge authoritative main into the admin-zone branch
ci / vet-and-test (pull_request) Successful in 49s
2026-10-12 02:37:57 +00:00
weichen 62bfc6068a Declare admin-zone sections from compiled plugins (surface 2.9.0)
ci / vet-and-test (pull_request) Successful in 46s
Why: the admin zone could not host plugin-declared partitions - KTD4
requires the RDM audit partition to be reachable by RDM domain admins and
base superadmins without hardcoding RDM into the base AdminZone.
Evidence: go test ./internal/app ./internal/server ./pkg/extension green
incl. new admin-zone recording/endpoint tests; surface snapshot test
green after surface.txt update; CI (Dev composite) is authoritative.
Impact: extension surface 2.8.0 -> 2.9.0 (Registry.AddAdminZone +
AdminZoneSpec, fail-closed on invalid specs); applyCompiled records
declarations into the plugin summary; new authenticated GET
/api/admin-zone/sections returns the flattened, ordered declaration list.
Entry visibility stays metadata-only - data capability remains enforced
by each section's own command surface (fail-closed).
Refs: WP3-I3/KTD4; task card feature-adminzone-plugin-sections.
2026-10-12 02:28:51 +00:00
weichen 0a7413903d Harden cross-instance HITL/cancel command paths and add disk-full rejection tests (U7)
ci / vet-and-test (pull_request) Successful in 48s
Why: two-process HTTP acceptance (Dev tests/integration/multi-instance, added in
the matching Dev-side change) exposed three real cross-instance defects that the
U5 library-level tests bypassed via direct Manager calls, plus the U7 fault
matrix needs explicit disk-full rejection evidence.
Evidence: go test ./... green on SQLite and real PG (AGENT_PG_TEST_DSN);
new regressions TestCrossInstanceResolveDoesNotSettleCard (runmgr, PG-gated),
TestUploadRejectsCleanlyWhenObjectDirUnwritable and
TestArtifactOverwriteRejectsCleanlyWhenObjectDirUnwritable (server).
Impact: ResolveHITL no longer settles the card when the run executes on another
instance (command-bus fallback stays available); cancelRun enqueues the command
with the real conversation id and no longer reports a false local success;
object-store failures now provably leave zero DB/object residue.
Refs: collaboration storage proposal U7 (sections 3/5/7); task card
feature/u7-load-fault-release.
2026-10-12 02:28:48 +00:00
weichen 696ef15168 Merge pull request 'Add project-level surface aggregation and task-phase guidance (U6)' (#163) from feature/generative-ui-u6 into main
ci / vet-and-test (push) Successful in 53s
2026-10-12 02:12:59 +00:00
weichen add1fbb045 Add project-level surface aggregation and task-phase guidance (U6)
ci / vet-and-test (pull_request) Successful in 54s
Why: U6 extends pages from conversations into the task/project
workspace - surfaces aggregate across conversations under the owner's
project scope (project resolved authoritatively from the conversation
row, never self-reported by the model), and the catalog guidance now
organizes pages by task phase (requirements/execution/acceptance)
while stating that task facts come only from real task records.

Evidence: build/vet clean; surfacesvc suite green incl. the new
project-aggregation test (owner isolation, cross-owner zero, history
retained post-disable); genui/store suites green.

Impact: New endpoint GET /api/projects/{id}/surfaces; Submit resolves
project from conversation; catalog text guidance. Existing endpoints
unchanged.

Refs: proposal r2 U6; Dev task feature/generative-ui-u6
2026-10-12 02:12:42 +00:00
weichen d835ad7466 Merge pull request 'List compiled plugins in /api/plugins (WP3-I4)' (#162) from feature/plugins-compiled-listing into main
ci / vet-and-test (push) Successful in 45s
2026-10-12 01:59:21 +00:00
weichen 9aa51e5250 List compiled plugins in /api/plugins
ci / vet-and-test (pull_request) Successful in 47s
Why: compiled-in plugins (generative-ui, rdm) never appeared in the admin
plugins list: GET /api/plugins only scanned the directory host, so binary-
level capabilities were invisible to operators (user-reported gap).
Evidence: go test ./internal/app/ ./internal/server/ green locally incl.
new TestApplyCompiledRecordsSummary and TestListPluginsMergesCompiled;
CI (make test) is authoritative.
Impact: applyCompiled records each plugin's registered commands/intents/
skills into a summary handed to the server; /api/plugins merges synthesized
inproc entries (backend.mode=inproc, apiVersion=extension surface version,
provides from the registration record). Directory entries win on id
collision. No public extension-surface change.
Refs: WP3-I4; task card feature-plugins-compiled-listing.
2026-10-12 01:44:56 +00:00
weichen e49f165d0b Merge pull request 'Material-intake guaranteed loop: deterministic spawn, choice chain, queue, cardflow on (Dev task feature/material-intake-flow)' (#161) from feature/material-intake-flow into main
ci / vet-and-test (push) Successful in 42s
2026-10-11 17:47:56 +00:00
weichen 409390fff1 Merge authoritative main into the task branch
ci / vet-and-test (pull_request) Successful in 42s
2026-10-11 17:37:51 +00:00
weichen 4c637d4071 Merge pull request 'Repair object-shaped children maps into arrays in page specs' (#160) from fix/genui-children-repair into main
ci / vet-and-test (push) Successful in 49s
2026-10-11 16:57:51 +00:00
weichen 432b916cba Repair object-shaped children maps into arrays in page specs
ci / vet-and-test (pull_request) Successful in 52s
Why: Live E2E on the dev instance - the model keyed section children
as an object map (children.item.*), the strict decode rejected it, and
the page fell back to a file artifact; the guidance fix made the model
call present, this closes the remaining shape gap.

Evidence: genui suite green incl. the reproduction case; post-deploy
E2E rerun recorded in the Dev task.

Impact: Repair strategy only; validation unchanged.

Refs: Dev task fix/genui-page-visibility (E2E follow-up)
2026-10-11 16:57:48 +00:00
weichen 16ed7b281b Add background queue for system tasks and flip ui.cardflow default on
Why: at the 3-per-conversation cap the guaranteed material task was dropped silently, and long deliverables stayed in chat because cardflow defaulted off (user ruling 2026-10-11).
Evidence: new queue lifecycle test (queued card, hard-reject for model path, FIFO start on release, card promote) passes; full go test ./... exit 0 (local, SELF).
Impact: queueable system spawns create a queued progress card at capacity and start FIFO when a slot frees (memory-only queue; restart limitation noted); model delegate_task still hard-rejects; ui.cardflow default true (EffectNewRun).
Refs: Dev task .task/feature-material-intake-flow.md I3.
2026-10-11 16:56:59 +00:00
weichen 42867aabf4 Add deliverable-choice chain: confirm_choice tool, notice contract, backstops
Why: after material organization completes, weak models report and end the turn without offering the user a choice of formal deliverables, and even after a user picks one the model may close the turn without starting the background task.
Evidence: new tests (notice-choice chain, deliverable backstop, free-input path, invalid args, notice contract) pass; full go test ./... exit 0 (local, SELF).
Impact: new confirm_choice tool issues a HITL choice card and routes the answer into a hard delegate instruction; material notices instruct the model to report then offer choices; stop hook adds two one-shot backstops (choice not offered in material notice runs; choice made but no background delegation).
Refs: Dev task .task/feature-material-intake-flow.md I2.
2026-10-11 16:53:53 +00:00
weichen 02858f486b Add deterministic material-intake background path for attachment sends
Why: the dual-speed entry hint is advisory only; real runs (MiniMax, 3 attachments, no text) end with a clarifying question and zero delegation, so background cards and result backfill never start.
Evidence: new unit tests in runmgr/material_test.go and runtime/material_flow_test.go all pass; full go test ./... exit 0 (local, SELF).
Impact: attachment-carrying runs now guarantee a material-organization background task (pre-spawn on attachment-only/heavy sends, stop-hook backstop otherwise when the model did not delegate); subagent receives inline parsed text with budget plus knowledge_search guidance; content-key dedup prevents duplicate spawns (in-flight, completed, resend); model-authored delegate_task gains a materials appendix; settings runtime.materialIntake (default on) and runtime.materialIntakeChars added.
Refs: Dev task .task/feature-material-intake-flow.md I1; user ruling 2026-10-11.
2026-10-11 16:45:49 +00:00
weichen faae3d3901 Merge pull request 'Add model identity and in-database vector top-k via pgvector (U6 slice 3)' (#159) from feature/collab-storage-u6-vector into main
ci / vet-and-test (push) Successful in 10m52s
2026-10-11 16:31:25 +00:00
weichen 9b33df6625 Merge authoritative main into the task branch
ci / vet-and-test (pull_request) Failing after 10m28s
2026-10-11 16:31:15 +00:00
weichen 5d5cef9fcd Add model identity and in-database vector top-k via pgvector (U6 slice 3)
ci / vet-and-test (pull_request) Successful in 56s
Why: plan U6 vector slice (KTD11 correctness step) - same-dimension vectors from different models must never mix, and PG tiers should run filtered exact top-k in the database instead of scanning every row in-process.

Evidence: full suite green on SQLite and real PostgreSQL 18.6 with pgvector (35 packages each tier) - pgvector_test covers model-identity and dimension isolation, update/delete atomicity (BLOB and vec_pg swap together), shared top-5 with score monotonicity, Recall@10 = exact-scan parity, deny-all predicate leakage; storage-migrate tests green after excluding the derived vec_pg column.

Impact: v25 migration adds model/content_hash columns and (PG, best-effort DO block) the pgvector extension plus vec_pg column - absence degrades to in-process scan; UpsertKnowledgeVectorExt dual-writes BLOB (cross-tier authority) and vec_pg; SearchKnowledgeVectorsModel/SharedModel filter by model identity and use SQL exact top-k (ORDER BY vec_pg <=> q) with automatic degradation; embedders expose Model() and the rag pipeline threads it through indexing and both retrieve paths; storage-migrate excludes vec_pg as a derived column.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U6 slice 3, KTD11; R9.
2026-10-11 16:31:12 +00:00
weichen 1fa03a3cbb Merge pull request 'Add AdminProvider capability and HostProjectID context (EXT-18)' (#158) from feature/base-ext-admin-project into main
ci / vet-and-test (push) Successful in 54s
2026-10-11 16:23:31 +00:00
weichen 4ae125a3ef Add AdminProvider capability and HostProjectID context (EXT-18)
ci / vet-and-test (pull_request) Successful in 57s
Why: RDM KTD4 dual-layer admin requires base isAdmin visibility (superuser slot); KTD5 host-project binding needs trusted current-project context.

Evidence: SELF 4 ext18 tests pass + surface snapshot regenerated + full pkg/extension + internal/app suites green.

Impact: extension.Request gains HostProjectID; optional AdminProvider interface (EXT-17 pattern); inprocHost implements IsAdmin via GetUserAuthState; three dispatch sites set HostProjectID from conversation.

Refs: RDM docs/proposals U1 base extension requests
2026-10-11 16:23:15 +00:00
weichen 25fbea8522 Merge pull request 'Add same-transaction outbox, S3-compatible blobstore, and real-process acceptance (U5 remainder)' (#157) from feature/collab-storage-u5-outbox into main
ci / vet-and-test (push) Successful in 1m4s
2026-10-11 15:58:18 +00:00
weichen e1dc93d147 Merge authoritative main into the task branch
ci / vet-and-test (pull_request) Successful in 1m10s
2026-10-11 15:58:03 +00:00
weichen eb4796cf61 Add same-transaction outbox, S3-compatible blobstore, and real-process acceptance (U5 remainder)
ci / vet-and-test (pull_request) Successful in 46s
Why: plan U5 remainder (KTD5/KTD9 + exit criteria) - commands must commit atomically with business writes, artifacts need a shared object store for multi-instance, and the two-real-process acceptance matrix must run in CI.

Evidence: full suite green on SQLite and real PostgreSQL 18.6 (35 packages each tier) - outbox tx test (rollback removes the command, commit makes both visible, idempotent re-enqueue), S3 store tests (SigV4-signed round trip, 404 sentinel, idempotent delete, presigned GET, env fallback), real-process acceptance (testbinary self-spawn of two serve processes with per-process identity and command consumers against one schema: same-conversation contention, cross-instance cancel command claimed exactly once, stale-owner settle fenced with run unchanged).

Impact: EnqueueCommandInTx for in-transaction command outbox and AppendEvents now notifies in the same transaction (KTD6 front half); blobstore.S3 (SigV4, presign, prefix isolation) injected from BLOBSTORE_S3_* env when configured, local fallback otherwise; realproc_test provides the process-level multi-instance acceptance gate (PG-env gated, runs in CI test-pg job).

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U5 KTD5/KTD9, section 5 exit criteria; R4/R7/R10.
2026-10-11 15:57:12 +00:00
weichen 1df183b104 Merge pull request 'Guide the model to present pages for page-like deliverables' (#156) from fix/genui-page-visibility into main
ci / vet-and-test (push) Successful in 47s
2026-10-11 15:56:08 +00:00
weichen 820aa4432e Guide the model to present pages for page-like deliverables
ci / vet-and-test (pull_request) Successful in 47s
Why: Field evidence from the dev instance - the present tool was
projected and enabled but never called, because every resident prompt
module steers 'make me a report' toward file artifacts; the tool
description carried only the component catalog, not when to use it.

Evidence: genui suite green (catalog assertion extended implicitly by
existing shape test); end-to-end model behavior verified post-deploy
in the Dev task.

Impact: Tool description text only (self-gating: description appears
only when the activation gate projects the tool).

Refs: Dev task fix/genui-page-visibility
EOF
)
2026-10-11 15:44:33 +00:00
weichen eb12f102f2 Merge pull request 'Add the surface action pipeline: ledger, guards, dispatch, submit (U5)' (#155) from feature/generative-ui-u5 into main
ci / vet-and-test (push) Successful in 1m6s
2026-10-11 14:24:34 +00:00
weichen f5eb27df86 Add the surface action pipeline: ledger, guards, dispatch, submit (U5)
ci / vet-and-test (pull_request) Successful in 1m10s
Why: The approved U5 scope needs the action half of the loop: an
idempotent ledger (migration v24), a single-choke SubmitAction (plugin
activation, current-revision, action whitelist, params-hash
idempotency), dispatch through the command bus so DangerHigh lands on
the existing inline confirm card with no Confirmed passthrough (the
endpoint does not even bind the field), the button-only
generative-ui.submit storing form data in the plugin KV ledger, and
catalog discipline rejecting unknown action ids at present time.

Evidence: go build/vet clean; full go test ./... green; surfacesvc/
genui/store suites also green under AGENT_PG_TEST_DSN; red matrix
covers disabled/stale-revision/cross-owner/unknown-action/idempotent
same-and-different-params/danger-high-await-confirm and the endpoint
forged-confirmed case.

Impact: Additive migration v24 + new endpoint/command; existing
endpoints and command semantics untouched (B-class whitelist starts
empty per the approved scope).

Refs: U5 action-scope r2 (approved, Documents #71); UI #46/#47;
Frontend #56; Dev task feature/generative-ui-u5
2026-10-11 14:24:31 +00:00
weichen 44d45e1092 Merge pull request 'Re-pin agent-system-core to 89f3906 to match Dev core' (#154) from feature/core-repin-89f3906 into main
ci / vet-and-test (push) Successful in 1m3s
2026-10-11 13:54:32 +00:00
weichen ec9d5f78fe Re-pin agent-system-core to 89f3906 to match Dev core
ci / vet-and-test (pull_request) Successful in 1m7s
Why: PR #152 pinned the bundled core back to 7a98a7e citing "Dev core is
at 7a98a7e", but Dev origin/main pins modules/core at 89f3906; the
misalignment breaks Dev check-pins for any tree that materializes
recorded pointers (exposed by a fresh task worktree running make test).

Evidence: go build ./... and internal/server + internal/store tests
green on the repinned tree.

Impact: restores modules/backend/deps/agent-system-core == Dev
modules/core (89f3906), unblocking Dev gitlink bumps.

Refs: Dev task feature/admin-auth-gate-fix.
2026-10-11 13:52:34 +00:00
weichen 51ad3ce115 Merge pull request 'Fall back to session cookie on stale Bearer key and report authVia' (#153) from feature/admin-auth-gate-fix into main
ci / vet-and-test (push) Successful in 47s
2026-10-11 13:48:25 +00:00
weichen 6681c3b47c Fall back to session cookie on stale Bearer key and report authVia
ci / vet-and-test (pull_request) Successful in 44s
Why: a browser holding a revoked API key in localStorage sent the Bearer
header on every request; key precedence 401'd all APIs despite a valid
login session, locking admins out of user management. /api/me also
reported mustChangePassword for key principals, who cannot change
passwords (session-owned), dead-ending the forced-change gate.

Evidence: new TestStaleKeySessionFallback covers fallback and authVia;
TestAuthMiddleware/TestAdminAccountFlow unchanged and green; full
go test ./... green.

Impact: invalid key falls back to a valid session cookie before 401
(cookie-less programmatic clients unaffected); /api/me gains
authVia ("key"|"session"|""), letting the frontend keep key holders
out of the change-password gate.

Refs: Dev task feature/admin-auth-gate-fix.
2026-10-11 13:45:52 +00:00
weichen 9a1c911e53 Merge pull request 'Pin agent-system-core to 7a98a7e (align with Dev core)' (#152) from feature/u5-core-pin into main
ci / vet-and-test (push) Failing after 0s
2026-10-11 12:30:58 +00:00
weichen bfeeefab94 Pin agent-system-core to 7a98a7e (align with Dev core)
ci / vet-and-test (pull_request) Failing after 1s
Why: Dev modules/core is at 7a98a7e7 (core PR #26); backend's internal pin lagged at 89f3906 after a parallel merge, failing Dev check-pins.

Evidence: build and store/runmgr suites green with the aligned pin; check-pins green after this merge.

Impact: gitlink bump only.

Refs: core PR #26/#28 lineage.
2026-10-11 12:30:49 +00:00
weichen 840757240b Merge pull request 'Add execution ownership leases with epoch fencing and persistent run commands (U5)' (#151) from feature/collab-storage-u5-ownership into main
ci / vet-and-test (push) Successful in 1m5s
2026-10-11 12:26:35 +00:00
weichen 9422bf9c28 Merge authoritative main into the task branch
ci / vet-and-test (pull_request) Failing after 1m11s
2026-10-11 12:26:21 +00:00
weichen 603f1c0fa4 Add execution ownership leases with epoch fencing and persistent run commands (U5)
ci / vet-and-test (pull_request) Successful in 57s
Why: plan U5/KTD4+KTD5 - before a second instance may serve writes, conversation execution must have a single valid owner (DB lease, epoch increments on takeover), stale owners must not settle (in-transaction fence check), and cancel/HITL must reach the executing instance via a persistent command inbox.

Evidence: full suite green twice-relevant on SQLite and real PostgreSQL 18.6 (35 packages each tier) - lease tests (claim/contention/re-claim/renew fencing/expiry takeover/monotonic epoch, command inbox idempotent enqueue, atomic concurrent claim) and multi-instance tests (same-conversation contention rejected, stale-owner settle fenced with run unchanged, cross-instance cancel consumed exactly once by the owner instance, non-owner does not consume).

Impact: v23 migration (conversation_executions + run_commands); ClaimConversation stacks a DB lease on PG tiers (15min, release on conversation release); settle path fenced (fenced runs emit run.fenced, no terminal write); StartCommandConsumer polls the inbox for commands on locally-owned executions (cancel routes to local cascade or DB-level close-out; HITL revalidates gate and rejects expired explicitly); server cancel/HITL fall back to command enqueue with 202 when the run is not local; app wires instance identity (host:addr on PG tiers) and starts the consumer.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U5 KTD4/KTD5; R4/R10.
2026-10-11 12:25:31 +00:00
weichen 9f7b3c5405 Merge pull request 'Pin agent-system-core to 89f3906 (surface TS types and event union)' (#150) from feature/generative-ui-u4-coredep into main
ci / vet-and-test (push) Successful in 1m3s
2026-10-11 12:22:45 +00:00
weichen fadce52dce Pin agent-system-core to 89f3906 (surface TS types and event union)
ci / vet-and-test (pull_request) Successful in 1m2s
Why: Core #27/#28 landed the surface snapshot types and the
surface.updated union entry; the backend dep copy must match Dev's
modules/core pin or check-pins fails.

Evidence: scripts/check-pins.sh + combined make test in the Dev
workspace after re-pinning.

Impact: Gitlink bump only.

Refs: Core PR #27/#28; Dev task feature/generative-ui-u4
2026-10-11 12:22:41 +00:00
weichen 6abf112765 Merge pull request 'Add the generative-ui plugin, capability gating, and surface projection' (#149) from feature/generative-ui-u3 into main
ci / vet-and-test (push) Successful in 42s
2026-10-11 10:38:27 +00:00
weichen 367dc207b3 Add the generative-ui plugin, capability gating, and surface projection
ci / vet-and-test (pull_request) Successful in 42s
Why: Generative UI U3 - the agent needs a real submission path and the
next turn needs to see prior pages: a built-in in-process plugin
registers generative-ui.present (exposeAsTool) carrying the catalog
(prompt=validation single source) with a schema-repair strategy
measured against real MiniMax-M3 deviations; the command bus gains an
owner-scoped tool gate so a disabled plugin stops injecting its tool
(execution guard remains in surfacesvc); ctxasm projects saved surfaces
back into context so the model can revise existing structure; doc-refs
are authorized against the owner's real artifacts.

Evidence: unit suites green (genui guards/epoch/doc-ref, gate fail-closed,
projection content); full go test ./... green; cmdbus/ctxasm also green
under AGENT_PG_TEST_DSN; real-model closed loop (env-gated
TestRealModelSurfaceLoop) passes: task->spec->save->projection->revise
with revision 1->2 and structure change, CNY 0.08 total.

Impact: New internal/genui package; additive cmdbus/ctxasm/runmgr/
runtime hooks; no storage or contract changes (U2 layer untouched).

Refs: Documents proposal r2 U3; Core PR #25/#26; Backend #143; Dev task
feature/generative-ui-u3
2026-10-11 10:38:15 +00:00
weichen 2bfebe378c Merge pull request 'Pin agent-system-core to the node-id fields merge (align with Dev core)' (#148) from feature/u6-core-pin into main
ci / vet-and-test (push) Successful in 42s
2026-10-11 09:33:28 +00:00
weichen f8db2225bf Pin agent-system-core to the node-id fields merge (align with Dev core)
ci / vet-and-test (pull_request) Successful in 41s
Why: Dev top-level modules/core advanced to 7a98a7e7 (core PR #26, optional node id/source TS contract fields); check-pins requires the backend internal pin to match.

Evidence: go build and store/pgbus suites green with the bumped pin; check-pins green in the Dev worktree after this merge.

Impact: gitlink bump only (additive optional contract fields).

Refs: core PR #26.
2026-10-11 09:33:19 +00:00
weichen 8c113a70b8 Merge pull request 'Catch up event streams by cursor with real reconnect and periodic rescue (U6 slice 1)' (#147) from feature/collab-storage-u6-events into main
ci / vet-and-test (push) Successful in 42s
2026-10-11 09:30:30 +00:00
weichen fc75847580 Catch up event streams by cursor with real reconnect and periodic rescue (U6 slice 1)
ci / vet-and-test (pull_request) Successful in 43s
Why: plan section 2/KTD6 found the listener fetched only the last event of a batch (intermediate events invisible to live subscribers) and kept waiting on a dead connection after a drop (no actual reconnect).

Evidence: pgbus suite green twice against real PostgreSQL - batch of 64 all visible with continuous seq, 1200 events across pages, connection-kill reconnect recovers the outage window, no-notify events delivered by the periodic rescue; full backend suite green on both tiers.

Impact: Listener tracks per-conversation cursors (first-seen catch-up from 0 is harmless under SSE seq dedup), pages via EventsAfter to the notified watermark, rebuilds the connection with backoff and resumes known streams, and a rescue ticker covers lost notifications; Notifier contract unchanged.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U6 slice 1, KTD6; R5.
2026-10-11 09:30:29 +00:00
weichen 9ee87dcf2d Merge pull request 'Pin agent-system-core to 7a98a7e (TS contract authority for node ids)' (#146) from feature/generative-ui-u2-coredep into main
ci / vet-and-test (push) Successful in 39s
2026-10-11 08:46:50 +00:00
weichen b302446986 Merge remote-tracking branch 'origin/main' into feature/generative-ui-u2-coredep
ci / vet-and-test (pull_request) Successful in 40s
2026-10-11 08:46:43 +00:00
weichen 5191efd891 Pin agent-system-core to 7a98a7e (TS contract authority for node ids)
ci / vet-and-test (pull_request) Successful in 39s
Why: Core PR #26 made ts/types.ts carry the node id/source fields; the
backend dep copy must match Dev's modules/core pin or check-pins fails.

Evidence: scripts/check-pins.sh in the Dev workspace; combined make
test rerun after re-pinning.

Impact: Gitlink bump only.

Refs: Core PR #26; Dev task feature/generative-ui-u2
2026-10-11 08:46:19 +00:00
weichen a466838440 Merge pull request 'Cover the surface tables in storage-migrate copy list' (#145) from feature/storage-migrate-new-tables into main
ci / vet-and-test (push) Successful in 37s
2026-10-11 08:33:01 +00:00
weichen 1d5672e4fe Cover the surface tables in storage-migrate's copy list
ci / vet-and-test (pull_request) Successful in 40s
Why: the copy-order coverage guard caught that backend PR #143's new tables (surfaces, surface_revisions, plugin_activation) were absent from the migration list - they would have been silently skipped on real migrations.

Evidence: TestCopyOrderCoversAllFixtureTables failed on the merged main and passes with the addition; full migrator suite green on SQLite and real PostgreSQL.

Impact: copyOrder list only; ordered after their referenced owners/projects.

Refs: backend PR #143; Dev U4 task.
2026-10-11 08:32:45 +00:00
weichen e3b1e0310e Merge pull request 'Add the storage-migrate tool: inventory, backup, and identity-governed migration (U4)' (#144) from feature/collab-storage-u4-migrator into main
ci / vet-and-test (push) Failing after 39s
2026-10-11 08:30:58 +00:00
weichen 40be71a3f1 Merge authoritative main into the task branch
ci / vet-and-test (pull_request) Failing after 41s
2026-10-11 08:30:53 +00:00
weichen 093b86fe7e Add the storage-migrate tool: inventory, backup, and identity-governed migration (U4)
ci / vet-and-test (pull_request) Successful in 38s
Why: plan U4/R7 - the SQLite-to-PostgreSQL path needs a managed entry that never replays SQLite dumps as PG SQL, fixes migration identity against a frozen snapshot, refuses foreign or non-empty targets, verifies per-table counts and ordered checksums, and keeps secrets out of reports (KTD14).

Evidence: integration tests against real PostgreSQL green twice (end-to-end with bool/NULL/Unicode/BLOB/sequence assertions, idempotent rerun, interrupted-table resume, foreign-data and wrong-snapshot refusals, unknown-owner block); SQLite unit tests for inventory/backup/coverage; full backend suite green on both tiers.

Impact: new cmd/storage-migrate binary (make build now emits bin/storage-migrate); store gains a DB() accessor documented for ops tooling; no server or runtime behavior change.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U4 and section 6; docs/guides/team-storage.md section 10.
2026-10-11 08:30:01 +00:00
weichen d79514302f Merge pull request 'Add the Surface store, activation guard, and EXT-17 host capability' (#143) from feature/generative-ui-u2 into main
ci / vet-and-test (push) Successful in 40s
2026-10-11 08:27:02 +00:00
weichen 70983e19fe Merge remote-tracking branch 'origin/main' into feature/generative-ui-u2
ci / vet-and-test (pull_request) Successful in 40s
2026-10-11 08:26:55 +00:00
weichen 75829fa442 Add the Surface store, activation guard, and EXT-17 host capability
ci / vet-and-test (pull_request) Successful in 40s
Why: Generative UI proposal U2 needs the host side of the contract:
owner-isolated surface storage with revision CAS, an execution-level
enable/disable ledger (default-off, epoch bumped on every transition so
in-flight submits after a disable are rejected), a single-choke-point
service that validates specs (strict decode + catalog rules) before
saving, event publish with pending recovery, the optional SurfaceHost
extension capability, and direct API endpoints that share the same
guard (no bypass by entrance).

Evidence: go build/vet clean; full go test ./... passes (SQLite);
surface store tests also pass under AGENT_PG_TEST_DSN (PostgreSQL);
surface.txt regenerated via the AST snapshot test.

Impact: Additive migration v22 (new tables only); existing endpoints and
plugin_grants semantics untouched; old consumers unaffected. U3+ (data
binding, actions, frontend) deliberately out of scope.

Refs: Documents proposal 2026-10-10-generative-ui-plugin.md (r2 U2);
Core PR #25; Dev task feature/generative-ui-u2
2026-10-11 08:26:24 +00:00
weichen c8d66ca3cd Merge pull request 'Make notice redelivery iterative instead of self-recursive' (#142) from feature/notice-delivery-recursion-fix into main
ci / vet-and-test (push) Successful in 39s
2026-10-11 08:12:14 +00:00
weichen 1ef1fdeac9 Make notice redelivery iterative instead of self-recursive
ci / vet-and-test (pull_request) Successful in 42s
Why: deliverNotices re-invoked itself synchronously from a defer after a failed startNoticeRun requeued the batch - a persistently failing delivery (busy conversation, missing row, slow DB) recursed without bound and overflowed the stack (real-PG CI job 2352, internal/runmgr).

Evidence: regression test TestNoticeDeliveryFailureIsBounded (missing conversation row) hangs on the old code and passes with the fix; full backend suite green on SQLite and against real PostgreSQL 18.6 + Redis; clean-HEAD reproduction isolated an unrelated count=2 admin-login artifact.

Impact: failed batches now return after requeue (retried on the next enqueue or restart); notices arriving during delivery are picked up by a bounded loop tail instead of defer recursion; no behavior change on the success path.

Refs: Dev PR #333; plan R8 real-database gate.
2026-10-11 08:12:03 +00:00
weichen 05c3447ebb Merge pull request 'Add shared execution budgets and the action-authz matrix (U3 slice 2)' (#141) from feature/collab-storage-u3-budget-authz into main
ci / vet-and-test (push) Successful in 38s
2026-10-11 05:18:20 +00:00
weichen f84733d128 Add shared execution budgets and the action-authz matrix (U3 slice 2)
ci / vet-and-test (pull_request) Successful in 43s
Why: plan U3 remainder (R8/KTD13) - execution admission must be bounded and shared (leases in the database, parents yield while waiting so budgets cannot deadlock parent-child), and project authorization must distinguish read/edit/approve/execute/admin with default-deny for unconfigured actions.

Evidence: full suite green on SQLite and against real PostgreSQL 18.6 + Redis (isolated instance); new tests - admission_pg_test (enforcement, per-user, concurrent no-oversell), runmgr admission tests (saturated-budget sync spawn progresses, background children share the budget, bounded queue 429), server action-matrix tests (default deny, legacy adapter read-only, action-aware authoritative + fail-closed).

Impact: admission_leases (v21) with atomic lease accounting under an advisory lock; runtime.execBudgetGlobal/PerUser/execQueueMax settings (all zero = off, historical behavior preserved); conversation turns, background and sync subagents share the pool, parents yield around synchronous children and HITL waits; queue-full rejects 429 before writing; server gains ProjectAction matrix + SetProjectActionAuthorizer with legacy EXT-8 read fallback.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U3 R8/KTD13; docs/guides/team-storage.md section 9.
2026-10-11 05:18:12 +00:00
weichen 3def525902 Merge pull request 'Close request atomicity: locked message append, card versions, atomic rate limit, recovery holds (U3 slice 1)' (#140) from feature/collab-storage-u3-atomicity into main
ci / vet-and-test (push) Successful in 40s
2026-10-11 04:38:02 +00:00
weichen 5cd67be982 Close request atomicity: locked message append, card versions, atomic rate limit, recovery holds (U3 slice 1)
ci / vet-and-test (pull_request) Successful in 41s
Why: the approved plan U3 (KTD3/KTD7/KTD10 plus minimal recovery isolation) requires concurrent submits to not silently overwrite, unknown-side-effect runs to never auto-resume, and the shared rate limiter to degrade conservatively instead of failing open.

Evidence: full backend suite green twice-relevant paths against real PostgreSQL 18.6 and Redis (isolated instance) and on SQLite; new tests - card_version_test (concurrent single winner), recovery_hold_test + recovery_holds_test (skip/release/fail-closed CRUD), ratelimit atomic-window and degraded-fallback tests.

Impact: AppendMessageWithRequest now allocates seq under the conversation advisory lock in one transaction; activity_cards gain version (v19) with SaveCardVersioned/GetCardVersion/CardVersion and cardsvc user mutations (status/pin/action/resolve-by-ref) return 409-mapped conflicts while system mirrors keep blind writes; recovery_holds (v20) with admin list/release endpoints and fail-closed resume skip; Redis limiter uses one Lua INCR+PEXPIRE script and falls back to a per-instance limit with a Degraded counter.

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U3 KTD3/KTD7/KTD10; R3/R6/R7/R8.
2026-10-11 04:37:50 +00:00
weichen f480c5b520 Merge pull request 'Retry transient SQLITE_BUSY at migration Begin and race on a materialized db' (#139) from feature/collab-storage-pg-tier-fix2 into main
ci / vet-and-test (push) Successful in 40s
2026-10-11 04:09:00 +00:00
weichen 6f20e6cc03 Retry transient SQLITE_BUSY at migration Begin and race on a materialized db
ci / vet-and-test (pull_request) Successful in 39s
Why: the connection-open phase (DSN pragmas incl. journal_mode on first open) can return SQLITE_BUSY immediately - the busy handler does not cover that phase - so four racing migrators failed in 0.03s on slow CI disks regardless of the 30s timeout (jobs 2293/2285).

Evidence: local -count=30 green (fast disk) vs CI deterministic failure at 0.03s (no timeout wait occurred); fix verified with -count=5 locally plus full store suite.

Impact: bounded retry (5 attempts, backoff) on Begin BUSY only - statements inside the IMMEDIATE transaction remain lock-protected; the racing test now materializes the file/WAL first, matching the KTD8 scenario (concurrent migrators on a provisioned database).

Refs: Dev PR #325; plan KTD8.
2026-10-11 04:08:50 +00:00
weichen efbd7b1d04 Merge pull request 'Raise the SQLite busy timeout to 30s for serialized migrators' (#138) from feature/collab-storage-pg-tier-fix1 into main
ci / vet-and-test (push) Failing after 40s
2026-10-11 03:42:55 +00:00
weichen 6f698d0ba5 Raise the SQLite busy timeout to 30s for serialized migrators
ci / vet-and-test (pull_request) Successful in 42s
Why: with _txlock=immediate the whole migration pass holds the write lock; four racing migrators on a slow CI disk exceeded the 5s busy timeout (job 2285: SQLITE_BUSY in TestMigrateRacingSQLite) although the local tier passes.

Evidence: go test -count=3 on the migration atomicity suite green locally; CI failure log cited in the Dev PR.

Impact: local SQLite tier only - lock contention waits up to 30s instead of erroring; no schema or API change.

Refs: Dev PR #325; plan KTD8.
2026-10-11 03:42:47 +00:00
weichen 2497508271 Merge pull request 'Add the team PG tier, atomic migrations and the connection budget (U2)' (#137) from feature/collab-storage-pg-tier into main
ci / vet-and-test (push) Successful in 41s
2026-10-11 03:11:12 +00:00
weichen 8633711b1d Add the team PG tier, atomic migrations and the connection budget (U2)
ci / vet-and-test (pull_request) Successful in 38s
Why: the approved collaboration plan U2 (KTD1/KTD8/KTD12) requires an explicit fail-closed team deployment tier, DDL-and-ledger-atomic migrations serialized across migrators, and a configurable PG connection budget with sanitized effective values.

Evidence: real PostgreSQL 18.6 + pgvector + Redis 8.10.2 local gate - full backend suite green twice with AGENT_PG_TEST_DSN/AGENT_REDIS_TEST_ADDR (isolated svc.sh instance collabtest:8897, redis 6399); SQLite suite green without env.

Impact: config tier+pool keys and validation; store PoolOptions + statement_timeout DSN param + single-transaction migration with PG advisory lock and SQLite _txlock=immediate + unknown-schema-version refusal; app tier guard fail-closed (team requires DSN+auth, no SQLite fallback); CLI flags and explain rows; tests for tier/pool parsing, atomicity fault-injection, racing migrators, old-schema upgrade; fixed pre-existing real-PG test defects (blind &search_path= appends in runmgr/pgbus helpers, pg_trgm per-schema install race, exhausted seq-conflict retries).

Refs: proposals/2026-10-11-collaboration-storage-and-distribution.md U2, KTD1/KTD8/KTD12; R2/R3/R7.
2026-10-11 03:10:55 +00:00
weichen ea4651b1f8 Merge pull request 'Remove the empty -ws shell directory after each apply' (#136) from feature/agent-s0-workspace-overlay into main
ci / vet-and-test (push) Successful in 39s
2026-10-11 02:20:47 +00:00
weichen bd4b3bca94 Remove the empty -ws shell directory after each apply
ci / vet-and-test (pull_request) Successful in 40s
Why: relaxAndRemove cleared the per-exec id directory but left the -ws
parent behind, so every conversation accumulated an empty husk - the
live probe caught it in the workspace listing.

Evidence: go test ./... green; TestOverlayCleansShellDir pins the
behavior with a real bwrap roundtrip.

Impact: One extra os.Remove of the (now empty) parent; failures are
best-effort as before.

Refs: Dev task feature/agent-s0-workspace-overlay (fix-up of #135).
2026-10-11 02:18:01 +00:00
weichen 0c34b3595e Merge pull request 'Add the S0 task workspace overlay: frozen base, private upper, CAS apply' (#135) from feature/agent-s0-workspace-overlay into main
ci / vet-and-test (push) Successful in 41s
2026-10-11 02:14:36 +00:00
weichen 01fd3b6fd4 Add the S0 task workspace overlay: frozen base, private upper, CAS apply
ci / vet-and-test (pull_request) Successful in 43s
Why: The sandbox tier wrote the session directory directly, so a buggy
or runaway script could leave the workspace half-mutated and nothing
bounded what a timeout had already written. The plan's first landing
shape (section 4.4) swaps the session bind for a frozen-base overlay:
each exec freezes the session dir (hardlink clone + sha256 baseline),
runs on a private upper, and only a clean exit promotes the change set
back via CAS (baseline hash match -> atomic replace/remove; externally
changed targets double-store as *.conflict-* instead of clobbering).
Timeouts and kills discard the upper entirely - half-written state
never lands.

Evidence: go test ./... green including bwrap integration roundtrips
(TestSandboxWorkspaceOverlayRoundtrip: add/modify/delete through a real
overlayfs upper with whiteout handling and ws cleanup;
TestSandboxWorkspaceOverlayTimeoutDiscards: partial writes discarded on
timeout) and unit anchors for freeze manifest (.agentx excluded), CAS
add/modify, conflict double-store, and the change summary.

Impact: Opt-in via --exec-workspace overlay / AGENT_SYSTEM_EXEC_WORKSPACE
(only with --exec-backend sandbox; fail-closed on unknown modes). The
model sees a one-line change summary appended to the tool output, so
discards and conflict files are visible. Default behavior is unchanged.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md section
4.4; Dev task feature/agent-s0-workspace-overlay.
2026-10-11 02:04:05 +00:00
weichen eda90ed310 Merge pull request 'Log R4 reminder injection for observable evidence' (#134) from feature/r4-injection-log into main
ci / vet-and-test (push) Successful in 55s
2026-10-11 01:24:40 +00:00
weichen 5ebd1bb525 Log R4 reminder injection for observable evidence
ci / vet-and-test (pull_request) Successful in 56s
Why: Reminders are attached to the request copy and never reach the
session journal, so a transcript grep proves nothing about whether the
R4 trigger fired - the first verification attempt was invalid evidence.
The dual-speed nudge already logs for exactly this reason.

Evidence: go test ./internal/runtime green; the follow-up live probe
must show the [runtime] r4 escalation reminder injected line in the
instance log.

Impact: Two log lines, nothing else.

Refs: Dev task feature/agent-e4-escalation-ledger (third fix-up).
2026-10-11 01:22:49 +00:00
weichen 6647d4879c Merge pull request 'Switch the R4 failure trigger to a sliding window' (#133) from feature/agent-e4-escalation-ledger into main
ci / vet-and-test (push) Successful in 42s
2026-10-10 18:44:09 +00:00
weichen 6709325c38 Switch the R4 failure trigger to a sliding window
ci / vet-and-test (pull_request) Successful in 40s
Why: The consecutive-failure counter never fired in the live suite (0
reminders despite 20 failures) because real failures interleave with
successes - each success reset the streak. The pattern to catch is
"spinning overall", not "N in a row".

Evidence: go test ./... green; the anchor test now covers scattered
failures (3 in the last 6 outcomes escalates, 2 does not, once per run);
the next live run must show the reminder actually appearing in the
transcript.

Impact: recentOutcomes ring (last 6) replaces consecToolFailures; the
zero-hit search trigger keeps its own threshold (2); message wording
matches the window semantics.

Refs: Dev task feature/agent-e4-escalation-ledger (second fix-up).
2026-10-10 18:43:03 +00:00
weichen ebec49db3d Merge pull request 'Restrict the source tag to material reads; never tag the model's own files' (#132) from feature/agent-e4-escalation-ledger into main
ci / vet-and-test (push) Successful in 43s
2026-10-10 18:22:26 +00:00
weichen 03b4fdabaf Restrict the source tag to material reads; never tag the model's own files
ci / vet-and-test (pull_request) Successful in 42s
Why: The first E4 live comparison regressed badly (toolFailures 10->24,
toolCalls 63->166) and the instance transcript shows the loop: tagging
kernel read_file/grep results put a trailing [src_N] line on the
model's own workspace files, the model treated it as external
tampering, and re-read/re-edit cycles cascaded into the read-before-
write and file-changed guards.

Why: The ledger only ever needed SOURCES - searches always, and
workspace reads only under inputs/ (the upload material area). Grep
cannot distinguish materials from self-produced files, so it stays
untagged.

Evidence: go test ./... green; new anchor TestEvidenceLedger
OnlyTagsMaterialReads pins the inputs/-only rule; the re-run live
comparison follows on the Dev side.

Impact: evidenceTools loses read_file/grep; isEvidenceRead gates
workspace reads to inputs/; everything else (search tags, escalation
reminders, prompts) unchanged.

Refs: Dev task feature/agent-e4-escalation-ledger (fix-up of #131).
2026-10-10 18:11:24 +00:00
weichen d116eda0bc Merge pull request 'Add R4 escalation reminders and the structured source-id evidence ledger (E4)' (#131) from feature/agent-e4-escalation-ledger into main
ci / vet-and-test (push) Successful in 41s
2026-10-10 17:50:21 +00:00
weichen 21b8461525 Add R4 escalation reminders and the structured source-id evidence ledger (E4)
ci / vet-and-test (pull_request) Successful in 43s
Why: The plan's E4 remainder under the MiniMax-only constraint: the
harness must detect the model going in circles and inject a
change-of-strategy instruction (R4 - the model cannot be trusted to
notice its own repeat-fail loop), and number sourcing needs addressable
evidence: tool results carry [src_N|tool] tags so "found in materials"
becomes "found in source src_N" for both the model and the fact-check
notice.

Why: Failure counting lives in one place (onEvent end, keyed by the
slot status) because tool execution is parallel; the ledger and tag
append are serialized by the same mutex, and escalation reminders fire
once per condition streak - following the dual-speed nudge finding that
MiniMax ignores vague reminders but follows concrete instructions.

Evidence: go test ./... green; new anchors TestEscalationOnConsecutive
ToolFailures, TestEscalationOnZeroHitSearches, TestEvidenceLedger
TagsSource, TestEvidenceLedgerSkipsNonEvidenceTools. Live before/
after comparison runs on the Dev side per ruling 6.

Impact: knowledge/web/tavily searches, fs_read_file and the workspace
read_file/grep results now end with a source tag and enter the run
ledger (2k-char head); two consecutive tool failures or two empty
search results inject a concrete change-of-strategy reminder;
base.deliverables v7 and agent-sub-research v3 explain the tags;
fact-check notice points at src results. Non-evidence tools and
failures are unchanged.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md (E4,
R4); Dev task feature/agent-e4-escalation-ledger.
2026-10-10 17:41:38 +00:00
weichen bd687d27d4 Merge pull request 'Add estimate-marking awareness and a second fact-check round (E3)' (#130) from feature/agent-e3-evidence-gate into main
ci / vet-and-test (push) Successful in 45s
2026-10-10 16:10:42 +00:00
weichen 06f5835211 Add estimate-marking awareness and a second fact-check round (E3)
ci / vet-and-test (pull_request) Successful in 41s
Why: The E1 live comparison was the only metric that got worse
(unverifiedFacts 7 -> 12): the gate gave one correction round and then
shipped bare numbers. The design (E3) calls for number sourcing with a
mechanically checkable marking escape: numbers labeled as
targets/assumptions are honest planning quantities, not fabricated
facts, so the gate must let them pass - and the model must be told that
marking is the sanctioned way out.

Why: The marking decision is clause-scoped, not a fixed window - a
window wrongly blessed "3,500" because a neighboring clause said
"target 85%" (caught by the existing test). The second round only
fires when the unverified set actually changed, because the
continuation budget is shared across all final gates and a repeat
block on identical tokens starves the downstream gates (also caught by
an existing test).

Evidence: go test ./... green; new anchors TestMarkedAsEstimate,
TestUnverifiedFactsMarksPass, TestFactCheckGateRounds; the live eval
gains an unverifiedUnmarkedFacts field sharing the same marking rule.

Impact: base.deliverables v6 adds the hard number-sourcing rule;
research/explore/general sub-agent profiles v+1 require citations for
key numbers; fact gate: marked-in-clause numbers pass, round 2
escalates to "last interception" with the marking instruction, then
passes through with the check line. Eval report keeps the strict
series and adds the E3-series field.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md (E3,
D4); Dev task feature/agent-e3-evidence-gate.
2026-10-10 16:08:19 +00:00
weichen 017b40f761 Merge pull request 'Bump the agent-squad pin (E1 chain)' (#129) from feature/bump-squad into main
ci / vet-and-test (push) Successful in 42s
2026-10-10 10:12:30 +00:00
weichen 288e4763a3 Merge authoritative main into the bump branch
ci / vet-and-test (pull_request) Successful in 40s
2026-10-10 10:11:07 +00:00
weichen 5714c739de Merge pull request 'Wire the dual-speed lane into the real run path: entry hint, escalating nudge, think-stripped cards' (#128) from feature/dual-speed-real-path into main
ci / vet-and-test (push) Successful in 41s
2026-10-10 10:09:51 +00:00
weichen 7ef6ef86cd Bump the agent-squad pin for the E1 chain
ci / vet-and-test (pull_request) Successful in 42s
2026-10-10 10:09:01 +00:00
weichen c449a176f9 Restore the E1 agent-native kernel pin c348441 over the stale submodule state captured by the merge
ci / vet-and-test (pull_request) Successful in 43s
2026-10-10 10:06:55 +00:00
weichen 8355794e45 Merge remote-tracking branch 'origin/main' into feature/dual-speed-real-path
# Conflicts:
#	internal/runtime/llm_direct.go
#	internal/runtime/llm_test.go
2026-10-10 10:05:38 +00:00
weichen b3f885f215 Strip inline think blocks from subagent summaries and card bodies
ci / vet-and-test (pull_request) Successful in 43s
Why: the round-6 real-model dual-speed run showed MiniMax writing its
reasoning as inline <think> blocks; finishChild's partsText fed them into
the background notice turn and the inbox result card verbatim.
Evidence: new runmgr test covers closed-block stripping with body preserved
and conservative handling of unclosed blocks; go test ./... 32 pkgs green.
Impact: partsText removes closed <think>...</think> spans before summary,
notice, and card assembly; unclosed blocks are kept as-is.
Refs: Dev task feature/dual-speed-real-path.
2026-10-10 10:01:15 +00:00
weichen 3f2c7c7033 Nudge long-running turns onto the dual-speed path via the kernel reminder hook
Why: the dual-speed slow lane shipped as an observable capability plus soft
prompt guidance, but on the real (non-mock) path models routinely grind
through heavy document tasks synchronously for minutes with zero user-visible
output - the pattern only engages when the model happens to think of it.
Evidence: five new unit tests over an injected clock cover fire-after-threshold,
quiet-before-threshold, disabled-by-setting, and the replied/delegated guard
paths (reminders observed in the captured kernel requests); full go test
./... 32 pkgs green. Real-model run recorded in the Dev task card.
Impact: runDirect tracks run start, user-reply, and background-delegation
state; reminders() injects a one-shot dual-speed nudge (SR-B10) once elapsed
time crosses runtime.dualSpeedNudgeSeconds (default 75s, 0 disables) while
the user has neither a reply nor a background lane. New setting registered.
Refs: Dev task feature/dual-speed-real-path; SR-B10 (RDM requirements/01 V2.1).
2026-10-10 09:11:18 +00:00
weichen eef60ff0b9 Merge pull request 'Enable parallel tool execution, fix delegate truncation, sandbox deps layer (E1)' (#127) from feature/agent-e1-parallel-fixes into main
ci / vet-and-test (push) Successful in 46s
2026-10-10 09:04:57 +00:00
weichen 4df81ef1da Enable parallel tool execution, fix delegate truncation, sandbox deps layer
ci / vet-and-test (pull_request) Successful in 43s
Why: Real runs were fully serial: the direct path never set the kernel's
parallel tool mode, delegate_task handed the parent only the child's
160-char summary (the full text was dropped), MiniMax streamed its
reasoning inside content <think> blocks, and the EXT-16 sandbox hid
~/.local so python-docx/openpyxl/pptx could not even import. All four
were measured defects (F1/F2/F3/F5) in the delegation/routing proposal.

Why: Serial-only mutual exclusion is not enough for order-sensitive
tools - the card-stream test failed with out-of-order appends - so the
ordering moved into the kernel (SerialTools partition, Agent-Native
#44); process events pair by CallID (#42) and tool results write back
through the injected call context (#43).

Evidence: go test ./... green, including new regression anchors:
TestDelegateToolReturnsFullText, TestToolEventsPairByCallID,
TestToolExecModeAndSerialSet, TestOpenLLMClientReasoningSplitAuto,
TestSandboxArgvDepsLayer, TestExecEnvJoinsPythonPath.

Impact: Direct runs now execute independent same-turn calls
(searches, reads, delegate_task, review_findings) concurrently by
default (runtime.parallelTools=false restores sequential); card flow,
plans, workflows, exec primitives and file edits keep model-declared
order. delegate_task returns the capped full child text. MiniMax-*
models get reasoning_split=true so thinking streams via
reasoning_content. --exec-deps-dir/AGENT_SYSTEM_EXEC_DEPS_DIR mounts a
read-only dependency layer into the sandbox and merges PYTHONPATH.
Bumps deps/agent-native to c348441.

Refs: proposal 2026-10-10-agent-delegation-routing-sandbox.md (E1,
F1-F5); Agent-Native #42/#43/#44; Dev task feature/agent-e1-parallel-fixes.
2026-10-10 09:02:10 +00:00
weichen c6912b63e7 Merge pull request 'Keep the final reply clean when checks continue or the verifier is skipped' (#126) from feature/final-reply-pollution-fix into main
ci / vet-and-test (push) Successful in 43s
2026-10-10 07:56:41 +00:00
weichen 18aa99cb95 Keep the final reply clean when checks continue or the verifier is skipped
ci / vet-and-test (pull_request) Successful in 1m1s
Why: real run run_b520bf2e3037 showed the user-facing reply being displaced by
internal bookkeeping - the fact-check correction round's closing text plus the
same-source verifier skip notice became the visible answer while the actual
summary was folded into the process timeline (frontend splits at the last
process part). Same tail pollution reproduced on every same-model run since.
Evidence: go test ./... -count=1 green (32 pkgs); new regression tests cover
the demotion of the pre-check summary and the skip notice becoming a check
part; three old-contract tests updated to assert notices land as process
checks instead of final text.
Impact: stop() fact-check branch now demotes the standing summary (aligned
with the gate and cap branches) and asks the model for a fresh full reply;
verifier errors record as process checks (pass-state for same-source skip,
fail-state for real errors) instead of StopDecision.Notice text; the cap
notice likewise becomes a failed check line.
Refs: Dev task feature/final-reply-pollution-fix; diagnosis in
.task/feature-dual-speed-tasks.md lineage and session analysis 2026-10-10.
2026-10-10 07:48:03 +00:00
weichen d2bb936784 Merge pull request 'Make background subagents observable as the slow lane of dual-speed tasks' (#125) from feature/dual-speed-tasks into main
ci / vet-and-test (push) Successful in 44s
2026-10-10 06:52:50 +00:00
weichen 43087a79f8 Make background subagents observable as the slow lane of dual-speed tasks
ci / vet-and-test (pull_request) Successful in 45s
Why: SR-B10 (RDM requirements/01 V2.1, ruling J-03) defines the task-level
dual-speed pattern: a fast lane gives immediate feedback while deep work runs
in background with queryable progress and results posted back to the inbox.
The base system had the background spawn/notice machinery but nothing the
user could see while it ran or collect afterwards, so heavy tasks just made
the user wait on a synchronous turn.
Evidence: go test ./... -count=1 green (32 pkgs); new runmgr tests cover the
card lifecycle on success (progress card -> done/ok with result summary and
artifact line, card.created/updated/status events) and on failure
(done/danger with the error), plus the cards==nil no-op path.
Impact: spawnBackground now registers an inbox progress card linked to the
child run and rewrites it with the result on completion; the delegate_task
description and base-method working prompt teach the fast-feedback-first
pattern; a #background mock demo exercises the whole path end to end.
Refs: RDM docs/requirements/01 UR-27/SR-B10; .tmp 03-V2 §任务快慢双速通用模式;
Dev task feature/dual-speed-tasks.
2026-10-10 06:40:00 +00:00
weichen 421fee81c3 Merge pull request 'Let in-proc plugins register the EXT-8 resource authorizer (fix rdm.* unknown-command)' (#124) from feature/ext8-inproc-authorizer into main
ci / vet-and-test (push) Successful in 41s
2026-10-10 02:08:04 +00:00
weichen 72b2098030 Let in-proc plugins register the EXT-8 resource authorizer (fix rdm.* unknown-command)
ci / vet-and-test (pull_request) Successful in 42s
Why: pluginRegistry never overrode RegisterResourceAuthorizer, so
in-proc extensions hit the declarative-branch rejection; register()
aborted wholesale and every plugin command vanished from the bus. The
RDM binary ships the EXT-8 member adapter since step-4, meaning the
served RDM process answers 'unknown or disabled command' for every
rdm.* call — caught by the MS1 re-acceptance smoke replay (fb/report/
confirm/lifecycle smokes all failed on rdm.baseline).

Evidence: TestInprocPluginRegistersAuthorizerAndCommands — red on the
parent commit (registration aborts, dispatch rejected, nil-slot denies),
green with the fix (command committed; the shared authz slot answers
alice-allow/bob-deny through projectAuthz). Full go test ./... green;
RDM MS0 acceptance re-run follows on the pin bump.

Impact: pluginRegistry.RegisterResourceAuthorizer fills the shared
hostcaps.authz slot (first registrant wins, mirroring the composition
root); compiled/declarative branch unchanged.

Refs: RDM G1-8 MS1 re-acceptance; EXT-8 (d89e748 seam); RDM #113.
2026-10-10 02:06:12 +00:00
weichen e03d3ac6c6 Merge pull request 'Narrow the blocking gate to hard facts and add fact-consistency checks (quality P5)' (#123) from feature/agent-quality-phase05 into main
ci / vet-and-test (push) Successful in 41s
2026-10-10 00:34:26 +00:00
weichen c802be812d Merge origin/main into feature/agent-quality-phase05
ci / vet-and-test (pull_request) Successful in 40s
2026-10-10 00:30:05 +00:00
weichen 9a531a61d9 Merge pull request 'Wire plugin slash aliases and the intent declaration surface (G2-2)' (#122) from feature/g2-2-slash-intents into main
ci / vet-and-test (push) Successful in 40s
2026-10-10 00:20:13 +00:00
weichen 9c4edc4dc9 Update the extension surface snapshot for AddIntent and IntentSpec
ci / vet-and-test (pull_request) Successful in 40s
Why: TestSurfaceSnapshot compares the exported-symbol list verbatim;
Registry.AddIntent and IntentSpec (SurfaceVersion 2.7.0-experimental)
are intentional additions.

Evidence: pkg/extension suite green; full go test ./... 32/32 packages
green.

Impact: surface.txt only.
2026-10-10 00:18:32 +00:00
weichen 9786dc1456 Wire plugin slash aliases and the intent declaration surface (G2-2)
Why: plan G2-2 requires plugin commands reachable via slash aliases
through the same bus and audit envelope (EXT-10②), and a natural-
language routing entry so '帮我写项目建议书' can reach the compile
flow. The agent-command lib now carries a validated Slash field
(lib PR #6, pin bumped to 4f583ca); the bus binds plugin-spec aliases
into the same slash table; intents are declared via the extension
registry and routed deterministically (normalized containment, longest
phrase first) to commands through the bus — the system.intent builtin
(/intent) is the chat-entrance dispatch point with a candidate list on
miss.

Evidence: TestPluginSlashAliasBindsAndDispatches — alias binds, case/
whitespace-normalized resolution, slash and button entrances hit the
same command handler with committed receipts; TestIntentDeclareMatch
Route — declare/validate, miss on unrelated text, normalized hit,
bus-dispatch through the chat entrance, longest-phrase priority;
server suite green with the dual-entry pair registered; full go test
./... 31/31 packages green.

Impact: cmdbus plugin-spec slash binding + IntentStore; extension
Registry.AddIntent + IntentSpec (SurfaceVersion 2.7.0-experimental);
pluginRegistry.AddIntent wires plugin-declared intents; server
builtin system.intent; command-pairs fixture extended.

Refs: plan G2-2 (line 366); EXT-10②; lib PR #6 (4f583ca).
2026-10-10 00:15:21 +00:00
weichen ef060d4ee5 Keep only hard facts in the blocking gate and add fact-consistency checks
ci / vet-and-test (pull_request) Successful in 40s
Why: User rulings 4/5 plus the P2-P4 live evidence. Keyword scoring dimensions
(readability, actionability) kept failing valid documents - normal Chinese
section headings flagged as column titles, loops of 4 corrections - because
they grade writing preferences, not verifiable facts. Unverified numbers
(~10-13 per run) remained the largest quality leak through every phase.
And the verifier runs on the same model as the worker (lightweight falls back
to chat), so "independent" verification was self-review. Separately, svc
seeding after boot never registered the tavily tools until a restart because
keys were snapshotted at startup.

Changes: quality.HardFacts (lint actually ran + zero errors) replaces the
full ScoreBytes projection in CheckArtifacts - keyword dimensions remain in
ScoreBytes for eval visibility only. New fact-consistency check
(runtime.factCheck, default on): extract percentages/dates/large numbers from
deliverable text, trace each against the material corpus (attachment text +
tool evidence); misses get exactly one correction round (fix, drop, or mark
as target/estimate), never a loop. runVerifier skips same-source model
verification unless runtime.verifierSameModel is set (hard facts + fact check
still run). Search tools now register lazily, reading tavily keys per call
(settings/seeding take effect immediately).

Evidence: SELF - TestHardFactsIgnoresKeywordDimensions,
TestHardFactsKeepsLintUnavailableBlocking, TestUnverifiedFactsAndNotice,
TestFactCheckContinuationOnce (fact note fires, corrected run passes);
go vet + go test ./... pass; live before/after on the golden task set
recorded on the phase05 task card per the policy clause.

Impact: Gate stops rejecting valid documents on stylistic grounds; invented
figures get one forced correction round; same-model verification no longer
pretends to be independent; tavily config is hot. No contract or storage
changes.
2026-10-10 00:04:57 +00:00
weichen 1a3fbfb3d8 Merge pull request 'Add the working-method prompt set and demote the compose rewrite (quality P4)' (#121) from feature/agent-quality-phase04 into main
ci / vet-and-test (push) Successful in 41s
2026-10-09 23:40:29 +00:00
weichen 6be8980c17 Add the working-method base prompt and demote the compose rewrite
ci / vet-and-test (pull_request) Successful in 38s
Why: User rulings 4 plus the P3 live evidence. The base library taught layout
standards but not how to work: no planning discipline (plan tool used in 2 of
31 tool runs), no retry-escalation rule (same failing call retried verbatim),
no check-API-first rule (phantom API calls), and nothing about edit
granularity - the P3 after-run showed one turn of 27 single-line edits plus a
self-review spiral (117 tools). The final-reply rewrite layer sees only
160-char summaries and fabricated chapters/promised unfinished work; with the
transcript now authoritative the working model already has full evidence, so
per ruling 4 the rewrite defaults off (kept as fallback when a work loop ends
with no user-facing text at all, plus opt-in setting).

Changes: new base.method module (plan-first for 3+ steps, switch approach
after two failures, check API signatures before scripting, batch edits with
multi_edit, persist to completion - never end on a promise, trace numbers/
dates/names back to source material or mark speculation, honest final
report); tool-description overrides for read_file/edit_file/grep/glob/
run_python/run_shell (dedicated tools over cat/sed, build pipeline for Word);
plan reminder now also nudges once when NO plan exists after several tool
turns (ignorable, once per run); runtime.composeFinal default true->false
with empty-body fallback.

Evidence: SELF - TestPlanlessNudgeFiresOnce (fires once, never repeats),
TestResolveSystemDefaultBase updated for the ninth base block; go vet +
go test ./... pass; live before/after on the golden task set recorded on the
phase04 task card per the policy clause.

Impact: Behavior-surface change only; no contract or storage changes.
2026-10-09 23:12:53 +00:00
weichen 5b9406dd91 Merge pull request 'Open kernel workspace tools and the markdown-source build pipeline (quality P3)' (#120) from feature/agent-quality-phase03 into main
ci / vet-and-test (push) Successful in 43s
2026-10-09 18:29:38 +00:00
weichen 77da63edac Open the kernel workspace tools and the markdown-source build pipeline
ci / vet-and-test (pull_request) Successful in 41s
Why: User rulings 1 and 3. The direct conversation path exposed only
run_shell/run_python, so any file edit meant rewriting and re-running whole
generation scripts (diagnosis: 24 calls with 9 failures in one turn, 6KB
scripts regenerated 8 times). The kernel already ships line-numbered reads,
exact edit/multi_edit with read-before-write, grep and glob; they were wired
only into the squad path. Deliverables had no editable source: docx output
came from one-shot python scripts, so "make section 3 more detailed" rebuilt
everything.

Now: WorkspaceToolsCheckpointed (sandbox = conversation workspace, checkpoint
hooks wired) is registered into the direct run and its decls join the tool
surface (plan mode keeps the read-only subset). Word deliverables get a
markdown-source pipeline - python3 -m housestyle.build renders .md
deterministically via docx_kit and runs lint + outline on every build
(write-time diagnostics, exit 1 on error so the model fixes the source and
rebuilds). run_python kits remain for xlsx/pptx/images (ruling 3).
base-tools/base-deliverables prompts and the house-style SKILL document the
new loop.

Evidence: SELF - TestDirectRunExposesKernelWorkspaceTools (read->edit->grep
against the real sandbox FS), TestDirectRunWithoutSandboxHidesKernelTools,
three new BuildTests in recipes (renders+lint passes, heading-skip errors
exit 1, CLI entry); go vet + go test ./... pass; live before/after on the
golden task set recorded on the phase03 task card per the policy clause.

Impact: Direct conversations can now read and edit workspace files
surgically; docx deliverables become edit-source + rebuild instead of
regenerate. No contract or storage changes.
2026-10-09 18:01:06 +00:00
weichen aa8eaf3f01 Merge pull request 'Align the nested agent-squad pin with the Dev top-level pointer' (#118) from feature/agent-quality-phase02 into main
ci / vet-and-test (push) Successful in 40s
2026-10-09 17:10:06 +00:00
weichen 52775178dc Merge origin/main into feature/agent-quality-phase02
ci / vet-and-test (pull_request) Successful in 41s
2026-10-09 17:05:34 +00:00
weichen 1a550895c3 Merge pull request 'Add the EXT-16 exec backend interface: direct status quo and bwrap sandbox' (#119) from feature/ext-16-exec-backend into main
ci / vet-and-test (push) Successful in 41s
2026-10-09 17:02:57 +00:00
weichen 4badecbf57 Add the EXT-16 exec backend interface: direct status quo and bwrap sandbox
ci / vet-and-test (pull_request) Successful in 41s
Why: Q2 ruled exec tools stay disabled for now but code execution must
return later; the recovery path treats model-generated code as
untrusted. This abstracts the interpreter launch point in
exec_interpreter.go into an ExecBackend port (direct = byte-identical
status quo; sandbox = bwrap isolation) so downstream products can
re-enable execution behind isolation without further base surgery.

Evidence: go build/vet clean; go test ./... 32/32 packages green;
sandbox escape negatives all blocked locally (bwrap 0.11.1): home
secret unreadable (tmpfs /home), sibling session unreadable
(session-only bind), network blocked (--unshare-net), 400-process
spawn storm capped by ulimit -u 256 (fork: retry), 60s sleep killed at
3s timeout, orphans die with the namespace; session-dir write visible
on host; direct backend regression (exit=7 / exit=timeout texts)
unchanged.

Impact: run_shell/run_python route through the backend; workflow
ScriptCommand and the squad kernel ExecConfig stay direct (follow-up
scope); config key execBackend + env AGENT_SYSTEM_EXEC_BACKEND + CLI
--exec-backend; extension.Options.ExecBackend (SurfaceVersion
2.6.0-experimental); DisableExecTools remains the top-priority
fail-closed switch.

Refs: proposal 2026-10-09-ext-g1-b1-b3-exec-backend.md section 4
(EXT-16, G2-8 prototype); Backend #110 (DisableExecTools).
2026-10-09 17:01:32 +00:00
weichen eb697ab1c0 Merge origin/main into feature/agent-quality-phase02
ci / vet-and-test (pull_request) Successful in 41s
2026-10-09 17:00:09 +00:00
weichen db5efc9b83 Align the nested agent-squad pin with the Dev top-level pointer
ci / vet-and-test (pull_request) Successful in 40s
Why: Dev's test-pins requires nested library pins to match Dev's top-level
gitlinks; Dev moved agent-squad to 8bbf850 (nested native pin alignment,
Agent-Squad #49) for the quality P2-I2 combination.

Evidence: SELF - 8bbf850 is Agent-Squad main head (#49 merged); combined
make test recorded on the Dev phase02 task card.

Impact: Pin only.
2026-10-09 16:54:48 +00:00
weichen 31965fd837 Merge pull request 'Point the agent-squad pin at the nested native-pin bump (Squad #49)' (#117) from feature/squad-pin-8bbf850 into main
ci / vet-and-test (push) Successful in 40s
2026-10-09 16:40:38 +00:00
weichen 5544a0304d Point the agent-squad pin at the nested native-pin bump (Squad #49)
ci / vet-and-test (pull_request) Successful in 40s
Why: Dev umbrella pin consistency requires backend's agent-squad gitlink
to match Dev's; squad main moved to 8bbf850 (nested agent-native pin at
3b92cb63, same as backend's own native pin after #115), so backend must
carry it before Dev can consume 8157e68+.

Evidence: gitlink-only change; Squad #49 merge 8bbf850 records
deps/agent-native=3b92cb63 (verified via ls-tree).

Impact: nested gitlink only; no backend source change.

Refs: Squad PR #49; Dev check-pins cascade; Backend #115/#116.
2026-10-09 16:39:21 +00:00
weichen 8157e68cf1 Merge pull request 'Decouple runmgr from cmdbus via view types in CommandSource (EXT-14 lint fix)' (#116) from feature/ext14-cmdsource-view into main
ci / vet-and-test (push) Successful in 42s
2026-10-09 16:33:23 +00:00
weichen 6337cd87f8 Decouple runmgr from cmdbus via view types in CommandSource
ci / vet-and-test (pull_request) Successful in 40s
Why: the EXT-14 wiring made runmgr import cmdbus for the CommandSource
interface signatures, which adds a new edge backend/runmgr ->
backend/cmdbus outside the architecture baseline ratchet; the Dev
umbrella lint (lint-architecture) rejects the gitlink bump because of
it. The dependency direction was always meant to be inverted: runmgr
defines the port, the composition root adapts the bus.

Evidence: go build/vet clean; go test ./... 32/32 packages green;
grep confirms no internal/cmdbus import left in non-test runmgr files;
Dev-side lint re-run scheduled on the gitlink bump PR.

Impact: runmgr.CommandTool/CommandRequest/CommandResponse are local
view types; app.busAdapter now converts cmdbus types at the boundary
(Entrance still fixed to agent by runmgr). No behavior change; the
runmgr tests speak the view types instead of cmdbus ones.

Refs: Dev umbrella make test failure 'architecture: new dependency
edges backend/runmgr backend/cmdbus'; proposal 2026-10-09-ext-g1-b1-b3
(EXT-14); Backend #114.
2026-10-09 16:31:43 +00:00
weichen 822cda2833 Merge pull request 'Make the kernel transcript the authoritative cross-turn context (quality P2-I2)' (#115) from feature/agent-quality-phase02 into main
ci / vet-and-test (push) Successful in 41s
2026-10-09 16:26:51 +00:00
weichen 8c17a2228b Merge origin/main into feature/agent-quality-phase02
ci / vet-and-test (pull_request) Successful in 42s
2026-10-09 16:25:22 +00:00
weichen 545dc404e5 Merge pull request 'Implement EXT-14: commands-as-tools dispatched through the command bus' (#114) from feature/ext-13-15-shared-runs into main
ci / vet-and-test (push) Successful in 43s
2026-10-09 16:19:11 +00:00
weichen 81da190ac1 Merge origin/main into feature/agent-quality-phase02
ci / vet-and-test (pull_request) Successful in 42s
2026-10-09 16:18:24 +00:00
weichen 020926234e Expose commands as in-run tools dispatched through the command bus (EXT-14)
ci / vet-and-test (pull_request) Successful in 43s
Why: RDM's capabilities are rdm.* commands; the agent run has no bridge
to them, so the V0.1 end-to-end goal (full-conversation operation) and
the agent-driven two-trees-one-net access are unreachable (Q4 option A).
Evidence: SELF - red tests first: TestCommandToolsProjected and
TestCommandToolDispatchesThroughBus failed on undefined symbols, then
green; TestCommandToolsFiltering and TestCommandToolDispatchEnvelope
cover projection filtering (risk default low, description falls back to
title, unexposed commands absent) and the full dispatch receipt with
Entrance=agent preserved through CanonicalEntrance; go test ./... 32/32
packages ok.
Impact: agent-command pin cc1ad77 -> 5c6dd0c (library PRs #4/#5: optional
ExposeAsTool field + "agent" entrance). cmdbus gains CommandTool view and
Bus.CommandTools; runmgr gains SetCommandSource with projection into
runCtx.Tools (dedup: registry tools win) and routing in ExecuteTool -
model calls dispatch through the bus with Entrance=agent, Actor/Owner =
conversation owner, ConversationID set; high-danger commands keep the
inline-confirm-card flow via the bus; unregistered source = current
registry behavior. Composition root wires a busAdapter. No
pkg/extension change.
Refs: proposal 2026-10-09-ext-g1-b1-b3 section 2; Agent-Command #4/#5
2026-10-09 16:14:57 +00:00
weichen 9fa1787d01 Merge pull request 'Implement EXT-13/15: shared retrieval inside runs and shared project listing/conversations' (#113) from feature/ext-13-15-shared-runs into main
ci / vet-and-test (push) Successful in 41s
2026-10-09 15:49:33 +00:00
weichen 59f62cace1 Nudge CI: TestToolCallDeltaBeforeExecution ordering flake (passes 8/8 locally)
ci / vet-and-test (pull_request) Successful in 41s
2026-10-09 15:47:36 +00:00
weichen 439865616a Share retrieval inside runs and open project listing/conversations to members (EXT-13/15)
ci / vet-and-test (pull_request) Failing after 44s
Why: A member's agent cannot see published project knowledge inside a run
(runCtx.Retrieve and hostcaps.search are owner-only), and members can
neither see shared projects nor open conversations in them - both are P0
prerequisites for RDM G1 (EXT-13/EXT-15, proposal
2026-10-09-ext-g1-b1-b3-exec-backend).
Evidence: Red tests first - TestRunContextRetrieveShared (undefined
SetSharedKnowledgeAllow), TestInprocHostKnowledgeSearchShared (0 shared
hits), TestSharedProjectListAndConversation (empty list) all failed before
and pass after; go test ./... 32/32 packages ok.
Impact: runmgr gains SetSharedKnowledgeAllow and routes Retrieve through
RetrieveShared when the predicate is registered (nil = unchanged); hostcaps
search consults the same EXT-8 slot; GET /api/projects appends
adapter-authorized shared projects (projectView with shared flag, deduped);
POST /api/projects/{id}/conversations falls back to
CreateConversationInProject when the own-project path misses and the
adapter authorizes; store gains ListAllProjects (candidates only, callers
must filter through the adapter) and CreateConversationInProject.
Revocation takes effect immediately on both paths; absence of an adapter
is byte-for-byte the previous behavior. No pkg/extension surface change.
Refs: docs/proposals/2026-10-09-ext-g1-b1-b3-exec-backend.md (EXT-13, EXT-15)
2026-10-09 15:44:48 +00:00
weichen 4675a25c18 Make the kernel transcript the authoritative cross-turn context
Why: Every turn rebuilt context from UI messages with 160-char tool-result
summaries and truncated arguments (quality diagnosis root cause 1). The kernel
session transcript already persists full-fidelity messages; P2-I2 switches the
direct path to it as the authority, per the approved plan (Documents
2026-10-09 §4 P2, user ruling 2).

First run (or after structural divergence) seeds the transcript once with the
full ctxasm assembly; later runs pass only the current user message. Dynamic
context - tail notes (personal instructions, session notes, world-book
entries), env update (date, workspace files) and newly hit task-layer prompt
modules - refreshes once per non-seed run via Reminders, so the frozen seed
system prompt stays accurate. Structural divergence (excluded messages,
variant switching, regeneration) changes a fingerprint and reseeds the
transcript from the message store - the regenerate/exclude/variant mapping.
Legacy transcripts without a fingerprint are adopted, not wiped, preserving
crash-pairing repair. Near the window threshold a 9-section compaction
(aux.session-compact, extractive fallback) summarizes older turns with the
kernel's append-only compaction entry; the kernel request-side micro-clear
still bounds within-run spikes. Backend's nested agent-native pin moves to
the reasoning-capture/replay kernel head.

Evidence: SELF - new tests TestContextRevisionStableOnAppendAndChangesOnDivergence,
TestTranscriptReseedsOnVariantSwitch, TestBuildCurrentUserCarriesAttachmentsAndKnowledge;
TestDirectRunSeedsStoreHistoryOnce updated to authoritative semantics; go vet
+ go test ./... pass; live before/after on the golden task set recorded on
the phase02 task card per the policy clause.

Impact: Direct conversations with a workspace sandbox now carry full tool
results, arguments and attachment text across turns; MiniMax thinking replays
from the transcript via the kernel change. No-conversation fallback keeps the
pre-I2 assembly path.
2026-10-09 15:38:33 +00:00
weichen 449d8a6943 Merge pull request 'Keep original materials and cleared results reachable across turns (quality P2-I1)' (#112) from feature/agent-quality-phase02 into main
ci / vet-and-test (push) Successful in 43s
2026-10-09 15:22:46 +00:00
weichen 9fe2a592f3 Merge origin/main into feature/agent-quality-phase02
ci / vet-and-test (pull_request) Successful in 43s
2026-10-09 15:12:16 +00:00
weichen 908fb253d7 Keep tool-result dumps out of the deliverable gate and cards
ci / vet-and-test (pull_request) Successful in 45s
Why: The live after-run (golden minutes-pm-tool) showed the truncation spill
artifact tool-run_shell-<call>.txt entering CheckArtifacts as a .txt
deliverable; its non-UTF-8 payload failed the scorer four times in a row.
These files exist to keep oversized tool output reachable - they are context
relief, not deliverables, and must not occupy delivery cards either.

Evidence: SELF - TestLatestDeliverablesKeepsLastPerName extended with a
tool-dump ref and asserts exclusion; go test ./internal/... passes; single-task
live rerun recorded on the phase02 task card.

Impact: latestDeliverables (gate input, verifier refs, reply deliverable list,
final artifact_ref cards) skips kind=data tool- prefixed refs. The spill files
remain artifacts for context backfill.
2026-10-09 14:56:34 +00:00
weichen 43297795f5 Keep original materials and cleared results reachable across turns
ci / vet-and-test (pull_request) Successful in 48s
Why: Real-run before/after evidence (phase01 card) showed later turns never
re-read the uploaded source (0 original reads in plan-followup) because the
attachment only existed in the current turn's injection and the context window
hard default was 32000 chars - a 25k-char transcript tripped the clear-on-first
request, replacing old tool results with a pointer the model has no tool to
follow. MiniMax is an interleaved-thinking model whose docs require <think>
blocks in assistant history; dropping them degrades later turns.

Evidence: SELF - new tests TestThinkReplayWrapsAssistantHistory,
TestContextWindowScalesFromTokens, TestThinkReplayEnabled,
TestPersistInputTextsAndListWorkspaceInputs; go test ./internal/... passes;
live before/after comparison on the golden task set recorded on the phase02
task card per the new policy clause.

Impact: models.chat.contextWindow/outputReserve/contextBuffer (tokens,
defaults 128k/16k/4k) drive the in-run clear threshold, converted to chars;
cleared tool results spill to <sandbox>/.agent/results with readable paths;
attachment parse text persists as inputs/<name>.md and the env section lists
all workspace inputs every turn; context.thinkReplay (auto|on|off, auto =
MiniMax) replays thinking blocks into request-side assistant history only.
2026-10-09 14:28:39 +00:00
weichen c0b52ac8bd Merge pull request 'Repair the quality-gate failure chain and add the live-model eval baseline' (#111) from feature/agent-quality-phase01 into main
ci / vet-and-test (push) Successful in 48s
2026-10-09 14:12:47 +00:00
weichen b68833f320 Read the eval target model from the user settings scope
ci / vet-and-test (pull_request) Successful in 47s
Why: The live report's model field came back empty because the settings
endpoint nests models under the user scope ({defaults, project, schema, user}),
not at the top level; the before-baseline report had to be annotated by hand.

Evidence: SELF - vet with -tags live passes; the scope shape verified against
GET /api/settings on the qa instance.

Impact: Future live reports carry the served model without manual annotation.
2026-10-09 13:04:08 +00:00
weichen d88dcb999f Wire the lint skills root from the runner instead of the app root
Why: lint-boundaries rejects new app->quality edges (architecture ratchet);
runtime already depends on quality for the scorer and holds SkillsDir, so the
injection belongs there. Same behavior, allowed dependency direction.

Evidence: SELF - go vet + go test ./internal/runtime ./internal/app pass;
lint-boundaries edge list unchanged.

Impact: quality.SetLintEnvRoot now called from LLMRunner.Execute when
SkillsDir is configured (idempotent, process-wide).
2026-10-09 12:55:54 +00:00
weichen 613fb27ba0 Add the live-model task eval harness (phase 0 baseline)
Why: No task-level measurement of real agent behavior existed, so the
failure chain documented in the diagnosis was invisible to CI. The harness
drives a configured backend over its public HTTP surface (upload, message,
poll, HITL auto-resolve), computes per-turn metrics (tool failures, full
regenerations, API-discovery calls, original-material reads, plan usage,
gate checks) and outcome checks (unverified facts vs material text, final
claims vs artifact headings), and writes a JSON + Markdown report.

Evidence: SELF - built behind the live build tag (CI skips); vet passes with
-tags live; drove the qa instance (remote main 1ebd46e) through the four
golden tasks end to end; report referenced on the phase01 task card.

Impact: New eval/live package (tasks.json + runner). Materials stay outside
the repo per eval/README; runs are opt-in via AGENT_EVAL_BASE.

Refs: Dev task feature/agent-quality-phase01; documents proposal
2026-10-09-agent-quality-improvement-plan.md §4 P0.
2026-10-09 12:49:44 +00:00
weichen 989379e924 Repair the quality-gate failure chain observed on real runs
Why: Real transcripts (dev instance, 62 assistant turns) showed the artifact
gate silently off and self-deadlocking. lintFile ran python3 -m housestyle.lint
without the skills PYTHONPATH, so the import failed and nil was treated as "no
issues" (same docx: 9 lint errors with PYTHONPATH, 0 without). stop() re-scored
every artifact registered in the run, including superseded versions, so a fixed
document could never pass and the composed reply reported the old verdict.
ctxasm rendered history artifacts as "[产出工件 …]" - the exact pattern
ClaimGate treats as a forged tool marker, so models copying the format were
rejected. scoreReadable flagged table cells like "项目概述撰写" as column
headings. Prompts and recipes said "python -m" (no python on PATH; exit 127)
and housestyle.summarize was invoked as a module that does not exist.

Evidence: SELF - new tests TestLintUnavailableFailsVisibleWhenSkillsRootSet,
TestLintCommandEnvCarriesSkillsRoot, TestDocxHeadingMarkedAndTableCellNotFlagged,
TestLatestDeliverablesKeepsLastPerName, TestRecipesUseRunnablePythonCommands,
TestLibraryUsesRunnablePythonCommands; go test ./internal/... passes; manual:
python3 -m housestyle.outline_cli reads a real artifact docx cleanly, lint
without PYTHONPATH exits 1 with ModuleNotFoundError and is now surfaced as
自检未运行 when the skills root is configured.

Impact: compose root injects quality.SetLintEnvRoot(<data>/skills). Final-turn
checks score only the newest artifact per name; deliverable lists and reply
cards follow the same dedupe. History artifact placeholders no longer collide
with the forged-marker regex. docx text extraction marks Title/HeadingN styles
so readability only judges real headings, with line numbers in the reason.
Check reminders no longer invite the model to ignore failed checks. Oversized
tool output is truncated by runes, not bytes.

Refs: Dev task feature/agent-quality-phase01 (quality gap diagnosis 2026-10-09).
2026-10-09 12:34:21 +00:00
weichen f1c9cc0644 Merge pull request 'Keep DOCX tracked-change/hyperlink text and add an exec hard switch' (#110) from feature/v020-docx-and-exec-switch into main
ci / vet-and-test (push) Successful in 43s
2026-10-09 10:39:18 +00:00
weichen 68c1a7272a Keep tracked-change and hyperlink text in DOCX and add an exec hard switch
ci / vet-and-test (pull_request) Successful in 43s
Why: The DOCX parser only read w:r runs that were direct children of w:p, so
text inside w:ins, w:hyperlink and w:fldSimple was dropped with no warning,
violating the no-silent-degradation rule. Separately, runtime.exec is a
user-writable setting, so multi-user deployments could not reliably turn the
exec primitives off.
Evidence: SELF - red tests TestParseDocxTrackedChangesAndHyperlinks and
TestExecHardDisabledOverridesSettings failed before and pass after; backend
go test ./... (32 packages) passes; a mock-mode binary started with
--disable-exec-tools hides run_shell/run_python/register_artifact from
/api/tools even after the user sets runtime.exec=true.
Impact: DOCX paragraphs are decoded from the token stream at any depth;
deleted/moved-from text, field codes and mc:Fallback are excluded and
revised documents carry an explicit warning. New Options.DisableExecTools
(SurfaceVersion 2.5.0-experimental), CLI --disable-exec-tools, config key
disableExecTools and env AGENT_SYSTEM_DISABLE_EXEC_TOOLS; zero value keeps
current behavior.
Refs: Agent-System-RDM docs/proposals/2026-10-09-V22-review-and-implementation-plan.md (G0-7, B-5)
2026-10-09 10:36:02 +00:00
weichen 399e2e53e6 Merge pull request 'Close the EXT-8 remaining blocks: vector sharing, hostapi face, citation consistency' (#109) from feature/ext8-remaining-shared-reads into main
ci / vet-and-test (push) Successful in 38s
2026-10-08 17:22:46 +00:00
weichen 92ede3b970 Close the EXT-8 remaining blocks: vector sharing, hostapi face, citation consistency
ci / vet-and-test (pull_request) Successful in 37s
Why: EXT-8 closure needs the vector channel shared (not just fulltext), the /host/v1 knowledge face honoring the authorizer, and citation-ref-to-artifact consistency pinned as an acceptance test.
Evidence: TestRetrieveSharedVectorChannel (hash embedder: authorized shared vector hit retrievable, denied yields zero), TestSharedSearchFace (hostapi switches to SharedSearch when wired), TestSharedCitationRefDownloadable (shared hit -> Ref -> artifact content download for the authorized member); go test ./... 32/32 packages ok.
Impact: store.SearchKnowledgeVectorsShared; rag.Pipeline.RetrieveShared (own + shared fulltext + shared vector, RRF-merged); server searchKnowledge prefers RetrieveShared when the retriever implements it; hostapi Server.SharedSearch wired in the composition root through the same authorizer slot. Default behavior unchanged.
Refs: EXT-8 proposal acceptance; slices 1-2.
2026-10-08 17:16:10 +00:00
weichen 46c2abbca8 Merge pull request 'Wire the EXT-11 scoped-host factory through the composition root' (#108) from feature/ext-11-scoped-host-factory into main
ci / vet-and-test (push) Successful in 39s
2026-10-08 17:08:07 +00:00
weichen bbe39b475f gofmt the ext-11 files
ci / vet-and-test (pull_request) Successful in 42s
Why: The implementation commit left two files unformatted; keep the tree gofmt-clean.
Evidence: gofmt -l now empty; targeted test still green.
Impact: Formatting only.
Refs: Follow-up to the wiring commit.
2026-10-08 17:02:16 +00:00
weichen 6162817ea3 Wire the scoped-host factory through the composition root
Why: The red run pinned the missing wiring; downstream recovery paths need a legal handle outside dispatch.
Evidence: TestScopedHostFactory green (fail-closed before wiring; Owner/PluginID, KV roundtrip, prefix enforcement, empty-arg rejection after); go test ./... all packages ok.
Impact: wireScopedHostFactory injected in app.Run after hostcaps assembly; no dispatch-path change.
Refs: EXT-11; red run at 182d8b5.
2026-10-08 17:01:51 +00:00
weichen 182d8b5143 Add the EXT-11 scoped-host surface with a red factory test
Why: EXT-11 is accepted; the surface (SetScopedHostFactory/NewScopedHost, SurfaceVersion 2.4.0-experimental) must land with the behavioral red pinned - wiring absent, NewScopedHost fails closed with ErrNotLinked.
Evidence: TestScopedHostFactory red at the wired-factory assertion; row follows on the task card.
Impact: pkg/extension additive surface (surface.txt +2 symbols); internal wireScopedHostFactory stub. Behavior unchanged.
Refs: EXT-11 proposal; task card.
2026-10-08 17:00:53 +00:00
weichen 8a9a354649 Merge pull request 'Implement shared knowledge retrieval, card mirror switch, and SSE project gate (EXT-8 slice 2)' (#107) from feature/ext8-slice2-shared-retrieval into main
ci / vet-and-test (push) Successful in 41s
2026-10-08 16:24:19 +00:00
weichen 1de663a8d3 Implement shared knowledge retrieval, card mirror switch, and SSE project gate
ci / vet-and-test (pull_request) Successful in 40s
Why: The three pinned reds define EXT-8 slice 2 - adapter-granted cross-owner knowledge must be retrievable, conversation cards must stay out of public project streams when the host opts out, and project-stream subscription must honor the authorizer.
Evidence: TestConvCardMirrorSwitch, TestSharedKnowledgeRetrieval, TestProjectSSEScopeGate green; go test ./... 32/32 packages ok; rows follow on the task card.
Impact: store gains predicate-based SearchKnowledgeShared (FTS/LIKE shared variants + PG); searchKnowledge appends adapter-authorized hits; cardsvc emit gate (no publish = no persistence = clean replay); events project scope consults authorizeSharedProject. Default behavior unchanged everywhere.
Refs: EXT-8/EXT-12 proposals; red run at surface commit.
2026-10-08 16:18:07 +00:00
weichen d3381b208b Add the EXT-12 mirror switch surface and slice-2 red tests
Why: Knowledge retrieval cross-owner and conversation-card mirroring are the registered EXT-8 slice-2 items; the fields/wiring land with the acceptance reds pinned first.
Evidence: mirror red (project channel still emitted with switch off); shared-retrieval red (grant yields 0 hits); SSE-scope red (grant yields non-200); rows follow on the task card.
Impact: Options.DisableConvCardMirror passthrough (extension/link/app); cardsvc.MirrorConvCards (default true = current behavior); SurfaceVersion 2.3.0-experimental. Behavior unchanged.
Refs: EXT-8/EXT-12 proposals; task card.
2026-10-08 16:08:20 +00:00
weichen c5575aab27 Merge pull request 'Implement EXT-9 idempotent knowledge publish and revoke with the upload index switch' (#106) from feature/ext-9-index-publish into main
ci / vet-and-test (push) Successful in 40s
2026-10-08 16:01:50 +00:00
weichen fbc924c240 Implement EXT-9 idempotent publish, prefix-enforced revoke, and the upload index switch
ci / vet-and-test (pull_request) Successful in 41s
Why: The red acceptance tests pin the EXT-9 semantics - held material must not enter retrieval until published, revoke must stop retrieval immediately and idempotently, manual sources must stay out of plugin reach.
Evidence: TestKnowledgePublishIdempotentAndRevoke and TestUploadPreIndexSwitch green; go test ./... 32/32 packages ok; rows follow on the task card.
Impact: hostcaps KnowledgePublish (per-ref row+vector cleanup then existing write path) and KnowledgeDelete (plugin:<id> prefix enforced); store DeleteKnowledgeBySource with vector cleanup; upload indexing gated by DisableUploadIndexing (default false = unchanged).
Refs: EXT-9 proposal; red run at surface commit.
2026-10-08 15:55:02 +00:00
weichen 1ee32822c3 Add the EXT-9 publish/revoke surface and red acceptance tests
Why: EXT-9 is the other step-4 hard prerequisite; the surface (Options.DisableUploadIndexing passthrough, Host.KnowledgePublish idempotent replace, Host.KnowledgeDelete prefix-enforced revoke, SurfaceVersion 2.2.0-experimental) must land with the acceptance reds pinned before implementation.
Evidence: app red = publish not implemented; server red = switch-on still indexes (hits=1); rows recorded on the task card.
Impact: pkg/extension additive surface (surface.txt updated); internal Options passthrough; Server.DisableUploadIndexing field; host method stubs. Behavior unchanged.
Refs: EXT-9 proposal; task card.
2026-10-08 15:53:39 +00:00
weichen 251c204a6e Merge pull request 'Consult the EXT-8 resource authorizer in artifact read paths' (#105) from feature/ext-8-resource-authorizer into main
ci / vet-and-test (push) Successful in 42s
2026-10-08 15:42:12 +00:00
weichen a4783d18e7 Consult the EXT-8 authorizer in project artifact read paths
ci / vet-and-test (pull_request) Successful in 41s
Why: With the surface and injection seam in place, the read endpoints must fall back to the adapter when owner-scoped lookup misses, keeping default-deny and fail-closed semantics.
Evidence: TestProjectResourceAuthorizerSharesReads green (default deny, shared list/meta/content via adapter, third-party denied, authorizer error fail-closed); go test ./... 32/32 packages ok.
Impact: listArtifacts/getArtifact/artifactContent consult authorizeSharedProject; store gains gated helpers (GetArtifactByID/ArtifactContentByID/ListProjectArtifactsAll) documented internal-only; writes untouched; SSE project stream and project card aggregate deliberately deferred to EXT-12 (conversation-card mirroring privacy).
Refs: EXT-8; red run at d89e748; task card.
2026-10-08 15:34:18 +00:00
weichen d89e74880b Add the EXT-8 authorizer surface and a red sharing test
Why: EXT-8 is accepted (Documents #49); the surface (Registry.RegisterResourceAuthorizer, ResourceAuthorizer, SurfaceVersion 2.1.0-experimental) plus the Server injection seam must land before the read-path consultation, and the behavioral red run proves the gap.
Evidence: go test ./internal/server -run TestProjectResourceAuthorizerSharesReads FAIL (shared read 404 with adapter set); default-deny section passes; row recorded on the task card.
Impact: pkg/extension additive surface (surface.txt updated); projectAuthz slot in hostcaps; Server.ProjectAuthz + setter; app wiring after compiled load; declarative registry fails closed. Read behavior unchanged.
Refs: EXT-8 in Documents 2026-10-08-ext-c1-c7-rdm-shared-contracts; task card.
2026-10-08 15:32:04 +00:00
weichen e0b1c27f1f Scope command audit storage and queries to the requesting owner
ci / vet-and-test (pull_request) Successful in 42s
Why: GET /api/commands/audit leaked every tenant's audit rows (command ids, entrances, conversation ids, param digests) to any authenticated user; owner was never recorded nor filtered.
Evidence: Red run at a7cc69b FAIL (alice sees bob's row); after fix go test ./... 32/32 packages ok, including TestCommandAuditOwnerScoping and the v18 backfill test (rows recorded on the task card).
Impact: v18 migration adds command_audit.owner with actor-based backfill (empty-actor rows stay invisible via API, kept in-table); AppendAudit/ListAudits gain owner params; cmdbus port and fakes updated; handler passes authenticated owner. No public extension surface change.
Refs: Task card R1; U0 proposal section 5 C7 finding.
2026-10-08 14:02:19 +00:00
weichen a7cc69b61e Add a red test for cross-tenant command audit leakage
Why: GET /api/commands/audit returns every tenant's audit rows; the regression test pins the desired owner-scoped behavior before the fix.
Evidence: go test ./internal/server -run TestCommandAuditOwnerScoping FAIL (owner alice sees bob's row; output recorded on the task card).
Impact: Test only; suite red on this commit by design.
Refs: Task card R1; U0 proposal section 5 C7 security finding.
2026-10-08 13:57:30 +00:00
weichen fe7c245f5c Merge pull request 'Point the nested squad pin at the a2a adapter and pin alignment' (#103) from feature/q3-squad-pin into main
ci / vet-and-test (push) Successful in 44s
Merge pull request 'Point the nested squad pin at the a2a adapter and pin alignment' (#103) from feature/q3-squad-pin into main
2026-10-08 04:03:24 +00:00
weichen aa03de4045 Point the nested squad pin at the a2a adapter and pin alignment
ci / vet-and-test (pull_request) Successful in 43s
Why: Dev-level pin consistency (check-pins hard-errors on nested drift);
squad main advanced to 819b0c5 (a2a adapter module #47, StepSpec.Brief #46,
nested agent-native pin #48).

Evidence: full backend suite green against 819b0c5 (SELF); CI authoritative
(GATE).

Impact: nested deps/agent-squad 633e853 -> 819b0c5; no backend source
changes (backend consumes squad via wfcompile/squadadapter surfaces which
are unchanged since #99-#102).

Refs: Q3 plan WP4/WP6; Dev card .task/feature-agent-stack-q3.md
2026-10-08 04:01:20 +00:00
weichen 4791b2e760 Merge pull request 'Add the task-level A/B dual-run harness with demo.brief evidence (Q3 WP5)' (#102) from feature/q3-ab-harness into main
ci / vet-and-test (push) Successful in 47s
Merge pull request 'Add the task-level A/B dual-run harness with demo.brief evidence (Q3 WP5)' (#102) from feature/q3-ab-harness into main
2026-10-08 03:49:31 +00:00
weichen 196c8c09f7 Add the task-level A/B dual-run harness with demo.brief evidence (Q3 WP5)
ci / vet-and-test (pull_request) Successful in 42s
Why: Q3 WP5 requires a same-batch dual-run harness (direct vs squad
compiled graph) that records differences without demanding byte-identical
output, feeding DecideSwitch evidence for the runtime.entry default-switch
decision.

Evidence: eval/ab_test.go mechanics (theme-based scoring, status/theme/
latency diff notes, P50, DecideSwitch wiring, RunFunc error tolerance,
empty batch); wfcompile TestABDemoBriefDualRun runs the demo.brief scenario
on both entries (direct = scripted single-agent answer; squad = real
RunCompiled with stub backend and judge) and logs the report: both sides
100/100, summary-form difference recorded, decision '切换条件已满足…合入前
需要产品确认'. Full backend suite green (SELF); CI authoritative (GATE).

Impact: new eval/ab.go harness (pure mechanics, no runtime dependency);
no product default is flipped - runtime.entry stays direct until product
confirmation per the DecideSwitch report semantics (evidence recorded in
the task card and PR).

Refs: Q3 plan WP5 (2026-10-07-agent-stack-q2q3-plans.md); task card
.task/feature-agent-stack-q3.md
2026-10-08 03:46:54 +00:00
weichen 2e42ed83f0 Merge pull request 'Complete the delegation task brief with output format and tool sections (Q3 WP3)' (#101) from feature/q3-delegation-template into main
ci / vet-and-test (push) Successful in 1m15s
Merge pull request 'Complete the delegation task brief with output format and tool sections (Q3 WP3)' (#101) from feature/q3-delegation-template into main
2026-10-08 03:33:06 +00:00
weichen 7642dcec24 Complete the delegation task brief with output format and tool sections (Q3 WP3)
ci / vet-and-test (pull_request) Successful in 48s
Why: Q3 acceptance requires the delegation template to state goal, output
format, tools and boundaries; SubagentFactory's brief had four of six
sections and the kernel Request gained the two carriers (agent-native #40).

Evidence: adapter_test task-brief assertions extended (输出格式/工具面
sections present with declared values); full backend suite green (SELF);
CI authoritative (GATE).

Impact: nested deps/agent-native 1ac21c6 -> 37c5e6f (Request.OutputFormat/
Tools + delegate_task schema); SubagentFactory renders the two sections
when declared, omits them when empty (no behavior change for existing
callers).

Refs: Q3 plan WP3; task card .task/feature-agent-stack-q3.md
2026-10-08 03:31:03 +00:00
weichen 76b9cf36a8 Merge pull request 'Retire the legacy workflow executor and switch squad steps to agent-native (Q3 WP2)' (#100) from feature/q3-wp2-retire into main
ci / vet-and-test (push) Successful in 50s
Merge pull request 'Retire the legacy workflow executor and switch squad steps to agent-native (Q3 WP2)' (#100) from feature/q3-wp2-retire into main
2026-10-08 03:25:44 +00:00
weichen 70afe40b00 Retire the legacy workflow executor and switch squad steps to agent-native (Q3 WP2)
ci / vet-and-test (pull_request) Successful in 47s
Why: Q3 WP2 retires the SpawnSubagent-compression execution path: all
WorkflowDef execution now goes through the wfcompile compiled graph, and
squad steps run on the agent-native kernel backend instead of the
agent-system step backend.

Evidence: runtime-level P4 parity tests (workflow_compiled_test.go):
registered branch lookup/errors, planned branch confirmation gate +
dynamic 8-step cap fail-closed before any HITL, model script switch
default-off, missing-slot HITL collection with compile-time rendering,
judge stream routed to the configured lightweight model (nil = fail-closed
verifier-unavailable). squadadapter TestOrchestrateRunsOnKernelStub:
four-phase skeleton runs end-to-end on the kernel stub (write_file
materialization, seeded skills, exec allowlist). server TestSquadOrchestration
reworked: kernel steps run in-process (no subagent child runs). Full
backend suite green; wfcompile/squadadapter/runmgr green with -race
(pre-existing llm_test fake race on main recorded as debt, CI does not
gate -race).

Impact: DELETED runtime/workflow.go + workflow_script.go (legacy step loop,
critic, script host) and their tests; squadadapter stepBackend
'agent-system' + profileForRole deleted; BuildBackends exported and
registers only agent-native; Orchestrate DefaultBackend -> agent-native;
builtin roles declare agent-native with write_file added to allowlists and
writer run_shell -> run_command (kernel tool name); runmgr default workflow
runner -> CompiledWorkflowRunner; mock #workflow demo runs compiled
(host-only steps). Model script branch keeps the single script-level
confirmation (no extra plan confirmation), matching legacy.

Refs: Q3 plan WP2 (2026-10-07-agent-stack-q2q3-plans.md); task card
.task/feature-agent-stack-q3.md
2026-10-08 03:24:30 +00:00
weichen a1aba1527a Merge pull request 'Add the WorkflowDef-to-squad compiler with host intent steps (Q3 WP1)' (#99) from feature/q3-wfcompile into main
ci / vet-and-test (push) Successful in 42s
Merge pull request 'Add the WorkflowDef-to-squad compiler with host intent steps (Q3 WP1)' (#99) from feature/q3-wfcompile into main
2026-10-08 02:38:39 +00:00
weichen 017c96baa5 Add the WorkflowDef-to-squad compiler with host intent steps (Q3 WP1)
ci / vet-and-test (pull_request) Successful in 44s
Why: Q3 WP1 requires core.WorkflowDef workflows to run as compiled squad
graphs (P4 limits and DeniedTools enforced at the compile entry) so the
legacy SpawnSubagent-compression executor can be retired in WP2.

Evidence: internal/wfcompile conformance (zero model, zero network; script
step runs the real codeorch host like the legacy tests): demo.brief shape
(hitl/prompt+critic/artifact/card mapping, evaluator-optimizer loop budget
= maxRetries+1, needs chain), P4 limits (dynamic >8 rejected, dynamic
script rejected, DeniedTools, critic >3, workflow retries >6), demo.brief
E2E (critic fail->retry->pass, drops record, artifact from step-ref file,
card, hitl sequence without plan confirmation for registered workflows,
BB out_length visible in the draft task brief), critic exhaustion fails
closed, demo.script E2E (plugin script exempt from confirmation, wf.tool/
ask/artifact/step callbacks), workflowScripts off fails, model script
double confirmation with reject paths. Full backend suite green (SELF);
CI authoritative (GATE).

Impact: new package internal/wfcompile (compiler + host IntentRunner +
RunCompiled driver); nested deps/agent-squad 06934bc -> 633e853 (StepSpec
.Brief). No existing backend behavior changes (legacy executor untouched
until WP2).

Refs: Q3 plan WP1 (2026-10-07-agent-stack-q2q3-plans.md); task card
.task/feature-agent-stack-q3.md
2026-10-08 02:36:39 +00:00
weichen 06934bccab Merge pull request 'Point the nested squad pin at the Q2 topology library completion' (#98) from feature/q2-squad-pin into main
ci / vet-and-test (push) Successful in 43s
Merge pull request 'Point the nested squad pin at the Q2 topology library completion' (#98) from feature/q2-squad-pin into main
2026-10-08 01:35:07 +00:00
weichen 30bbf5483f Point the nested squad pin at the Q2 topology library completion
ci / vet-and-test (pull_request) Successful in 42s
Why: Q2 (topology library) landed in Agent-Squad main across WP1-4
(12 builtin topologies with scripted-backend conformance); backend consumes
squad via squadadapter and must track the completed surface.

Evidence: GOWORK=off go build ./... and go test ./... -count=1 green against
squad bcc8d08 (SELF); backend CI on the PR head is authoritative (GATE).

Impact: nested deps/agent-squad c462bc4 -> bcc8d08 (additive squad API:
topology registry, RunOptions knobs, mailbox/handoff/board state); no
backend source changes required.

Refs: Q2 plan (2026-10-07-agent-stack-q2q3-plans.md); Dev task card
.task/feature-agent-stack-q2.md
2026-10-08 01:32:45 +00:00
weichen 6e02a1f574 Merge pull request 'Point the nested core pin at the TypeScript contract authority update' (#97) from feature/n3-core-pin into main
ci / vet-and-test (push) Successful in 41s
Point the nested core pin at the TypeScript contract authority update
2026-10-07 18:05:37 +00:00
weichen 26bd9286ad Make the rewind happy test tolerant of the settle-release window
ci / vet-and-test (pull_request) Successful in 41s
Why: run terminal status persists before the conversation claim is
released; on the slow CI runner the rewind POST landed inside that
window and got a 409. Retry the POST until the claim frees (bounded).
Evidence: go test -count=5 ./internal/server/ -run TestRewindFilesHTTPHappy
green; full suite green.
Impact: test-only.
Refs: Agent-System-Backend#94 #97
2026-10-07 18:04:08 +00:00
weichen 31a4e7576a Point the nested core pin at the TypeScript contract authority update
ci / vet-and-test (pull_request) Failing after 40s
Why: Dev pins modules/core at a253a40b (the TS contract authority);
the combination check requires the nested pin to follow.
Evidence: tree-only pin move.
Impact: none.
Refs: Agent-System-Core#24
2026-10-07 17:55:04 +00:00
weichen e59b25c816 Merge pull request 'Split usage accounting end to end, retry by default, and take the WP6 pins (N3 WP6)' (#96) from feature/n3-usage-retry-wp6 into main
ci / vet-and-test (push) Successful in 41s
Split usage accounting end to end, retry by default, and take the WP6 pins (N3 WP6)
2026-10-07 17:19:27 +00:00
weichen 8c5612a0a3 Split usage accounting end to end, retry by default, and take the WP6 pins (N3 WP6)
ci / vet-and-test (pull_request) Successful in 44s
Why: subagent runs recorded zero tokens, cache hits were invisible in
run records, the usage event skipped terminal paths, and stream retry
existed only on the direct-chat path (audit N20 residual).
Evidence: go test -count=1 ./... green; TestRetryAttemptsMapping
(setting 0=off→-1, N→N), TestDirectRunUsageSplitOnRecord (Last* and
cache land in the run usage), TestSubRunAccountsTokens (subagent
accounting), TestAddUsageAccumulatesSplitFields (handle chain); pins
carry Agent-Native#35/#37/#38/#39, Agent-Squad#41, Core Usage fields.
Impact: run.Usage carries Last*/cache on every path; subagent runs are
accounted; runtime.llmRetries maps onto the library default (setting 0
disables via -1); the verifier wraps retry+fallback explicitly; retry
events surface as process parts.
Refs: Agent-Native#35-#39, Agent-Squad#41, N3 plan WP6
2026-10-07 17:17:56 +00:00
weichen e5034e177d Merge pull request 'Wire the memory port: store-backed adapter, loop injection, migration parity (N3 WP5)' (#95) from feature/n3-memory-wp5 into main
ci / vet-and-test (push) Successful in 41s
Wire the memory port: store-backed adapter, loop injection, migration parity (N3 WP5)
2026-10-07 16:24:58 +00:00
weichen 766a96e817 Wire the memory port: store-backed adapter, loop injection, migration parity (N3 WP5)
ci / vet-and-test (pull_request) Successful in 41s
Why: memory recall lived as a product-side interface assertion with no
library port, AgentSpec.MemoryScope had no consumer, and the model had
no write-back path. The port (Agent-Native#34) now exists; the product
adapts it over memory_entries.
Evidence: go test -count=1 ./... green; TestStoreMemoryRecallScopeAndLimit
(scope filters and limit), TestStoreMemoryAppendRoundtrip (project
context inheritance), TestRunCtxStoreMemory (runCtx outlet),
TestDirectRunInjectsMemoryBlock (the [记忆] block reaches the request
with the pre-migration text shape and deterministic order).
Impact: assembly-side injection removed (the loop injects per request
— same text, no duplication); memory_write available on the kernel
path; recall defaults to all scopes (pre-migration semantics), with
AgentSpec.MemoryScope narrowing where hosts pass it.
Refs: Agent-Native#34, Agent-Squad#38, N3 plan WP5
2026-10-07 16:23:24 +00:00
weichen 80a7468523 Merge pull request 'Take the checkpoint closure: retention on rewind, HTTP tests, eligibility pin (N3 WP3)' (#94) from feature/n3-checkpoint-wp3 into main
ci / vet-and-test (push) Successful in 42s
Take the checkpoint closure: retention on rewind, HTTP tests, eligibility pin (N3 WP3)
2026-10-07 16:00:33 +00:00
weichen f862892544 Take the checkpoint closure: retention on rewind, HTTP tests, eligibility pin (N3 WP3)
ci / vet-and-test (pull_request) Successful in 42s
Why: the library now pins eligibility, guards symlinks, reports partial
restores, and bounds the log (Agent-Native#33); the product wires
retention after successful rewinds and needed the HTTP-layer and
eligibility evidence.
Evidence: go test -count=1 ./... green; TestRewindFilesHTTPHappy and
TestRewindFilesHTTPBusy409 (HTTP happy + conflict),
TestBackendCheckpointEligibilityIsWriteFileOnly (the product surface
checkpoints exactly {fs_write_file}).
Impact: checkpoints.jsonl is compacted to the latest 200 anchors after
each successful rewind.
Refs: Agent-Native#33, Agent-Squad#37, N3 plan WP3
2026-10-07 15:59:14 +00:00
weichen c392ed81ee Merge pull request 'Wire GenAI telemetry: settings surface, service name, and agent names (N3 WP4)' (#93) from feature/n3-otel-wp4 into main
ci / vet-and-test (push) Successful in 44s
Wire GenAI telemetry: settings surface, service name, and agent names (N3 WP4)
2026-10-07 15:42:21 +00:00
weichen 904742ca4b Wire GenAI telemetry: settings surface, service name, and agent names (N3 WP4)
ci / vet-and-test (pull_request) Successful in 40s
Why: spans now follow the OTel GenAI convention (Agent-Native#32); the
product needs the settings surface for the exporter, a meaningful
service.name, and per-profile agent names.
Evidence: go test -count=1 ./... green; TestTelemetryFlagsDefaults
(otlpEndpoint registered default empty = no export, serviceName
default agent-system-backend); nested pins carry the convention work.
Impact: nothing is exported unless telemetry.otlpEndpoint (or the env
var) is set; agent spans are named by profile id.
Refs: Agent-Native#32, Agent-Squad#36, N3 plan WP4
2026-10-07 15:41:01 +00:00
weichen dcd7c9a444 Merge pull request 'Give conversation runs the intent journal via JournalOnly sessions (N3 WP2)' (#92) from feature/n3-session-wp2 into main
ci / vet-and-test (push) Successful in 40s
Give conversation runs the intent journal via JournalOnly sessions (N3 WP2)
2026-10-07 15:27:47 +00:00
weichen 9f3fdb26e2 Give conversation runs the intent journal via JournalOnly sessions (N3 WP2)
ci / vet-and-test (pull_request) Successful in 42s
Why: the product conversation path had zero mid-turn durability
(llm_direct ran without Session — intents, recovery pairing, and
side-effect replay semantics never applied). Transcript-as-authority
would have frozen the assembled prefix, so the journal is wired in
JournalOnly mode: recovery runs against the transcript, the context
stays on the ctxasm pipeline, and recovered outcomes inject as a
system note.
Evidence: go test -count=1 ./... green;
TestDirectRunPairsAfterKillSeed (crash-window transcript gets the
crashInterrupted pair, run proceeds) and
TestDirectRunSeedsStoreHistoryOnce (history appears exactly once in
every request, never enters the journal); the forged-marker and
harness suites pass unchanged.
Impact: conversation runs now journal tool intents and recover them on
the next run; no transparent resume (sealed interrupted stays; the
user keeps chatting). Opening the transcript fails soft (falls back to
the old no-session behavior).
Refs: Agent-Native#31, Agent-Squad#35, N3 plan WP2
2026-10-07 15:26:27 +00:00
weichen 7a6a6075c2 Merge pull request 'Resume gracefully-interrupted subagents and take the RunState repair pins (N3 WP1)' (#91) from feature/n3-runstate-wp1 into main
ci / vet-and-test (push) Successful in 41s
Resume gracefully-interrupted subagents and take the RunState repair pins (N3 WP1)
2026-10-07 15:08:58 +00:00
weichen 1770fac51f Align the forged-marker guard test with the N6 semantics
ci / vet-and-test (pull_request) Successful in 50s
Why: Stop.Replacement no longer rewrites the in-memory assistant; the
correction reaches the model as a user note. The guard's protective
intent is unchanged (forged body never replays as assistant speech).
Evidence: go test -count=1 ./... green.
Impact: test only.
Refs: Agent-Native#30, N3 plan WP1
2026-10-07 15:06:58 +00:00
weichen 97bc5093bd Resume gracefully-interrupted subagents and take the RunState repair pins (N3 WP1)
Why: InterruptRunning wrote done.json{status:interrupted} for every
child, and ResumeInterruptedChildren skipped anything with done.json —
graceful shutdowns never resumed (only SIGKILL-style deaths did). done
markers now split by status: terminal states block resume, interrupted
does not. Nested pins carry repairOrphanTail and the N6 fix.
Evidence: go test -count=1 ./... green; TestResumeAfterGracefulInterruptMarker
(resumed once and only once), TestResumeSkipsCompletedMarker (terminal
states still block).
Impact: after any restart, interrupted children with a stored request
re-spawn exactly once regardless of how the process stopped.
Refs: Agent-Native#30, Agent-Squad#34, N3 plan WP1
2026-10-07 15:05:57 +00:00
weichen 7980a8a219 Merge pull request 'Give subagents per-run transcripts, resume, persisted notices, and the Background request field (N2 WP5)' (#90) from feature/n2-subagent-wp5 into main
ci / vet-and-test (push) Successful in 45s
Give subagents per-run transcripts, resume, persisted notices (N2 WP5)
2026-10-07 14:25:04 +00:00
weichen 358cfe017c Give subagents per-run transcripts, resume, persisted notices, and the Background request field (N2 WP5)
ci / vet-and-test (pull_request) Successful in 54s
Why: child runs had no independent transcript, no resume path (sealed
interrupted only), in-memory completion notices lost on restart, and
the subagent Background declaration had no consumer.
Evidence: go test -count=1 ./... green; TestSubagentTranscriptFilesWritten
(request/transcript/done files), TestResumeInterruptedChildrenRespawns
(re-spawn with original task + transcript-path hint, resumed.json
idempotent), TestNoticesPersistAndRedeliverAfterRestart (persist →
reload → deliver → delete), migration v17 pending_notices with up/down
on both dialects.
Impact: per-run transcript dirs under the workspace root (auxiliary,
write failures never abort spawns); interrupted children with a stored
request re-spawn as background runs at startup (pre-WP5 runs stay
sealed); notices survive restart; delegate factory honors
Request.Background with a sync fallback when the host lacks background
support.
Refs: Agent-Native#29, Agent-Squad#33, N2 plan WP5
2026-10-07 14:23:39 +00:00
weichen b44f78b09a Merge pull request 'Validate plugin and bundled skills against the open spec (N2 WP4)' (#89) from feature/n2-skills-wp4 into main
ci / vet-and-test (push) Successful in 51s
Validate plugin and bundled skills against the open spec (N2 WP4)
2026-10-07 13:58:47 +00:00
weichen df6745e820 Point the nested native and squad pins at the skills spec validation
ci / vet-and-test (pull_request) Successful in 41s
Why: internal/skills/validate.go calls ValidateSkillDocument, which
only exists on agent-native main after #28; the pin must move for CI
to build the new code.
Evidence: nested pins at 7425b32b / 9d3faefb (merged mains).
Impact: none beyond the pins.
Refs: Agent-Native#28, Agent-Squad#32
2026-10-07 13:57:52 +00:00
weichen 48c5202de2 Validate plugin and bundled skills against the open spec (N2 WP4)
Why: plugin skills were written to the skill root unvalidated and the
bundled recipes carried no name — both now go through the library's
spec validator (name required + kebab-case + equals dir, description
required, known-keys whitelist).
Evidence: go test -count=1 ./... green; TestV2InvalidSkillRejected pins
the fail-closed path with plugin context + field location; Seed
validates each SKILL.md before write (recipes dogfood-compliant);
nested pins carry agent-native#28 (the validator).
Impact: invalid plugin skills now abort the plugin load with a located
error instead of landing silently; the architecture baseline gains
backend/plugins→backend/skills.
Refs: Agent-Native#28, Agent-Squad#32, N2 plan WP4
2026-10-07 13:57:17 +00:00
weichen 1291b378c0 Merge pull request 'Point the nested native and squad pins at the MCP 2026-07-28 work' (#88) from feature/n2-wp3-cascade into main
ci / vet-and-test (push) Failing after 45s
Point the nested native and squad pins at the MCP 2026-07-28 work
2026-10-07 13:43:33 +00:00
weichen 7365b4c471 Point the nested native and squad pins at the MCP 2026-07-28 work
ci / vet-and-test (pull_request) Successful in 46s
Why: native 77e7f978 (Agent-Native#27, N2 WP3) and the squad cascade
carry the MCP 2026-07-28 alignment; the nested pins follow for the Dev
combination check.
Evidence: go build + go test -count=1 ./... green on this combination.
Impact: none beyond the pins.
Refs: Agent-Native#27, Agent-Squad#31
2026-10-07 13:42:14 +00:00
weichen 5dfe70d68d Merge pull request 'Point the nested native and squad pins at the Anthropic client work' (#87) from feature/n2-wp2-cascade into main
ci / vet-and-test (push) Successful in 54s
Point the nested native and squad pins at the Anthropic client work
2026-10-07 13:26:33 +00:00
weichen 7aa3bc447d Point the nested native and squad pins at the Anthropic client work
ci / vet-and-test (pull_request) Successful in 44s
Why: native d6053baf (Agent-Native#26, N2 WP2) and squad 41ea2766
(Agent-Squad#30) carry the native Anthropic client; the nested pins
follow for the Dev combination check.
Evidence: go build + go test -count=1 ./... green on this combination.
Impact: none beyond the pins.
Refs: Agent-Native#26, Agent-Squad#30
2026-10-07 13:24:45 +00:00
weichen 8858ca9b8a Merge pull request 'Align context trimming to tool groups and take the N2 WP1 pins' (#86) from feature/n2-context-wp1 into main
ci / vet-and-test (push) Successful in 42s
Align context trimming to tool groups and take the N2 WP1 pins
2026-10-07 13:13:09 +00:00
weichen 7acbefd60d Align context trimming to tool groups and take the N2 WP1 pins
ci / vet-and-test (pull_request) Successful in 44s
Why: trimOldest dropped single messages without tool-pair awareness —
after leading users were evicted it could orphan tool results or leave
dangling calls (acceptance clause 2's product-side gap).
Evidence: go test -count=1 ./... green; TestTrimOldestKeepsToolPairs
(pairing holds across repeated trims), TestTrimOldestDropsLeadingOrphanToolRun,
TestTrimOldestStopsWhenGroupCrossesCurrentTurn, TestPrefixHashStableForIdenticalInput;
native pins carry the spill and cache-contract work (Agent-Native#25).
Impact: trimming now evicts whole tool groups; the window may be
slightly exceeded when a group would cross the current turn — pairing
is never broken. Report.Truncations still records the count.
Refs: Agent-Native#25, Agent-Squad#29, N2 plan WP1
2026-10-07 13:11:47 +00:00
weichen 68addfda05 Merge pull request 'Point the nested squad and native pins at the CWD-clean fix' (#85) from feature/native-cascade-follow into main
ci / vet-and-test (push) Successful in 42s
Point the nested squad and native pins at the CWD-clean fix
2026-10-07 11:52:29 +00:00
weichen fcf2c73589 Point the nested squad and native pins at the CWD-clean fix
ci / vet-and-test (pull_request) Successful in 43s
Why: native ead690be keeps workspace-less requests out of the process
CWD (the Dev CI build step requires a clean tree after make test);
squad 029edc08 carries it.
Evidence: go build + go test ./... green on this combination.
Impact: none beyond the pins.
Refs: Agent-Native#24, Agent-Squad#28, Agent-System-Dev#242
2026-10-07 11:51:11 +00:00
weichen 33e261b224 Merge pull request 'Point the nested squad and native pins at the I2 cascade' (#84) from feature/agent-stack-i2-cascade into main
ci / vet-and-test (push) Successful in 46s
Point the nested squad and native pins at the I2 cascade
2026-10-07 11:34:20 +00:00
weichen fb90e6239a Point the nested squad and native pins at the I2 cascade
ci / vet-and-test (pull_request) Successful in 44s
Why: squad main (551e11ec) and native main (3a1ad5a) now carry the
nested agent-intent 343b1e6 pins the Dev combination check requires;
the backend nested copies follow.
Evidence: go build ./... green on this combination; trees identical
to the merged library mains.
Impact: none beyond the pins.
Refs: Agent-Squad#27, Agent-Native#23, I2 increment 3 cascade
2026-10-07 11:33:14 +00:00
weichen 9d2937115c Merge pull request 'Wire the intent slow lane behind settings flags (I2 WP3)' (#83) from feature/agent-stack-i2 into main
ci / vet-and-test (push) Successful in 40s
Wire the intent slow lane behind settings flags (I2 WP3)
2026-10-07 11:23:57 +00:00
weichen 3f5c0f60e4 Point the nested agent-intent pin at the I2 port work
ci / vet-and-test (pull_request) Successful in 43s
Why: the slow-lane wiring references SemanticScorer/CommandGenerator,
which only exist on agent-intent main after #9/#10; the pin must move
for CI to build the new packages.
Evidence: deps/agent-intent now at 343b1e6 (merged main).
Impact: none by itself.
Refs: Agent-System-Backend#83, I2 increment 3
2026-10-07 11:22:58 +00:00
weichen 45b011af9f Wire the intent slow lane behind settings flags (I2 WP3)
ci / vet-and-test (pull_request) Failing after 34s
Why: I2 increments 1-2 shipped the agent-intent ports (SemanticScorer,
CommandGenerator); the backend still had no implementation, so the slow
lane could not be enabled in the product.
Evidence: go test ./... green; lint-boundaries ok; the two new
runtime.intentSemantic / runtime.intentGenerator flags are registered
with default false and the gate defaults to the I1 recognizer when both
are off (TestIntentSlowLaneFlagsDefaultOff, TestGate*).
Impact: no behavior change by default; enabling runtime.intentSemantic
routes ambiguous rule matches through embedding cosine scoring, and
runtime.intentGenerator lets the lightweight model propose commands on
fallback with library-side fail-closed validation. Nested agent-intent
pin moves to 343b1e6 (I2 increments 1-2).
Refs: I2 increment 3
2026-10-07 11:20:41 +00:00
weichen 3261135126 Merge pull request 'Point the nested agent-squad pin at the SQLite convergence' (#82) from feature/agent-stack-q1 into main
ci / vet-and-test (push) Successful in 40s
2026-10-07 10:33:02 +00:00
weichen 3de400b740 Point the nested agent-squad pin at the SQLite convergence
ci / vet-and-test (pull_request) Successful in 40s
Why: Agent-Squad PR #26 landed the SQLite Store, single-DB mounting
and the Executor-port fix; backend must carry the same head or Dev
check-pins fails.

Evidence: backend suite 30 packages ok at this combination.

Impact: Nested squad pin only.
2026-10-07 10:32:16 +00:00
weichen 5883d10889 Merge pull request 'Point the nested agent-squad pin at the Store and termination work' (#81) from feature/agent-stack-q1 into main
ci / vet-and-test (push) Successful in 41s
2026-10-07 10:03:14 +00:00
weichen 58d9a17b8a Point the nested agent-squad pin at the Store and termination work
ci / vet-and-test (pull_request) Successful in 42s
Why: Agent-Squad PR #25 landed the Store port, tiered checkpoints
and the termination registry; backend must carry the same head or
Dev check-pins fails.

Evidence: backend suite 30 packages ok at this combination.

Impact: Nested squad pin only.
2026-10-07 10:02:28 +00:00
weichen 7ea21c3ca2 Merge pull request 'Point the nested agent-squad pin at the attempt-id recovery' (#80) from feature/agent-stack-q1 into main
ci / vet-and-test (push) Successful in 42s
2026-10-07 09:20:29 +00:00
weichen 15fb79e30e Point the nested agent-squad pin at the attempt-id recovery
ci / vet-and-test (pull_request) Successful in 41s
Why: Agent-Squad PR #24 landed attempt ids and product-checked
interrupted recovery; backend must carry the same head or Dev
check-pins fails.

Evidence: backend suite 30 packages ok at this combination.

Impact: Nested squad pin only.
2026-10-07 09:19:43 +00:00
weichen a59980898f Merge pull request 'Point the nested agent-squad pin at the superstep engine' (#79) from feature/agent-stack-q1 into main
ci / vet-and-test (push) Successful in 40s
2026-10-07 09:09:54 +00:00
weichen 617a4566be Point the nested agent-squad pin at the superstep engine
ci / vet-and-test (pull_request) Successful in 41s
Why: Agent-Squad PR #23 landed the superstep engine; backend must
carry the same squad head or Dev check-pins fails.

Evidence: backend suite 30 packages ok at this combination.

Impact: Nested squad pin only.
2026-10-07 09:09:08 +00:00
weichen 6f27567acd Merge pull request 'Point the nested agent-squad pin at the native-archive cascade head' (#78) from feature/agent-stack-i1 into main
ci / vet-and-test (push) Successful in 41s
2026-10-07 08:34:58 +00:00
weichen 64e80c9bb6 Merge origin/main into the I1 backend branch
ci / vet-and-test (pull_request) Successful in 42s
2026-10-07 08:31:44 +00:00
weichen 40ebca7f07 Point the nested agent-squad pin at the native-archive cascade
ci / vet-and-test (pull_request) Successful in 41s
Why: Squad PR #22 landed the nested native bump; backend must
carry the same squad head or Dev check-pins fails.

Evidence: squad main head is the #22 merge.

Impact: Nested squad pin only.
2026-10-07 08:28:45 +00:00
weichen 12f50ef2dc Merge pull request 'Point the nested pins at the eval-v2 combination' (#77) from feature/agent-stack-i1 into main
ci / vet-and-test (push) Successful in 42s
2026-10-07 08:26:13 +00:00
weichen 0a26fb598b Point the nested pins at the eval-v2 combination
ci / vet-and-test (pull_request) Successful in 41s
Why: Agent-Intent PR #8 and its cascades (Squad #21, Native #22)
landed; backend's nested pins must carry those heads or Dev
check-pins fails.

Evidence: backend suite 30 packages ok at this combination.

Impact: Three nested gitlinks.
2026-10-07 08:25:27 +00:00
weichen 07250a221a Merge pull request 'Project the command types through the intent gate surface' (#76) from feature/agent-stack-i1 into main
ci / vet-and-test (push) Successful in 40s
2026-10-07 07:59:08 +00:00
weichen 517ae236b3 Project the command types through the intent gate surface
ci / vet-and-test (pull_request) Successful in 42s
Why: The boundary linter allows agent-intent imports only in the
adapter layer; route and runmgr consume aliases from intentgate
instead of importing the ecosystem library directly.

Evidence: lint-boundaries passes below; backend suite green.

Impact: intentgate re-exports Command/ActionRef and the kind
constants; route drops the agent-intent import.
2026-10-07 07:58:05 +00:00
weichen a70b5a2f6a Merge pull request 'Point the nested pins at the I1 router combination' (#75) from feature/agent-stack-i1 into main
ci / vet-and-test (push) Successful in 42s
2026-10-07 07:51:22 +00:00
weichen 6c7563bd15 Point the nested pins at the I1 router combination
ci / vet-and-test (pull_request) Successful in 41s
Why: Squad PR #20 and Agent-Native PR #21 completed the intent
cascade; backend's nested pins must carry those heads or Dev
check-pins fails.

Evidence: squad and native heads are direct merges on their mains;
backend suite green before the pin bump.

Impact: Two nested gitlinks.
2026-10-07 07:50:04 +00:00
weichen 4da3cfc787 Merge pull request 'Unify intent routing behind the route package and enable NFKC (I1 WP3/WP4)' (#74) from feature/agent-stack-i1 into main
ci / vet-and-test (push) Successful in 42s
2026-10-07 07:46:07 +00:00
weichen ff43c7c9aa Unify intent routing behind the route package and enable NFKC
ci / vet-and-test (pull_request) Successful in 41s
Why: I1 WP3 - chat routing and the HITL free-text pivot now share one
router consuming the library's command list, and both intent hosts
fold compatibility-equivalent input.

Evidence: go vet clean; go test ./... 30 packages ok, including the
existing workflow-route and server e2e suites unchanged (behavior
parity) and the new route package tests.

Impact: new internal/route package (unique router with explicit
priority rules); intentgate carries Commands and folds NFKC;
server.go and runmgr.go route through it; nested pins at intent
2554b50 and squad 6b7a991. Blueprint actions resolve to a squad
decision at the router level; per-run squad entry stays a recorded
follow-up (no silent misrouting).

Refs: Documents proposal 2026-10-07-agent-stack-n1q1i1-plans.md I1 WP3/WP4
2026-10-07 07:44:31 +00:00
weichen 303983bd3e Merge pull request 'Point the nested agent-squad pin at the WP5 cascade head' (#73) from feature/agent-stack-n1 into main
ci / vet-and-test (push) Successful in 40s
2026-10-07 06:40:18 +00:00
weichen 6526a0b0e3 Point the nested agent-squad pin at the WP5 cascade head
ci / vet-and-test (pull_request) Successful in 41s
Why: Agent-Squad PR #18 landed the nested agent-native WP5 bump;
backend must carry the same squad head or Dev check-pins fails.

Evidence: squad main head is the #18 merge; backend suite runs next.

Impact: Nested squad pin only.
2026-10-07 06:38:36 +00:00
weichen 395a9907d7 Merge pull request 'Consume the sunk gate mechanics and pin the WP5 library (N1 WP5/WP6)' (#72) from feature/agent-stack-n1 into main
ci / vet-and-test (push) Successful in 40s
2026-10-07 06:36:10 +00:00
weichen 4e3c0b60c1 Merge origin/main into the N1 backend branch
ci / vet-and-test (pull_request) Successful in 41s
2026-10-07 06:35:38 +00:00
weichen 5f9e77a500 Consume the sunk gate mechanics and pin the WP5 library
ci / vet-and-test (pull_request) Successful in 40s
Why: N1 WP5 host side - directRun.stop and the checklist reminders
now call the library gates package behind product ports, and the
nested native pin moves to the merged WP5 head.

Evidence: go vet clean; go test ./... 29 packages ok, including the
characterization suite unmodified and the new wp5_ab_test.go
(verify-cap continuation counts and gate-reason text parity).

Impact: llm.go drops the local regex family and deliverableGate;
verify.go drops checkDeliverables/parseVerdict/stripFence and the
local caps; workflow.go switches to gates.Verdict; reminders run
through gates.ProgressReminder. Behavior is pinned identical by the
existing suite plus the A/B tests. Squad reminders stay out of this
increment (adapter lacks per-member tool-turn signals - recorded as
WP6b).

Refs: Documents proposal 2026-10-07-agent-stack-n1q1i1-plans.md WP5/WP6
2026-10-07 06:31:58 +00:00
weichen f0b0db9aaa Merge pull request 'Validate plugin agent declarations and wire the squad ask port (N1 WP1/WP4)' (#71) from feature/agent-stack-n1 into main
ci / vet-and-test (push) Successful in 39s
2026-10-07 05:15:36 +00:00
weichen dc6d9985d8 Validate plugin agent declarations and wire the squad ask port
ci / vet-and-test (pull_request) Successful in 41s
Why: N1 increment three host side - plugin agent assets must reject
hooks/mcp/permissionMode declarations (WP1 acceptance), and squad
members need the kernel ask port wired to the HITL card (WP4).

Evidence: go vet clean; go test ./... 29 packages ok, including new
agent-asset tests (banned keys located to the field, unknown keys,
missing id) and the V2 asset suite. Nested pins at native c5b9dca
(PR #18) and squad fc9620e (PR #17).

Impact: plugins agents/ assets now parse via parseAgentAsset
(strict JSON with banned-key rejection); PermissionAskOf exported
and takes any host with the optional method; app.go wires
cfg.PermissionAsk for the squad path. Legacy behavior unchanged when
nothing registers the port.

Refs: Documents proposal 2026-10-07-agent-stack-n1q1i1-plans.md WP1/WP4
2026-10-07 05:14:06 +00:00
weichen d34c2e8568 Merge pull request 'Point the nested agent-squad pin at the cascaded N1 combination' (#70) from feature/agent-stack-n1-pins into main
ci / vet-and-test (push) Successful in 39s
2026-10-07 04:44:36 +00:00
weichen 9239353239 Point the nested agent-squad pin at the cascaded N1 combination
ci / vet-and-test (pull_request) Successful in 39s
Why: Agent-Squad PR #16 landed the nested agent-native bump; backend
must carry the same squad head or Dev check-pins fails.

Evidence: squad main head is the #16 merge; backend test suite runs
next.

Impact: Nested squad pin only.
2026-10-07 04:43:20 +00:00
weichen fe0c5a4f1b Merge pull request 'Wire the permission ask port to HITL and adopt the Policy object (N1 WP2/WP3)' (#69) from feature/agent-stack-n1 into main
ci / vet-and-test (push) Successful in 38s
2026-10-07 04:38:53 +00:00
weichen 4075cc6501 Point backend at the merged N1 ask-loop and Policy increment
ci / vet-and-test (pull_request) Successful in 38s
Why: Library PR #17 landed on agent-native main as c2befaf; backend
compiles with the merged combination, not the pre-merge branch head.

Evidence: go vet and full go test run next; native main ancestry of
9758a8d verified before the pin bump.

Impact: Nested agent-native pin only.
2026-10-07 04:37:43 +00:00
weichen 16dbe70014 Merge origin/main into the N1 backend branch 2026-10-07 04:37:33 +00:00
weichen 17f7e2b168 Wire the permission ask port to HITL and adopt the Policy object
ci / vet-and-test (pull_request) Successful in 40s
Why: N1 WP2/WP3 host side - the kernel ask loop must reach the HITL
confirm card, and tool gating should run through the typed Policy with
the session-allow store port instead of a hand-built Request.

Evidence: go vet clean; go test ./... all packages ok, including new
TestConvAllowStoreKeysByName, TestPermissionAskRoutesToHITL, and
permissionAskOf tests. Nested agent-native pin at 9758a8d (PR #17).

Impact: runmgr.ExecuteTool decides via agentpolicy.Policy with
convAllowStore (keying unchanged: conversation x tool name);
runCtx.PermissionAsk routes kernel asks to the shared toolConfirmCard
HITL; direct and subagent loops pass PermissionAsk into the kernel
config (nil for hosts without the method, preserving legacy ask=reject).

Refs: Documents proposal 2026-10-07-agent-stack-n1q1i1-plans.md WP2/WP3; Agent-Native PR #17
2026-10-07 04:04:10 +00:00
weichen 838f7a5e14 Merge pull request 'Skip wide sparse columns without dropping later xlsx sheets' (#68) from fix/ext-5-wide-cols into main
ci / vet-and-test (push) Successful in 41s
2026-10-06 15:39:42 +00:00
weichen e7fb276ee9 Skip wide sparse columns without dropping later xlsx sheets
ci / vet-and-test (pull_request) Successful in 41s
Why: review of EXT-5 found a column-cap cell (common styled XFD cells) wrongly reported the 200k-cell cap and aborted every remaining worksheet; malformed sheet XML was also dropped silently, contradicting the explicit-failure contract; the legacy-office parser id was missing from the registry.

Evidence: local GOWORK=off go test ./... 29 packages pass with new regressions (wide column keeps later sheets, bad sheet XML warns, registry maps doc/xls/ppt); go vet clean; recorded as SELF on the Dev task card.

Impact: xlsxSheetBlocks returns wide/badXML signals; wide cells skip with a per-sheet warning, cell-cap truncation unchanged; builtin:legacy-office added to ParsersDescription.

Refs: review of backend 727f370; documents proposal 2026-10-06-ext-5-host-parse-document sect.2.
2026-10-06 15:30:50 +00:00
weichen 736f0a0839 Merge pull request 'Expose builtin document parsing through the host handle' (#67) from feature/ext-5-host-parse into main
ci / vet-and-test (push) Successful in 43s
2026-10-06 15:14:21 +00:00
weichen 727f3702d0 Expose builtin document parsing through the host handle
ci / vet-and-test (pull_request) Successful in 46s
Why: downstream parsers are unreachable via extension.Host and the base lacked xlsx/pptx parsers; MS1 needs Excel and PPT ingest (EXT-5 request, documents proposal 2026-10-06-ext-5-host-parse-document).

Evidence: local GOWORK=off go test ./... all 29 packages pass incl. new ooxml_test (hand-built parts), ooxml_python_test (openpyxl/python-pptx round-trip), inproc owner-scope and limit tests, xlsx upload e2e; go vet clean; recorded as SELF on the Dev task card, PR CI to confirm.

Impact: Host gains ParseDocument (pure compute; errors only on ctx cancel or >100MiB input); builtin:xlsx (sheet order via workbook.xml rels, rich-text shared strings, formula cache, 1900/1904 dates, sparse cells) and builtin:pptx (sldIdLst order, titles, group shapes, AlternateContent Choice, tables, notes) added with 200k-cell/64MiB-part caps; legacy doc/xls/ppt get an actionable hint; ParsersDescription gains xlsx/pptx and the missing text ext; surface snapshot now records exported interface methods; docx part reads share the 64MiB cap.

Refs: documents 2026-10-06-ext-5-host-parse-document.md; EXT-1 host.go precedent.
2026-10-06 14:59:38 +00:00
weichen f9212d01b7 Merge pull request 'Merge EXT-1 inproc plugin command handler into main' (#66) from feature/ext-1-inproc-handler into main
ci / vet-and-test (push) Successful in 41s
2026-10-06 01:25:37 +00:00
285 changed files with 35087 additions and 2257 deletions
+28
View File
@@ -0,0 +1,28 @@
package main
import (
"os"
"os/exec"
"strings"
"testing"
)
func TestUnknownSubcommandExits(t *testing.T) {
if os.Getenv("DISPATCH_CHILD") == "1" {
os.Args = []string{"agent-system", "not-a-command"}
main()
return
}
cmd := exec.Command(os.Args[0], "-test.run=TestUnknownSubcommandExits")
cmd.Env = append(os.Environ(), "DISPATCH_CHILD=1")
out, err := cmd.CombinedOutput()
if err == nil {
t.Fatalf("未知子命令应非零退出:%s", out)
}
if !strings.Contains(string(out), "未知子命令") {
t.Fatalf("输出:%s", out)
}
if strings.Contains(string(out), "agent-system starting") || strings.Contains(string(out), "serving") {
t.Fatalf("未知子命令落入了服务路径:%s", out)
}
}
+192 -16
View File
@@ -9,17 +9,28 @@ import (
"flag"
"fmt"
"os"
"strings"
"time"
"agent-system-backend/internal/app"
"agent-system-backend/internal/config"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/identity"
)
var version = "0.1.0-dev"
func main() {
if len(os.Args) > 1 {
if len(os.Args) > 1 && !strings.HasPrefix(os.Args[1], "-") {
switch os.Args[1] {
case "version", "--version":
if len(os.Args) > 2 && os.Args[2] == "--json" {
if err := identity.WriteDocument(os.Stdout, "agent-system", version); err != nil {
fmt.Fprintln(os.Stderr, "version:", err)
os.Exit(1)
}
return
}
fmt.Println("agent-system", version)
return
case "serve":
@@ -30,6 +41,12 @@ func main() {
os.Exit(runConfig(os.Args[2:]))
case "user":
os.Exit(runUser(os.Args[2:]))
case "identity":
os.Exit(runIdentity(os.Args[2:]))
default:
fmt.Fprintf(os.Stderr, "agent-system: 未知子命令 %s\n", os.Args[1])
fmt.Fprintln(os.Stderr, "用法:agent-system <serve|version|identity|user|apikey|config> [参数]")
os.Exit(2)
}
}
fs := flag.NewFlagSet("serve", flag.ExitOnError)
@@ -44,22 +61,49 @@ func main() {
instance := fs.String("instance", "", "本进程标识(空 = 主机名:监听地址;启动对账只收口这个实例的 run)")
insecure := fs.Bool("insecure-listen", false, "允许在非回环地址上不开启认证(每次启动都会警告)")
allowedHosts := fs.String("allowed-hosts", "", "逗号分隔的额外主机名,供反向代理的 Host 与 Origin 校验")
noExec := fs.Bool("disable-exec-tools", false, "硬关闭执行原语 run_shell/run_python/register_artifact(用户设置无法覆盖;多用户部署在执行隔离就位前建议开启)")
execBackend := fs.String("exec-backend", "", "执行后端:direct(默认,现状直启)| sandbox(bwrap 隔离档:断网、只读系统根、仅会话目录可写、进程数限额)")
execDepsDir := fs.String("exec-deps-dir", "", "sandbox 档只读依赖层目录(site-packages 等;空=不挂载)")
execWorkspace := fs.String("exec-workspace", "", "sandbox 档工作区模式:空=直写会话目录 | overlay=冻结基底+私有 upper+CAS 回收(S0)")
tier := fs.String("tier", "", "部署档:空=auto(DSN→PG,否则 SQLite)| local(SQLite,不允许 DSN)| team(PG fail-closed:缺 DSN/未认证拒绝启动,PG 故障拒绝就绪)")
pgMaxOpen := fs.Int("pg-max-open", 0, "PG 最大打开连接(0=默认 16)")
pgMaxIdle := fs.Int("pg-max-idle", 0, "PG 最大空闲连接(0=默认 8)")
pgIdleSec := fs.Int("pg-conn-max-idle-sec", 0, "PG 空闲连接回收秒数(0=默认 300)")
pgLifetimeSec := fs.Int("pg-conn-max-lifetime-sec", 0, "PG 连接最长生命周期秒数(0=不限)")
pgQueryTimeoutSec := fs.Int("pg-query-timeout-sec", 0, "PG statement_timeout 秒数(0=不限;覆盖全部业务查询)")
fs.Parse(os.Args[1:])
cliSet := visited(fs)
resolved, err := config.Load(*cfgFile, os.LookupEnv, config.Config{
Addr: *addr, DataDir: *data, PluginsDir: *pluginsDir, Mock: *mock,
StorageDSN: *dsn, ArtifactsDir: *artifacts, AuthMode: *auth, Instance: *instance,
InsecureListen: *insecure, AllowedHostsRaw: *allowedHosts,
InsecureListen: *insecure, AllowedHostsRaw: *allowedHosts, DisableExecTools: *noExec,
ExecBackend: *execBackend, ExecDepsDir: *execDepsDir, ExecWorkspace: *execWorkspace,
Tier: *tier, PGMaxOpen: *pgMaxOpen, PGMaxIdle: *pgMaxIdle,
PGConnMaxIdleSec: *pgIdleSec, PGConnMaxLifetimeSec: *pgLifetimeSec, PGQueryTimeoutSec: *pgQueryTimeoutSec,
}, cliSet)
if err != nil {
fmt.Fprintln(os.Stderr, "agent-system:", err)
os.Exit(1)
}
if msgs := resolved.TierProblems(); len(msgs) > 0 {
for _, msg := range msgs {
fmt.Fprintln(os.Stderr, "agent-system: tier guard:", msg)
}
os.Exit(1)
}
if err := app.Run(app.Options{
Addr: resolved.Addr, DataDir: resolved.DataDir, PluginsDir: resolved.PluginsDir, ForceMock: resolved.Mock,
StorageDSN: resolved.StorageDSN, ArtifactsDir: resolved.ArtifactsDir, AuthMode: resolved.AuthMode,
InstanceID: resolved.Instance, InsecureListen: resolved.InsecureListen, AllowedHosts: resolved.AllowedHosts(),
DisableExecTools: resolved.DisableExecTools, ExecBackend: resolved.ExecBackend, ExecDepsDir: resolved.ExecDepsDir,
ExecWorkspace: resolved.ExecWorkspace,
Tier: resolved.Tier,
PGPool: store.PoolOptions{
MaxOpenConn: resolved.PGMaxOpen, MaxIdleConn: resolved.PGMaxIdle,
ConnMaxIdleTimeSec: resolved.PGConnMaxIdleSec, ConnMaxLifetimeSec: resolved.PGConnMaxLifetimeSec,
QueryTimeoutSec: resolved.PGQueryTimeoutSec,
},
}); err != nil {
fmt.Fprintln(os.Stderr, "agent-system:", err)
os.Exit(1)
@@ -76,6 +120,18 @@ func visited(fs *flag.FlagSet) map[string]bool {
name = "insecureListen"
case "allowed-hosts":
name = "allowedHosts"
case "disable-exec-tools":
name = "disableExecTools"
case "pg-max-open":
name = "pgMaxOpen"
case "pg-max-idle":
name = "pgMaxIdle"
case "pg-conn-max-idle-sec":
name = "pgConnMaxIdleSec"
case "pg-conn-max-lifetime-sec":
name = "pgConnMaxLifetimeSec"
case "pg-query-timeout-sec":
name = "pgQueryTimeoutSec"
}
set[name] = true
})
@@ -100,11 +156,24 @@ func runConfig(args []string) int {
instance := fs.String("instance", "", "")
insecure := fs.Bool("insecure-listen", false, "")
allowedHosts := fs.String("allowed-hosts", "", "")
noExec := fs.Bool("disable-exec-tools", false, "")
execBackend := fs.String("exec-backend", "", "")
execDepsDir := fs.String("exec-deps-dir", "", "")
execWorkspace := fs.String("exec-workspace", "", "")
tier := fs.String("tier", "", "")
pgMaxOpen := fs.Int("pg-max-open", 0, "")
pgMaxIdle := fs.Int("pg-max-idle", 0, "")
pgIdleSec := fs.Int("pg-conn-max-idle-sec", 0, "")
pgLifetimeSec := fs.Int("pg-conn-max-lifetime-sec", 0, "")
pgQueryTimeoutSec := fs.Int("pg-query-timeout-sec", 0, "")
fs.Parse(args[1:])
resolved, err := config.Load(*cfgFile, os.LookupEnv, config.Config{
Addr: *addr, DataDir: *data, PluginsDir: *pluginsDir, Mock: *mock,
StorageDSN: *dsn, ArtifactsDir: *artifacts, AuthMode: *auth, Instance: *instance,
InsecureListen: *insecure, AllowedHostsRaw: *allowedHosts,
InsecureListen: *insecure, AllowedHostsRaw: *allowedHosts, DisableExecTools: *noExec,
ExecBackend: *execBackend, ExecDepsDir: *execDepsDir, ExecWorkspace: *execWorkspace,
Tier: *tier, PGMaxOpen: *pgMaxOpen, PGMaxIdle: *pgMaxIdle,
PGConnMaxIdleSec: *pgIdleSec, PGConnMaxLifetimeSec: *pgLifetimeSec, PGQueryTimeoutSec: *pgQueryTimeoutSec,
}, visited(fs))
if err != nil {
fmt.Fprintln(os.Stderr, "agent-system:", err)
@@ -169,11 +238,33 @@ func runAPIKey(args []string) int {
}
}
// runUser:用户账号管理子命令(管理员发放制,R2)——create / reset-password / list。
// 密码未显式给定时生成随机值,明文仅此一次输出;创建/重置后首次登录强制改密。
// deliverCredential 凭据单次交付(KTD14):--out 时写 0600 JSON 文件,stdout 只报
// 属主/用途/文件路径不含明文;缺省沿用 stdout 一次输出。文件含 expiresAt 便于交接。
func deliverCredential(outPath, kind, owner, plaintext string, expiresAt any) error {
if outPath == "" {
fmt.Println(plaintext)
return nil
}
payload := map[string]any{"kind": kind, "owner": owner, "secret": plaintext}
if expiresAt != nil {
payload["expiresAt"] = expiresAt
}
buf, err := json.MarshalIndent(payload, "", " ")
if err != nil {
return err
}
if err := os.WriteFile(outPath, buf, 0o600); err != nil {
return err
}
fmt.Fprintf(os.Stderr, "凭据已写入 %s(0600,仅此一次生成;勿入库勿入日志)\n", outPath)
return nil
}
// runUser:create / reset-password / list / issue-login-key / revoke-login-key / bootstrap-admin。
// 默认 create 无初始密码并签发激活 Key;--generate-password 兼容旧随机密码路径。
func runUser(args []string) int {
if len(args) == 0 {
fmt.Fprintln(os.Stderr, "用法:agent-system user <create|reset-password|list> [--owner x --name y --admin --password p]")
fmt.Fprintln(os.Stderr, "用法:agent-system user <create|reset-password|list|issue-login-key|revoke-login-key|bootstrap-admin> [flags]")
return 2
}
sub := args[0]
@@ -182,8 +273,16 @@ func runUser(args []string) int {
dsn := fs.String("dsn", "", "PostgreSQL 档 DSN(空 = 嵌入式档)")
owner := fs.String("owner", "", "用户名(owner)")
name := fs.String("name", "", "显示名")
password := fs.String("password", "", "密码(空 = 随机生成并打印一次)")
admin := fs.Bool("admin", false, "管理员(可管理用户与密钥)")
password := fs.String("password", "", "密码(create 空=激活 Key;加 --generate-password 则随机)")
admin := fs.Bool("admin", false, "管理员")
role := fs.String("role", "user", "角色 user|admin")
openExisting := fs.Bool("open-existing", false, "只打开既有存储")
generatePassword := fs.Bool("generate-password", false, "create 时生成随机初始密码")
purpose := fs.String("purpose", "recover", "issue-login-key:activate|recover")
ttlMin := fs.Int("ttl-minutes", 30, "登录 Key 有效分钟(5–120)")
revokeAll := fs.Bool("revoke-all-credentials", false, "签发时撤销全部长期 API Key 与会话")
keyID := fs.String("id", "", "revoke-login-key 凭据编号")
out := fs.String("out", "", "凭据交付文件(0600 JSON,代替 stdout 明文;非交互推荐)")
fs.Parse(args[1:])
switch sub {
case "create":
@@ -191,14 +290,30 @@ func runUser(args []string) int {
fmt.Fprintln(os.Stderr, "user create: --owner 必填")
return 2
}
pw, err := app.CreateUserAccount(*dsn, *data, *owner, *name, *password, *admin)
isAdmin := *admin
switch *role {
case "user":
case "admin":
isAdmin = true
default:
fmt.Fprintf(os.Stderr, "user create: 未知角色 %q\n", *role)
return 2
}
pw, err := app.CreateUserAccount(*dsn, *data, *owner, *name, *password, isAdmin, *openExisting, *generatePassword)
if err != nil {
fmt.Fprintln(os.Stderr, "user create:", err)
return 1
}
fmt.Println(pw)
if *password == "" {
fmt.Fprintln(os.Stderr, "初始密码仅此一次输出;该用户首次登录须修改密码。")
if err := deliverCredential(*out, "login-key", *owner, pw, nil); err != nil {
fmt.Fprintln(os.Stderr, "user create:", err)
return 1
}
if *out == "" {
if *password == "" && !*generatePassword {
fmt.Fprintln(os.Stderr, "一次性激活 Key 仅此一次输出;兑换后设置密码。")
} else if *password == "" {
fmt.Fprintln(os.Stderr, "初始密码仅此一次输出;该用户首次登录须修改密码。")
}
}
return 0
case "reset-password":
@@ -206,18 +321,21 @@ func runUser(args []string) int {
fmt.Fprintln(os.Stderr, "user reset-password: --owner 必填")
return 2
}
pw, err := app.ResetUserPassword(*dsn, *data, *owner, *password)
pw, err := app.ResetUserPassword(*dsn, *data, *owner, *password, *openExisting)
if err != nil {
fmt.Fprintln(os.Stderr, "user reset-password:", err)
return 1
}
fmt.Println(pw)
if *password == "" {
if err := deliverCredential(*out, "password", *owner, pw, nil); err != nil {
fmt.Fprintln(os.Stderr, "user reset-password:", err)
return 1
}
if *out == "" && *password == "" {
fmt.Fprintln(os.Stderr, "新密码仅此一次输出;该用户下次登录须修改密码(其余会话已吊销)。")
}
return 0
case "list":
users, err := app.ListUserAccounts(*dsn, *data)
users, err := app.ListUserAccounts(*dsn, *data, *openExisting)
if err != nil {
fmt.Fprintln(os.Stderr, "user list:", err)
return 1
@@ -225,8 +343,66 @@ func runUser(args []string) int {
out, _ := json.MarshalIndent(users, "", " ")
fmt.Println(string(out))
return 0
case "issue-login-key":
if *owner == "" {
fmt.Fprintln(os.Stderr, "user issue-login-key: --owner 必填")
return 2
}
if *purpose != store.LoginKeyPurposeActivate && *purpose != store.LoginKeyPurposeRecover {
fmt.Fprintf(os.Stderr, "user issue-login-key: 无效 purpose %q\n", *purpose)
return 2
}
plain, err := app.IssueUserLoginKey(*dsn, *data, *owner, *purpose, *openExisting, *revokeAll, time.Duration(*ttlMin)*time.Minute)
if err != nil {
fmt.Fprintln(os.Stderr, "user issue-login-key:", err)
return 1
}
if err := deliverCredential(*out, "login-key", *owner, plain, time.Now().Add(time.Duration(*ttlMin)*time.Minute).UTC()); err != nil {
fmt.Fprintln(os.Stderr, "user issue-login-key:", err)
return 1
}
if *out == "" {
fmt.Fprintln(os.Stderr, "一次性登录 Key 仅此一次输出。")
}
return 0
case "revoke-login-key":
if *keyID == "" {
fmt.Fprintln(os.Stderr, "user revoke-login-key: --id 必填")
return 2
}
if err := app.RevokeUserLoginKey(*dsn, *data, *keyID, *openExisting); err != nil {
fmt.Fprintln(os.Stderr, "user revoke-login-key:", err)
return 1
}
fmt.Println("revoked")
return 0
case "bootstrap-admin":
if *owner == "" {
fmt.Fprintln(os.Stderr, "user bootstrap-admin: --owner 必填")
return 2
}
plain, err := app.BootstrapAdmin(*dsn, *data, *owner, *openExisting)
if err != nil {
fmt.Fprintln(os.Stderr, "user bootstrap-admin:", err)
return 1
}
if err := deliverCredential(*out, "bootstrap-admin", *owner, plain, time.Now().Add(30*time.Minute).UTC()); err != nil {
fmt.Fprintln(os.Stderr, "user bootstrap-admin:", err)
return 1
}
if *out == "" {
fmt.Fprintln(os.Stderr, "首位管理员激活 Key 仅此一次输出。")
}
return 0
default:
fmt.Fprintln(os.Stderr, "user: 未知子命令", sub)
return 2
}
}
// runIdentity:U1 I3 服务实例库内身份(manifest.instance_id ↔ agent_system_identity)。
// 与 serve --instance(runs 进程标识)无关。实现在 pkg/identity(公共化:下游
// 组合仓共用同一 CLI,未知子命令 usage 退出、不落服务路径——2026-10-12 裁定)。
func runIdentity(args []string) int {
return identity.Run(args)
}
+188
View File
@@ -0,0 +1,188 @@
package main
import (
"database/sql"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"agent-system-backend/internal/store"
)
// U4 集成(环境门控 AGENT_PG_TEST_DSN:CI test-pg job / 本机隔离实例):
// 夹具 → 受管迁移 → 校验和验证 → 幂等重跑 → 断点续传 → 非空目标拒绝 →
// 未知 owner 阻塞。每用例独立 PG schema(与 store 测试同隔离约定)。
func pgSchemaDSN(t *testing.T) (string, func()) {
t.Helper()
dsn := os.Getenv("AGENT_PG_TEST_DSN")
if dsn == "" {
t.Skip("AGENT_PG_TEST_DSN 未设置:真库集成由 CI 的必跑 job 覆盖")
}
schema := "mig_" + strings.ToLower(store.NewID("t")) + "_" + fmt.Sprint(os.Getpid())
admin, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatal(err)
}
if _, err := admin.Exec(`CREATE EXTENSION IF NOT EXISTS pg_trgm`); err != nil {
admin.Close()
t.Fatalf("preinstall pg_trgm: %v", err)
}
if _, err := admin.Exec("CREATE SCHEMA " + schema); err != nil {
admin.Close()
t.Fatalf("create schema: %v", err)
}
cleanup := func() {
admin.Exec("DROP SCHEMA IF EXISTS " + schema + " CASCADE")
admin.Close()
}
return store.AppendPGRuntimeParam(dsn, "search_path", schema+",public"), cleanup
}
func TestMigrateEndToEndAndIdempotent(t *testing.T) {
target, cleanup := pgSchemaDSN(t)
defer cleanup()
fixture := buildFixture(t)
if err := cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-e2e"}); err != nil {
t.Fatalf("首次迁移:%v", err)
}
// 幂等重跑(同身份):表级 done 对账,零重复行。
if err := cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-e2e"}); err != nil {
t.Fatalf("幂等重跑:%v", err)
}
// 目标侧关键断言:布尔映射、NULL 保留、Unicode、BLOB、SERIAL 序列推进、
// 复合主键事件序。
st, err := store.OpenPostgres(target, filepath.Join(t.TempDir(), "artifacts"))
if err != nil {
t.Fatal(err)
}
defer st.Close()
db := st.DB()
var pinned, archived bool
if err := db.QueryRow(`SELECT pinned, archived FROM conversations WHERE id='c1'`).Scan(&pinned, &archived); err != nil || !pinned || archived {
t.Fatalf("布尔映射:pinned=%v archived=%v err=%v", pinned, archived, err)
}
var title, note string
if err := db.QueryRow(`SELECT title, session_note FROM conversations WHERE id='c1'`).Scan(&title, &note); err != nil || !strings.Contains(note, "注") {
t.Fatalf("Unicode:title=%q note=%q err=%v", title, note, err)
}
var vec []byte
if err := db.QueryRow(`SELECT vec FROM knowledge_vectors WHERE doc_id='k1'`).Scan(&vec); err != nil || len(vec) != 12 {
t.Fatalf("BLOB 向量:len=%d err=%v", len(vec), err)
}
var ended sql.NullString
if err := db.QueryRow(`SELECT ended_at FROM runs WHERE id='r1'`).Scan(&ended); err != nil || ended.Valid {
t.Fatalf("NULL 保留:ended=%v err=%v", ended, err)
}
var seqMax int
if err := db.QueryRow(`SELECT MAX(seq) FROM events WHERE conversation_id='c1'`).Scan(&seqMax); err != nil || seqMax != 2 {
t.Fatalf("事件复合主键序:max=%d err=%v", seqMax, err)
}
// 复制的显式 id 之后,应用侧插入不撞序列(command_audit.id=7 已复制)。
if _, err := db.Exec(`INSERT INTO command_audit (time, command_id, source, entrance, state, operation_id, conversation_id, payload, owner) VALUES ('2026-10-11T03:00:00Z','cmd-2','http','chat','done','op-2','c1','{}','alice')`); err != nil {
t.Fatalf("序列推进失败(复制后插入撞主键):%v", err)
}
// 报告不含秘密值。
var report string
if err := db.QueryRow(`SELECT report FROM migration_control WHERE migration_id='mig-e2e'`).Scan(&report); err != nil || strings.Contains(report, "sk-secret-value") {
t.Fatalf("报告泄密:err=%v", err)
}
}
func TestMigrateResumeAfterInterrupt(t *testing.T) {
target, cleanup := pgSchemaDSN(t)
defer cleanup()
fixture := buildFixture(t)
if err := cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-resume"}); err != nil {
t.Fatal(err)
}
// 模拟断点:messages 已 done 后被部分回退(行删一半、状态回 pending),
// 重跑只重做该表,终态校验仍全绿。
db, err := sql.Open("pgx", target)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err := db.Exec(`DELETE FROM messages WHERE id='m3'`); err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`UPDATE migration_table_state SET state='pending' WHERE migration_id='mig-resume' AND table_name='messages'`); err != nil {
t.Fatal(err)
}
if err := cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-resume"}); err != nil {
t.Fatalf("断点续跑:%v", err)
}
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM messages`).Scan(&n); err != nil || n != 3 {
t.Fatalf("续跑后 messages=%d:重做该表应恢复 3 行", n)
}
}
func TestMigrateRefusesForeignDataAndWrongSnapshot(t *testing.T) {
target, cleanup := pgSchemaDSN(t)
defer cleanup()
fixture := buildFixture(t)
// 非空目标(人工数据、无迁移身份):拒绝。
st, err := store.OpenPostgres(target, filepath.Join(t.TempDir(), "artifacts"))
if err != nil {
t.Fatal(err)
}
st.Close()
db, err := sql.Open("pgx", target)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err := db.Exec(`INSERT INTO projects (id, name, settings, created_at, updated_at, owner) VALUES ('foreign', '外来', '{}', '2026-10-11T00:00:00Z', '2026-10-11T00:00:00Z', 'alice')`); err != nil {
t.Fatal(err)
}
err = cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-x"})
if err == nil || !strings.Contains(err.Error(), "非空目标") {
t.Fatalf("非空目标应拒绝:%v", err)
}
if _, err := db.Exec(`DELETE FROM projects WHERE id='foreign'`); err != nil {
t.Fatal(err)
}
// 同 migration_id 换快照:拒绝(仅同一冻结快照可续传)。
if err := cmdMigrate([]string{"--source", fixture, "--target", target, "--migration-id", "mig-snap"}); err != nil {
t.Fatal(err)
}
other := buildFixture(t) // 不同文件 ⇒ 不同 snapshot_id
err = cmdMigrate([]string{"--source", other, "--target", target, "--migration-id", "mig-snap"})
if err == nil || !strings.Contains(err.Error(), "快照不同") {
t.Fatalf("换快照应拒绝:%v", err)
}
// 其他 migration_id 已存在:拒绝。
err = cmdMigrate([]string{"--source", other, "--target", target, "--migration-id", "mig-other"})
if err == nil || !strings.Contains(err.Error(), "其他迁移记录") {
t.Fatalf("他身份应拒绝:%v", err)
}
}
func TestMigrateBlocksUnknownOwner(t *testing.T) {
target, cleanup := pgSchemaDSN(t)
defer cleanup()
fixture := buildFixture(t)
// 在夹具里造一个 users 之外的 owner 行(直接写源库)。
srcPath := fixture
st, err := store.OpenSQLite(srcPath) // 普通打开(夹具可写)
if err != nil {
t.Fatal(err)
}
if _, err := st.DB().Exec(`INSERT INTO knowledge (id, project_id, source, title, ref, chunk, created_at, owner) VALUES
('k_ghost', 'p1', 'upload', '幽灵', 'g.md', '…', '2026-10-11T01:00:00Z', 'ghost-user')`); err != nil {
t.Fatal(err)
}
st.Close()
err = cmdMigrate([]string{"--source", srcPath, "--target", target, "--migration-id", "mig-ghost"})
if err == nil || !strings.Contains(err.Error(), "未知 owner") {
t.Fatalf("未知 owner 应阻塞:%v", err)
}
}
+861
View File
@@ -0,0 +1,861 @@
// storage-migrate:SQLite → PostgreSQL 受管迁移入口(协作存储方案 U4/R7)。
//
// 纪律(方案 §5 U4 / §6):
// - 只处理 Backend 库;不执行 SQLite dump 作为 PG SQL(按行复制 + 类型映射);
// - 迁移身份(migration_id + source_snapshot_id + source_schema_version)落
// 目标库 migration_control:仅同一冻结快照可续传;目标已含其他数据即拒绝;
// - 报告只含计数、校验和与归属检查结论——不含 DSN、密钥与行正文(KTD14)。
//
// 用法:
//
// storage-migrate inventory --source agent-system.db [--artifacts dir] [--out report.json]
// storage-migrate backup --source agent-system.db --out snapshot.db
// storage-migrate migrate --source snapshot.db --target "postgres://…" \
// --migration-id <名> [--artifacts dir] [--out report.json]
package main
import (
"crypto/sha256"
"database/sql"
"encoding/base64"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
"time"
_ "modernc.org/sqlite" // SQLite 驱动(源,只读打开)
"agent-system-backend/internal/store"
)
func main() {
if len(os.Args) < 2 {
usage()
os.Exit(2)
}
var err error
switch os.Args[1] {
case "inventory":
err = cmdInventory(os.Args[2:])
case "backup":
err = cmdBackup(os.Args[2:])
case "migrate":
err = cmdMigrate(os.Args[2:])
default:
usage()
os.Exit(2)
}
if err != nil {
fmt.Fprintln(os.Stderr, "storage-migrate:", err)
os.Exit(1)
}
}
func usage() {
fmt.Fprintln(os.Stderr, "用法:storage-migrate <inventory|backup|migrate> [flags](-h 看各子命令 flags)")
}
// ---- 公共小件 ----
func openSourceRO(path string) (*sql.DB, error) {
if _, err := os.Stat(path); err != nil {
return nil, fmt.Errorf("source %s: %w", path, err)
}
// query_only:迁移源只读(冻结快照,任何写入都是缺陷)。
db, err := sql.Open("sqlite", fmt.Sprintf("file:%s?_pragma=query_only(1)", path))
if err != nil {
return nil, err
}
if err := db.Ping(); err != nil {
db.Close()
return nil, fmt.Errorf("source %s: %w", path, err)
}
return db, nil
}
func fileSHA256(path string) (string, int64, error) {
f, err := os.Open(path)
if err != nil {
return "", 0, err
}
defer f.Close()
h := sha256.New()
n, err := io.Copy(h, f)
if err != nil {
return "", 0, err
}
return hex.EncodeToString(h.Sum(nil)), n, nil
}
func nowISO() string { return time.Now().UTC().Format(time.RFC3339Nano) }
// ---- inventory:只读盘点(§6.1) ----
func cmdInventory(args []string) error {
fs := flag.NewFlagSet("inventory", flag.ExitOnError)
source := fs.String("source", "", "SQLite 库路径(活跃库或冻结快照)")
artifacts := fs.String("artifacts", "", "工件对象目录(可选;盘点文件清单与哈希)")
out := fs.String("out", "", "报告输出 JSON 路径(缺省打印到 stdout)")
fs.Parse(args)
if *source == "" {
return fmt.Errorf("--source 必填")
}
db, err := openSourceRO(*source)
if err != nil {
return err
}
defer db.Close()
rep, err := inventoryReport(db, *source, *artifacts)
if err != nil {
return err
}
return writeReport(rep, *out)
}
func inventoryReport(db *sql.DB, source, artifactsDir string) (map[string]any, error) {
version, err := sourceSchemaVersion(db)
if err != nil {
return nil, err
}
var mode string
if err := db.QueryRow(`PRAGMA journal_mode`).Scan(&mode); err != nil {
return nil, fmt.Errorf("journal_mode: %w", err)
}
var chk string
if err := db.QueryRow(`PRAGMA integrity_check`).Scan(&chk); err != nil || chk != "ok" {
return nil, fmt.Errorf("integrity_check: %v (%q)", err, chk)
}
tables, err := sqliteTableCounts(db)
if err != nil {
return nil, err
}
snap, size, err := fileSHA256(source)
if err != nil {
return nil, err
}
rep := map[string]any{
"command": "inventory", "source": source, "source_sha256": snap,
"source_bytes": size, "source_schema_version": version,
"journal_mode": mode, "integrity": chk, "table_counts": tables,
"generated_at": nowISO(),
}
if artifactsDir != "" {
manifest, err := artifactManifest(artifactsDir)
if err != nil {
return nil, err
}
rep["artifacts"] = manifest
}
return rep, nil
}
func sourceSchemaVersion(db *sql.DB) (int, error) {
var v int
err := db.QueryRow(`SELECT COALESCE(MAX(version),0) FROM schema_migrations`).Scan(&v)
if err != nil {
if strings.Contains(err.Error(), "no such table") {
return 0, fmt.Errorf("源库缺 schema_migrations(不是本工具管理的库):%w", err)
}
return 0, err
}
return v, nil
}
// sqliteTableCounts:仅盘点本工具认识的业务表(见 copyOrder)。
func sqliteTableCounts(db *sql.DB) (map[string]int64, error) {
out := map[string]int64{}
for _, t := range copyOrder {
var n int64
if err := db.QueryRow(`SELECT COUNT(*) FROM "` + t + `"`).Scan(&n); err != nil {
if strings.Contains(err.Error(), "no such table") {
continue // 旧库可能没有新表
}
return nil, fmt.Errorf("count %s: %w", t, err)
}
out[t] = n
}
return out, nil
}
func artifactManifest(dir string) (map[string]any, error) {
type entry struct {
Name, SHA string
Size int64
}
var entries []entry
hAll := sha256.New()
err := filepath.Walk(dir, func(p string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return err
}
sum, _, err := fileSHA256(p)
if err != nil {
return err
}
rel, _ := filepath.Rel(dir, p)
entries = append(entries, entry{rel, sum, info.Size()})
fmt.Fprintf(hAll, "%s %d %s\n", rel, info.Size(), sum)
return nil
})
if err != nil {
return nil, fmt.Errorf("artifacts %s: %w", dir, err)
}
sort.Slice(entries, func(i, j int) bool { return entries[i].Name < entries[j].Name })
return map[string]any{
"dir": dir, "files": entries, "file_count": len(entries),
"manifest_sha256": hex.EncodeToString(hAll.Sum(nil)),
}, nil
}
// ---- backup:一致性快照(§6.2:VACUUM INTO + 可打开校验 + 计数对账) ----
func cmdBackup(args []string) error {
fs := flag.NewFlagSet("backup", flag.ExitOnError)
source := fs.String("source", "", "活跃 SQLite 库路径")
out := fs.String("out", "", "快照输出路径(不得与源同文件;已存在则拒绝)")
fs.Parse(args)
if *source == "" || *out == "" {
return fmt.Errorf("--source 与 --out 必填")
}
if same, _ := filepath.EvalSymlinks(*source); same != "" {
if abs1, _ := filepath.Abs(same); abs1 != "" {
if abs2, _ := filepath.Abs(*out); abs2 == abs1 {
return fmt.Errorf("快照输出不得与源同文件")
}
}
}
if _, err := os.Stat(*out); err == nil {
return fmt.Errorf("快照输出已存在(不覆盖):%s", *out)
}
if err := os.MkdirAll(filepath.Dir(*out), 0o755); err != nil {
return err
}
// VACUUM INTO 在线安全(含 WAL 内容);query_only 会禁写,需普通打开。
db, err := sql.Open("sqlite", fmt.Sprintf("file:%s?_pragma=busy_timeout(30000)", *source))
if err != nil {
return err
}
defer db.Close()
if _, err := db.Exec(`VACUUM INTO ?`, *out); err != nil {
return fmt.Errorf("vacuum into: %w", err)
}
// 校验:快照可打开、完整性、逐表计数与源一致。
snap, err := openSourceRO(*out)
if err != nil {
return err
}
defer snap.Close()
var chk string
if err := snap.QueryRow(`PRAGMA integrity_check`).Scan(&chk); err != nil || chk != "ok" {
return fmt.Errorf("快照完整性校验失败:%v (%q)", err, chk)
}
srcCounts, err := sqliteTableCounts(db)
if err != nil {
return err
}
snapCounts, err := sqliteTableCounts(snap)
if err != nil {
return err
}
for t, n := range srcCounts {
if snapCounts[t] != n {
return fmt.Errorf("快照计数不符:%s 源=%d 快照=%d", t, n, snapCounts[t])
}
}
sum, size, err := fileSHA256(*out)
if err != nil {
return err
}
fmt.Printf("backup ok: %s(%d 字节,sha256 %s…,%d 表计数一致)\n", *out, size, sum[:12], len(srcCounts))
return nil
}
// ---- migrate:受管迁移(§6.3 + U4) ----
// copyOrder:按外键依赖排序的业务表全集(projects 最先,引用方在后)。
// admission_leases 是运行态执行许可(迁移无意义),明确排除。
var copyOrder = []string{
"projects", "users", "profiles", "settings", "plugins",
"conversations", "runs", "messages", "events", "artifacts",
"hitl_cards", "plans", "todos", "knowledge", "knowledge_vectors",
"activity_cards", "project_event_shares", "command_audit", "prompt_modules",
"sessions", "api_keys",
"plugin_grants", "plugin_schema", "plugin_kv", "background_tasks",
"memory_entries", "audit_events", "pending_confirms", "llm_usage_daily",
"secret_keys", "settings_snapshots", "regen_pending", "context_entries",
"user_commands", "pending_notices", "recovery_holds",
"surfaces", "surface_revisions", "surface_actions", "plugin_activation",
// v26 login_keys(#171 合入 main 时漏登记;本 PR 一并补齐)
"login_keys",
// U1 I3:服务实例 DB 身份(无外键;迁移须保留以免目标库丢锚)
"agent_system_identity",
"account_events",
"redeem_attempts",
}
const controlDDL = `
CREATE TABLE IF NOT EXISTS migration_control (
migration_id TEXT PRIMARY KEY,
source_snapshot_id TEXT NOT NULL,
source_schema_version INTEGER NOT NULL,
started_at TEXT NOT NULL,
finished_at TEXT,
verified_at TEXT,
report TEXT
);
CREATE TABLE IF NOT EXISTS migration_table_state (
migration_id TEXT NOT NULL,
table_name TEXT NOT NULL,
source_count INTEGER NOT NULL,
target_count INTEGER NOT NULL,
checksum TEXT NOT NULL,
state TEXT NOT NULL,
finished_at TEXT,
PRIMARY KEY (migration_id, table_name)
);`
func cmdMigrate(args []string) error {
fs := flag.NewFlagSet("migrate", flag.ExitOnError)
source := fs.String("source", "", "冻结快照路径(backup 子命令产物;不接受活跃库)")
target := fs.String("target", "", "目标 PostgreSQL DSN(须为空库或同身份续传库)")
migrationID := fs.String("migration-id", "", "迁移实例名(人工指定,报告与续传身份的一部分)")
artifacts := fs.String("artifacts", "", "工件对象目录(可选;仅盘点清单入报告,不复制)")
out := fs.String("out", "", "报告输出 JSON 路径(缺省打印到 stdout)")
fs.Parse(args)
if *source == "" || *target == "" || *migrationID == "" {
return fmt.Errorf("--source、--target、--migration-id 必填")
}
src, err := openSourceRO(*source)
if err != nil {
return err
}
defer src.Close()
snapID, _, err := fileSHA256(*source)
if err != nil {
return err
}
srcVersion, err := sourceSchemaVersion(src)
if err != nil {
return err
}
// 目标:先用 store 同款迁移器建 schema/扩展(OpenPostgres 会跑迁移与
// 未知版本门禁),再落控制表。DSN 不回显、不入报告。
st, err := store.OpenPostgres(*target, filepath.Join(filepath.Dir(*source), "migrate-artifacts"))
if err != nil {
return fmt.Errorf("open target: %w", err)
}
defer st.Close()
db := st.DB()
if _, err := db.Exec(controlDDL); err != nil {
return fmt.Errorf("control tables: %w", err)
}
if err := guardIdentity(db, *migrationID, snapID, srcVersion); err != nil {
return err
}
if err := ensureControlRow(db, *migrationID, snapID, srcVersion); err != nil {
return err
}
rep := map[string]any{
"command": "migrate", "migration_id": *migrationID,
"source_snapshot_id": snapID, "source_schema_version": srcVersion,
"started_at": nowISO(),
}
var tables []map[string]any
for _, t := range copyOrder {
if !sqliteHasTable(src, t) {
continue // 旧库无此表
}
entry, err := migrateTable(db, src, *migrationID, t)
if err != nil {
return fmt.Errorf("table %s: %w", t, err)
}
tables = append(tables, entry)
}
// 未知 owner 检查(U4:阻塞,不静默)。'' 与 'local' 是单租户默认身份。
if err := checkOwners(db, src); err != nil {
return err
}
// 校验:目标逐表重算计数与校验和,全部一致才落 verified。
if err := verifyTables(db, *migrationID, tables); err != nil {
return err
}
var totalSrc, totalDst int64
for _, e := range tables {
totalSrc += e["source_count"].(int64)
totalDst += e["target_count"].(int64)
}
rep["tables"] = tables
rep["rows_total"] = totalSrc
rep["verified_at"] = nowISO()
if *artifacts != "" {
manifest, err := artifactManifest(*artifacts)
if err != nil {
return err
}
rep["artifacts"] = manifest
}
repJSON, _ := json.Marshal(rep)
if _, err := db.Exec(`UPDATE migration_control SET finished_at=$1, verified_at=$2, report=$3 WHERE migration_id=$4`,
nowISO(), rep["verified_at"], string(repJSON), *migrationID); err != nil {
return err
}
return writeReport(rep, *out)
}
// guardIdentity:目标非空且身份不同即拒绝(R7:不覆盖他人数据)。
// - 存在其他 migration_id 的完成记录 → 拒绝;
// - 同 migration_id 但 source_snapshot_id 不同(换了快照续传)→ 拒绝;
// - 业务表已有数据而控制表为空(人工造过数据)→ 拒绝。
func guardIdentity(db *sql.DB, migrationID, snapID string, srcVersion int) error {
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM migration_control WHERE migration_id<>$1`, migrationID).Scan(&n); err != nil {
return err
}
if n > 0 {
return fmt.Errorf("目标库已有其他迁移记录(%d 条):拒绝覆盖,换空库或专项回迁方案", n)
}
var snap, started string
var sv int
err := db.QueryRow(`SELECT source_snapshot_id, source_schema_version, started_at FROM migration_control WHERE migration_id=$1`, migrationID).Scan(&snap, &sv, &started)
switch {
case err == sql.ErrNoRows:
// 新迁移:业务表必须为空。
busy, err := businessRows(db)
if err != nil {
return err
}
if busy > 0 {
return fmt.Errorf("目标库业务表已有 %d 行且无迁移身份:拒绝(非空目标不得作为迁移目标)", busy)
}
case err != nil:
return err
default:
if snap != snapID {
return fmt.Errorf("同 migration_id 但快照不同(库=%s… 本次=%s…):仅同一冻结快照可续传", snap[:12], snapID[:12])
}
_ = sv
_ = started
}
return nil
}
func businessRows(db *sql.DB) (int64, error) {
var total int64
for _, t := range copyOrder {
var n int64
if err := db.QueryRow(`SELECT COUNT(*) FROM "` + t + `"`).Scan(&n); err != nil {
if strings.Contains(err.Error(), "does not exist") {
continue
}
return 0, err
}
total += n
}
return total, nil
}
func ensureControlRow(db *sql.DB, migrationID, snapID string, srcVersion int) error {
_, err := db.Exec(`INSERT INTO migration_control (migration_id, source_snapshot_id, source_schema_version, started_at)
VALUES ($1,$2,$3,$4)
ON CONFLICT(migration_id) DO NOTHING`, migrationID, snapID, srcVersion, nowISO())
return err
}
func sqliteHasTable(db *sql.DB, name string) bool {
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=?`, name).Scan(&n); err != nil {
return false
}
return n > 0
}
// migrateTable:done 则对账(幂等重跑);pending 则按行复制(布尔映射 +
// 批量 INSERT + 序列推进)后落 done 与源侧校验和。
func migrateTable(db *sql.DB, src *sql.DB, migrationID, table string) (map[string]any, error) {
cols, pk, err := targetColumns(db, table)
if err != nil {
return nil, err
}
var state string
var srcCount, dstCountStored int64
var srcChecksum string
err = db.QueryRow(`SELECT state, source_count, target_count, checksum FROM migration_table_state WHERE migration_id=$1 AND table_name=$2`,
migrationID, table).Scan(&state, &srcCount, &dstCountStored, &srcChecksum)
if err != nil && err != sql.ErrNoRows {
return nil, err
}
if err == sql.ErrNoRows {
state = "pending"
}
// 源侧计数与校验和始终重算(冻结快照上必然稳定;差异=快照被换,拒绝)。
bools, err := boolColumns(db, table, cols)
if err != nil {
return nil, err
}
countNow, sumNow, err := sourceScan(src, table, cols, pk, bools)
if err != nil {
return nil, err
}
if state == "done" {
if countNow != srcCount || sumNow != srcChecksum {
return nil, fmt.Errorf("源与登记不符(疑似换快照):count %d/%d checksum %s…/%s…", countNow, srcCount, sumNow[:12], srcChecksum[:12])
}
entry := map[string]any{"table": table, "source_count": countNow, "target_count": dstCountStored, "checksum": sumNow, "state": "done(idempotent)"}
return entry, nil
}
if err := copyRows(db, src, table, cols); err != nil {
return nil, err
}
dstCount, _, err := targetScan(db, table, cols, pk, bools)
if err != nil {
return nil, err
}
if dstCount != countNow {
return nil, fmt.Errorf("复制后计数不符:源=%d 目标=%d", countNow, dstCount)
}
if err := advanceSequences(db, table); err != nil {
return nil, err
}
if _, err := db.Exec(`INSERT INTO migration_table_state (migration_id, table_name, source_count, target_count, checksum, state, finished_at)
VALUES ($1,$2,$3,$4,$5,$6,$7)
ON CONFLICT(migration_id, table_name) DO UPDATE SET source_count=excluded.source_count, target_count=excluded.target_count, checksum=excluded.checksum, state=excluded.state, finished_at=excluded.finished_at`,
migrationID, table, countNow, dstCount, sumNow, "done", nowISO()); err != nil {
return nil, err
}
return map[string]any{"table": table, "source_count": countNow, "target_count": dstCount, "checksum": sumNow, "state": "done"}, nil
}
// targetColumns:目标侧列清单(按名排序)+ 主键列。布尔列集合同取自目标
// information_schema(类型映射以 PG schema 为准——SQLite 侧 0/1 → bool)。
func targetColumns(db *sql.DB, table string) (cols []string, pk []string, err error) {
// 生成列(PG GENERATED ALWAYS,如 knowledge.tsv)与库内向量副本列
// (vec_pg:BLOB 权威的派生表示,U6-3)不复制不校验:PG 从业务列
// 派生或由 Upsert 双写,源库(SQLite)没有该列。
rows, err := db.Query(`
SELECT column_name, data_type FROM information_schema.columns
WHERE table_schema=current_schema() AND table_name=$1 AND is_generated='NEVER'
AND column_name NOT IN ('vec_pg') ORDER BY column_name`, table)
if err != nil {
return nil, nil, err
}
defer rows.Close()
exists := false
for rows.Next() {
var name, dtype string
if err := rows.Scan(&name, &dtype); err != nil {
return nil, nil, err
}
exists = true
cols = append(cols, name)
}
if !exists {
return nil, nil, fmt.Errorf("目标缺表 %s(迁移器未建?)", table)
}
prows, err := db.Query(`
SELECT a.attname FROM pg_index i
JOIN pg_attribute a ON a.attrelid=i.indrelid AND a.attnum=ANY(i.indkey)
WHERE i.indisprimary AND i.indrelid=$1::regclass`, table)
if err != nil {
return nil, nil, err
}
defer prows.Close()
for prows.Next() {
var p string
if err := prows.Scan(&p); err != nil {
return nil, nil, err
}
pk = append(pk, p)
}
if len(pk) == 0 {
pk = cols // 无主键表按全列序校验
}
return cols, pk, nil
}
func boolColumns(db *sql.DB, table string, cols []string) (map[string]bool, error) {
out := map[string]bool{}
rows, err := db.Query(`
SELECT column_name FROM information_schema.columns
WHERE table_schema=current_schema() AND table_name=$1 AND data_type='boolean'`, table)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var c string
if err := rows.Scan(&c); err != nil {
return nil, err
}
out[c] = true
}
return out, nil
}
// canonicalRow:稳定行编码(校验和用):列名排序、值规范化
// (bool→true/false、[]byte→base64、NULL→null、数值/字符串原样)。
func canonicalRow(cols []string, vals []any, bools map[string]bool) string {
type kv struct {
K string
V any
}
pairs := make([]kv, len(cols))
for i := range cols {
pairs[i] = kv{cols[i], normalize(vals[i], bools[cols[i]])}
}
b, _ := json.Marshal(pairs)
return string(b)
}
// normalize:布尔列按目标 schema 对称归一(源 int64 0/1、目标 bool 编码一致,
// 否则校验和必然不符——真库实测);[]byte→base64。
func normalize(v any, isBool bool) any {
if isBool {
return sqliteBool(v)
}
switch t := v.(type) {
case []byte:
return base64.StdEncoding.EncodeToString(t)
default:
return v
}
}
// hashChain:按主键序哈希链(前后衔接,顺序敏感)。
func hashChain(prev string, rowJSON string) string {
h := sha256.Sum256([]byte(prev + "|" + rowJSON))
return hex.EncodeToString(h[:])
}
// scanChecksum:源/目标共用的扫描对账(列清单一致、主键序、行哈希链)。
func scanChecksum(rows *sql.Rows, cols []string, bools map[string]bool) (int64, string, error) {
vals := make([]any, len(cols))
ptrs := make([]any, len(cols))
for i := range vals {
ptrs[i] = &vals[i]
}
var count int64
sum := ""
for rows.Next() {
if err := rows.Scan(ptrs...); err != nil {
return 0, "", err
}
sum = hashChain(sum, canonicalRow(cols, vals, bools))
count++
}
return count, sum, rows.Err()
}
func sourceScan(src *sql.DB, table string, cols, pk []string, bools map[string]bool) (int64, string, error) {
orderBy := "ORDER BY " + quoteJoin(pk, `"`)
rows, err := src.Query(`SELECT ` + quoteJoin(cols, `"`) + ` FROM "` + table + `" ` + orderBy)
if err != nil {
return 0, "", err
}
defer rows.Close()
return scanChecksum(rows, cols, bools)
}
func targetScan(db *sql.DB, table string, cols, pk []string, bools map[string]bool) (int64, string, error) {
rows, err := db.Query(`SELECT ` + quoteJoin(cols, `"`) + ` FROM "` + table + `" ORDER BY ` + quoteJoin(pk, `"`))
if err != nil {
return 0, "", err
}
defer rows.Close()
return scanChecksum(rows, cols, bools)
}
func quoteJoin(names []string, q string) string {
out := make([]string, len(names))
for i, n := range names {
out[i] = q + n + q
}
return strings.Join(out, ",")
}
// copyRows:按行复制。布尔列 0/1→false/true;其余原样(TEXT/INT/REAL/BLOB)。
func copyRows(db *sql.DB, src *sql.DB, table string, cols []string) error {
bools, err := boolColumns(db, table, cols)
if err != nil {
return err
}
colList := quoteJoin(cols, `"`)
ph := make([]string, len(cols))
for i := range ph {
ph[i] = fmt.Sprintf("$%d", i+1)
}
placeholders := strings.Join(ph, ",")
srcRows, err := src.Query(`SELECT ` + colList + ` FROM "` + table + `"`)
if err != nil {
return err
}
defer srcRows.Close()
vals := make([]any, len(cols))
ptrs := make([]any, len(cols))
for i := range vals {
ptrs[i] = &vals[i]
}
tx, err := db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
stmt, err := tx.Prepare(fmt.Sprintf(`INSERT INTO "%s" (%s) VALUES (%s) ON CONFLICT DO NOTHING`, table, colList, placeholders))
if err != nil {
return err
}
defer stmt.Close()
var batch int
for srcRows.Next() {
if err := srcRows.Scan(ptrs...); err != nil {
return err
}
args := make([]any, len(cols))
for i, c := range cols {
v := vals[i]
if bools[c] {
args[i] = sqliteBool(v)
} else {
args[i] = v
}
}
if _, err := stmt.Exec(args...); err != nil {
return fmt.Errorf("insert: %w", err)
}
batch++
}
if err := srcRows.Err(); err != nil {
return err
}
return tx.Commit()
}
func sqliteBool(v any) bool {
switch t := v.(type) {
case int64:
return t != 0
case bool:
return t
default:
return false
}
}
// advanceSequences:SERIAL/identity 列复制后推进序列(显式插入不推进,
// 后续应用插入会撞主键)。无序列表为空操作。
func advanceSequences(db *sql.DB, table string) error {
rows, err := db.Query(`
SELECT pg_get_serial_sequence($1, column_name)
FROM information_schema.columns
WHERE table_schema=current_schema() AND table_name=$1
AND (column_default LIKE 'nextval%' OR is_identity='YES')`, table)
if err != nil {
return err
}
defer rows.Close()
var seqs []string
for rows.Next() {
var s any
if err := rows.Scan(&s); err != nil {
return err
}
if s != nil {
seqs = append(seqs, s.(string))
}
}
if err := rows.Err(); err != nil {
return err
}
for _, s := range seqs {
if _, err := db.Exec(fmt.Sprintf(`SELECT setval('%s', COALESCE((SELECT MAX(id) FROM "%s"), 1))`, s, table)); err != nil {
return fmt.Errorf("setval %s: %w", s, err)
}
}
return nil
}
// checkOwners:业务行引用的 owner 必须存在于 users(单租户默认 ”/'local'
// 除外)。未知 owner 阻塞迁移(U4/R7:不静默带过未知归属)。
func checkOwners(db *sql.DB, src *sql.DB) error {
ownerTables := []string{"projects", "conversations", "runs", "artifacts", "knowledge", "hitl_cards", "profiles", "activity_cards", "project_event_shares", "command_audit"}
for _, t := range ownerTables {
if !sqliteHasTable(src, t) {
continue
}
rows, err := src.Query(`SELECT DISTINCT owner FROM "` + t + `" WHERE owner<>'' AND owner<>'local'`)
if err != nil {
return err
}
var unknown []string
for rows.Next() {
var o string
if err := rows.Scan(&o); err != nil {
rows.Close()
return err
}
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM users WHERE owner=$1`, o).Scan(&n); err != nil {
rows.Close()
return err
}
if n == 0 {
unknown = append(unknown, o)
}
}
rows.Close()
if err := rows.Err(); err != nil {
return err
}
if len(unknown) > 0 {
return fmt.Errorf("表 %s 引用未知 owner(users 无此行):%v——先在目标补建用户或修正归属,再迁移", t, unknown)
}
}
return nil
}
// verifyTables:目标逐表重算,与登记的源计数/校验和全部一致才通过。
func verifyTables(db *sql.DB, migrationID string, tables []map[string]any) error {
for _, e := range tables {
table := e["table"].(string)
wantCount := e["source_count"].(int64)
wantSum := e["checksum"].(string)
cols, pk, err := targetColumns(db, table)
if err != nil {
return err
}
bools, err := boolColumns(db, table, cols)
if err != nil {
return err
}
gotCount, gotSum, err := targetScan(db, table, cols, pk, bools)
if err != nil {
return err
}
if gotCount != wantCount || gotSum != wantSum {
return fmt.Errorf("表 %s 校验不符:count %d/%d checksum %s…/%s…", table, gotCount, wantCount, gotSum[:12], wantSum[:12])
}
}
return nil
}
func writeReport(rep map[string]any, out string) error {
b, err := json.MarshalIndent(rep, "", " ")
if err != nil {
return err
}
if out == "" {
fmt.Println(string(b))
return nil
}
if err := os.MkdirAll(filepath.Dir(out), 0o755); err != nil {
return err
}
return os.WriteFile(out, append(b, '\n'), 0o644)
}
+197
View File
@@ -0,0 +1,197 @@
package main
import (
"encoding/json"
"path/filepath"
"strings"
"testing"
"agent-system-backend/internal/store"
)
// U4 迁移工具测试:夹具覆盖全部当前 schema 的边值(布尔/空值/时间/JSON/
// Unicode/二进制向量/秘密字段),以及 inventory/backup 的行为面。
// 真库端到端(幂等/续传/非空拒/未知 owner)在 integration_test.go(PG 门控)。
// buildFixture:建一个全 schema 的 SQLite 库并填充边值行,返回路径。
func buildFixture(t *testing.T) string {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "fixture.db")
st, err := store.OpenSQLite(path)
if err != nil {
t.Fatal(err)
}
defer st.Close()
seedEdgeRows(t, st)
return path
}
func seedEdgeRows(t *testing.T, st *store.Store) {
t.Helper()
// users(含 is_admin/must_change 布尔)与单租户默认身份。
mustExec(t, st, `INSERT INTO users (id, owner, name, created_at, password_hash, password_set_at, must_change_password, is_admin) VALUES
('u1', 'alice', '爱丽丝', '2026-10-11T01:00:00Z', 'sha256:…', '2026-10-11T01:00:00Z', false, true),
('u2', 'bob', 'Bob 😊', '2026-10-11T01:00:00Z', '', '', true, false)`)
// 项目与会话(布尔 pinned/archived/allow_cross_read/cross_private、空 JSON)。
mustExec(t, st, `INSERT INTO projects (id, name, settings, created_at, updated_at, owner) VALUES
('p1', '项目一', '{}', '2026-10-11T01:00:00Z', '2026-10-11T01:00:00Z', 'alice'),
('p2', 'Emoji 项目 🚀', '{"a":null,"b":[1,2,"三"]}', '2026-10-11T02:00:00Z', '2026-10-11T02:00:00Z', '')`)
mustExec(t, st, `INSERT INTO conversations (id, project_id, title, profile_id, created_at, updated_at, owner, pinned, archived, allow_cross_read, cross_private, permission_mode, session_note) VALUES
('c1', 'p1', '会话', '', '2026-10-11T01:00:00Z', '2026-10-11T01:00:00Z', 'alice', true, false, false, true, '', 'note—注'),
('c2', 'p2', '', '', '2026-10-11T02:00:00Z', '2026-10-11T02:00:00Z', '', false, true, true, false, 'strict', '')`)
// 消息(request_id 幂等、parts JSON 含空对象/空数组/Unicode)与 seq 顺序。
mustExec(t, st, `INSERT INTO messages (id, conversation_id, run_id, role, parts, seq, created_at, request_id) VALUES
('m1', 'c1', '', 'user', '[{"type":"text","text":"你好"}]', 1, '2026-10-11T01:00:01Z', 'req-1'),
('m2', 'c1', '', 'assistant', '[{"type":"kv","kv":[]},{"type":"text","text":""}]', 2, '2026-10-11T01:00:02Z', ''),
('m3', 'c2', '', 'system', '[]', 1, '2026-10-11T02:00:01Z', 'req-2')`)
// runs(NULL ended_at/usage/error)。
mustExec(t, st, `INSERT INTO runs (id, parent_run_id, conversation_id, project_id, owner, kind, profile_id, status, phase, agent_name, created_at, instance_id) VALUES
('r1', '', 'c1', 'p1', 'alice', 'conversation', '', 'completed', '', '', '2026-10-11T01:00:00Z', 'host:1'),
('r2', 'r1', 'c1', 'p1', 'alice', 'subagent', '', 'running', 'thinking', '检索', '2026-10-11T01:00:03Z', 'host:2')`)
// events(复合主键 (conversation_id, seq))。
mustExec(t, st, `INSERT INTO events (conversation_id, seq, run_id, ts, v, type, payload) VALUES
('c1', 1, 'r1', '2026-10-11T01:00:01Z', 1, 'run.phase', '{"phase":"thinking"}'),
('c1', 2, 'r1', '2026-10-11T01:00:02Z', 1, 'message.appended', NULL),
('c2', 1, '', '2026-10-11T02:00:01Z', 1, 'x', '{}')`)
// 工件(size 0 与大值)。
mustExec(t, st, `INSERT INTO artifacts (id, project_id, name, kind, mime, size, source_run, content_path, created_at, updated_at, owner) VALUES
('a1', 'p1', '报告.md', 'doc', 'text/markdown', 0, 'r1', 'art/a1', '2026-10-11T01:00:00Z', '2026-10-11T01:00:00Z', 'alice'),
('a2', 'p2', '数据.xlsx', 'doc', '', 2147483647, '', '', '2026-10-11T02:00:00Z', '2026-10-11T02:00:00Z', '')`)
// HITL 卡(resolved=0、resolution NULL)。
mustExec(t, st, `INSERT INTO hitl_cards (id, run_id, conversation_id, payload, created_at, resolved, resolution, owner) VALUES
('h1', 'r2', 'c1', '{"q":"继续?"}', '2026-10-11T01:00:04Z', 0, NULL, 'alice')`)
// 知识 + 向量(BLOB 二进制:float32 little-endian)。
mustExec(t, st, `INSERT INTO knowledge (id, project_id, source, title, ref, chunk, created_at, owner) VALUES
('k1', 'p1', 'upload', '标题', 'doc.md', '正文内容…', '2026-10-11T01:00:00Z', 'alice')`)
mustExec(t, st, `INSERT INTO knowledge_vectors (doc_id, owner, ref, dim, vec, created_at) VALUES
('k1', 'alice', 'doc.md', 3, x'0000803F0000003F000080BF', '2026-10-11T01:00:00Z')`)
// 活动卡(pinned 布尔)。
mustExec(t, st, `INSERT INTO activity_cards (id, project_id, conversation_id, ref, owner, payload, status, pinned, created_at, updated_at, version) VALUES
('ac1', 'p1', 'c1', '', 'alice', '{"title":"卡"}', 'active', 1, '2026-10-11T01:00:00Z', '2026-10-11T01:00:00Z', 2)`)
// 命令审计(SERIAL 主键:显式 id 插入)。
mustExec(t, st, `INSERT INTO command_audit (id, time, command_id, source, entrance, state, operation_id, conversation_id, payload, owner) VALUES
(7, '2026-10-11T01:00:00Z', 'cmd-1', 'http', 'chat', 'done', 'op-1', 'c1', '{"actor":"alice"}', 'alice')`)
// 设置('' 默认租户 scope)。
mustExec(t, st, `INSERT INTO settings (owner, scope, "values") VALUES ('', 'runtime', '{"rateLimitPerMin":0}')`)
// 秘密字段(值不外泄:迁移按行复制,报告无正文)。
mustExec(t, st, `INSERT INTO secret_keys (id, owner, service, label, value, active, created_at) VALUES
('sk1', 'alice', 'openai', 'main', 'sk-secret-value-不打印', 1, '2026-10-11T01:00:00Z')`)
// 恢复隔离(业务态,应随迁)。
mustExec(t, st, `INSERT INTO recovery_holds (run_id, owner, conversation_id, reason, created_at) VALUES
('rh1', 'alice', 'c1', '副作用未知', '2026-10-11T01:00:00Z')`)
}
func must(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatal(err)
}
}
// mustExec:夹具直接走底层连接写行(业务代码不得如此;见 store.DB 注释)。
func mustExec(t *testing.T, st *store.Store, q string) {
t.Helper()
if _, err := st.DB().Exec(q); err != nil {
t.Fatal(err)
}
}
// ---- inventory ----
func TestInventoryReport(t *testing.T) {
path := buildFixture(t)
db, err := openSourceRO(path)
if err != nil {
t.Fatal(err)
}
defer db.Close()
rep, err := inventoryReport(db, path, "")
if err != nil {
t.Fatal(err)
}
if rep["source_schema_version"].(int) < 20 {
t.Fatalf("schema version: %v", rep["source_schema_version"])
}
counts := rep["table_counts"].(map[string]int64)
if counts["messages"] != 3 || counts["events"] != 3 || counts["users"] != 2 {
t.Fatalf("counts: %v", counts)
}
b, _ := json.Marshal(rep)
if strings.Contains(string(b), "sk-secret-value") {
t.Fatal("inventory 报告不得包含秘密值")
}
}
func TestInventoryMissingFile(t *testing.T) {
if _, err := openSourceRO(filepath.Join(t.TempDir(), "nope.db")); err == nil {
t.Fatal("缺文件应报错")
}
}
// ---- backup ----
func TestBackupVerifiesAndSnapshotsConsistent(t *testing.T) {
path := buildFixture(t)
out := filepath.Join(t.TempDir(), "snap.db")
if err := cmdBackup([]string{"--source", path, "--out", out}); err != nil {
t.Fatal(err)
}
snap, err := openSourceRO(out)
if err != nil {
t.Fatal(err)
}
defer snap.Close()
var n int
if err := snap.QueryRow(`SELECT COUNT(*) FROM messages`).Scan(&n); err != nil || n != 3 {
t.Fatalf("snap messages: %d %v", n, err)
}
// 已存在的输出拒绝覆盖。
if err := cmdBackup([]string{"--source", path, "--out", out}); err == nil {
t.Fatal("已存在快照应拒绝")
}
}
// ---- 夹具即文档:确认全部业务表在 copyOrder 内(防新表漏迁) ----
func TestCopyOrderCoversAllFixtureTables(t *testing.T) {
path := buildFixture(t)
db, err := openSourceRO(path)
if err != nil {
t.Fatal(err)
}
defer db.Close()
rows, err := db.Query(`SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
known := map[string]bool{}
for _, t := range copyOrder {
known[t] = true
}
known["schema_migrations"] = true // 迁移器管理,不随迁
// 运行态表不随迁:执行许可是进程瞬态;执行租约与跨实例命令属当前
// 实例的调度现场(接管语义在 U5 内由 lease/fencing 处理,不复制历史)。
for _, rt := range []string{"admission_leases", "conversation_executions", "run_commands"} {
known[rt] = true
}
// FTS5 虚表影子表:SQLite 内部结构,PG 侧由 v4 生成列/索引从 knowledge
// 行自动重建,不随迁。
for _, shadow := range []string{"knowledge_fts", "knowledge_fts_data", "knowledge_fts_idx", "knowledge_fts_docsize", "knowledge_fts_config"} {
known[shadow] = true
}
var missing []string
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
t.Fatal(err)
}
if !known[name] {
missing = append(missing, name)
}
}
if len(missing) > 0 {
t.Fatalf("copyOrder 漏表(新表加入 schema 后须同步工具清单):%v", missing)
}
}
+175
View File
@@ -0,0 +1,175 @@
package eval
import (
"context"
"fmt"
"sort"
"strings"
"time"
)
// ab.go — 任务级 A/B 双跑 harness(Q3 WP5):同一批任务分别经两个入口
// (direct 对话循环 / squad 编译图)执行,逐任务记录差异(不要求字节级
// 相同),汇总为两份 EntryScore 交 DecideSwitch 裁决默认入口切换。
// 本包保持纯机制(不依赖 runtime/squad);两侧 RunFunc 由调用方装配。
// ABTask 是对照批中的一个任务。
type ABTask struct {
ID string
Goal string
// Themes 期望产出要点(短词,非整句照抄):打分用 MissingThemes,
// 差异记录用要点覆盖对比。
Themes []string
}
// ABOutcome 是一个入口对一个任务的执行结果(调用方 RunFunc 填写)。
type ABOutcome struct {
// Completed 任务是否正常收口(失败/取消 = false)。
Completed bool
// Summary 交付摘要(要点覆盖打分的对象;调用方可拼接工件正文)。
Summary string
// FirstToken 首字延迟(入口收到任务到首个可见输出)。
FirstToken time.Duration
// Err 失败原因(Completed=false 时填写)。
Err string
}
// RunFunc 是一个入口的执行侧(direct 或 squad 编译图)。
type RunFunc func(ctx context.Context, task ABTask) (ABOutcome, error)
// ABTaskRecord 是一个任务的双跑对照记录(差异记录单位)。
type ABTaskRecord struct {
Task ABTask
Direct ABOutcome
Squad ABOutcome
// Diffs 差异注记(状态 / 要点覆盖 / 首字延迟),空 = 两侧一致。
Diffs []string
// DirectPoints / SquadPoints 单任务得分(满分 100:完成 + 要点全覆盖;
// 未完成 0 分;完成但缺要点按缺失比例扣)。
DirectPoints int
SquadPoints int
}
// ABReport 是一批双跑的对照报告(切换裁决的证据形态)。
type ABReport struct {
Records []ABTaskRecord
Direct EntryScore
Squad EntryScore
Decision SwitchDecision
// Text 是可附在切换 PR / 任务卡上的短报告(Report(Decision) + 差异清单)。
Text string
}
// RunAB 双跑同一批任务并汇总对照报告。任一侧 RunFunc 返回 error 视为该侧
// 该任务失败(记录进 outcome.Err),不中断整批(对照要的是全批差异面)。
func RunAB(ctx context.Context, tasks []ABTask, direct, squad RunFunc) ABReport {
rep := ABReport{}
if len(tasks) == 0 {
rep.Decision = DecideSwitch(nil, nil)
rep.Text = Report(rep.Decision)
return rep
}
var directFT, squadFT []time.Duration
var directSum, squadSum int
for _, task := range tasks {
rec := ABTaskRecord{Task: task}
rec.Direct = runOne(ctx, task, direct)
rec.Squad = runOne(ctx, task, squad)
rec.DirectPoints = scoreOutcome(rec.Direct, task.Themes)
rec.SquadPoints = scoreOutcome(rec.Squad, task.Themes)
rec.Diffs = diffRecord(rec)
rep.Records = append(rep.Records, rec)
directSum += rec.DirectPoints
squadSum += rec.SquadPoints
directFT = append(directFT, rec.Direct.FirstToken)
squadFT = append(squadFT, rec.Squad.FirstToken)
}
rep.Direct = EntryScore{Name: "direct", Score: directSum / len(tasks), FirstTokenP50: p50(directFT)}
rep.Squad = EntryScore{Name: "squad", Score: squadSum / len(tasks), FirstTokenP50: p50(squadFT)}
rep.Decision = DecideSwitch(&rep.Direct, &rep.Squad)
rep.Text = renderReport(rep)
return rep
}
func runOne(ctx context.Context, task ABTask, fn RunFunc) ABOutcome {
if fn == nil {
return ABOutcome{Err: "入口未装配"}
}
out, err := fn(ctx, task)
if err != nil {
out.Completed = false
if out.Err == "" {
out.Err = err.Error()
}
}
return out
}
// scoreOutcome 单任务打分:未完成 0;完成 = 100 - 缺失要点比例 × 100。
func scoreOutcome(out ABOutcome, themes []string) int {
if !out.Completed {
return 0
}
if len(themes) == 0 {
return 100
}
missing := MissingThemes(out.Summary, themes)
return 100 - len(missing)*100/len(themes)
}
// diffRecord 逐任务差异注记(状态 / 要点 / 首字延迟;不要求字节级相同)。
func diffRecord(rec ABTaskRecord) []string {
var diffs []string
if rec.Direct.Completed != rec.Squad.Completed {
diffs = append(diffs, fmt.Sprintf("状态:direct=%v squad=%v(%s / %s)",
rec.Direct.Completed, rec.Squad.Completed, rec.Direct.Err, rec.Squad.Err))
}
if len(rec.Task.Themes) > 0 {
dm := MissingThemes(rec.Direct.Summary, rec.Task.Themes)
sm := MissingThemes(rec.Squad.Summary, rec.Task.Themes)
if strings.Join(dm, ",") != strings.Join(sm, ",") {
diffs = append(diffs, fmt.Sprintf("要点缺失:direct=%v squad=%v", dm, sm))
}
}
gap := rec.Squad.FirstToken - rec.Direct.FirstToken
if gap < 0 {
gap = -gap
}
if gap > time.Second {
diffs = append(diffs, fmt.Sprintf("首字延迟差:%s(direct=%s squad=%s)",
gap, rec.Direct.FirstToken, rec.Squad.FirstToken))
}
if rec.Direct.Completed && rec.Squad.Completed &&
strings.TrimSpace(rec.Direct.Summary) != strings.TrimSpace(rec.Squad.Summary) {
diffs = append(diffs, "摘要文本不同(记录在案,不要求字节级相同)")
}
return diffs
}
func p50(ds []time.Duration) time.Duration {
if len(ds) == 0 {
return 0
}
sorted := append([]time.Duration(nil), ds...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i] < sorted[j] })
return sorted[len(sorted)/2]
}
// renderReport 报告全文:裁决 + 分数 + 逐任务差异清单。
func renderReport(rep ABReport) string {
var b strings.Builder
b.WriteString(Report(rep.Decision))
fmt.Fprintf(&b, "\n分数:direct=%d squad=%d;首字P50:direct=%s squad=%s\n",
rep.Direct.Score, rep.Squad.Score, rep.Direct.FirstTokenP50, rep.Squad.FirstTokenP50)
for _, rec := range rep.Records {
fmt.Fprintf(&b, "- %s:direct %d 分 / squad %d 分", rec.Task.ID, rec.DirectPoints, rec.SquadPoints)
if len(rec.Diffs) == 0 {
b.WriteString("(无差异注记)")
}
b.WriteString("\n")
for _, d := range rec.Diffs {
fmt.Fprintf(&b, " - %s\n", d)
}
}
return strings.TrimRight(b.String(), "\n")
}
+124
View File
@@ -0,0 +1,124 @@
package eval
import (
"context"
"strings"
"testing"
"time"
)
// ab_test.go — Q3 WP5:A/B 双跑 harness 机制面(差异记录 / 打分 / P50 /
// DecideSwitch 联动)。真实入口的集成对照在 internal/wfcompile 的
// demo.brief 双跑测试。
func stubRun(completed bool, summary string, ft time.Duration, errText string) RunFunc {
return func(context.Context, ABTask) (ABOutcome, error) {
return ABOutcome{Completed: completed, Summary: summary, FirstToken: ft, Err: errText}, nil
}
}
func TestRunABRecordsDiffsAndScores(t *testing.T) {
tasks := []ABTask{
{ID: "t1", Goal: "写简报", Themes: []string{"太空电梯", "核对标记"}},
{ID: "t2", Goal: "查资料", Themes: []string{"三来源"}},
}
direct := func(_ context.Context, task ABTask) (ABOutcome, error) {
switch task.ID {
case "t1":
return ABOutcome{Completed: true, Summary: "关于太空电梯的简报,核对标记:P4", FirstToken: 100 * time.Millisecond}, nil
default:
return ABOutcome{Completed: true, Summary: "对照完成,三来源齐", FirstToken: 200 * time.Millisecond}, nil
}
}
squad := func(_ context.Context, task ABTask) (ABOutcome, error) {
switch task.ID {
case "t1":
// 完成但缺一个要点 → 扣分 + 差异注记。
return ABOutcome{Completed: true, Summary: "关于太空电梯的简报", FirstToken: 100 * time.Millisecond}, nil
default:
return ABOutcome{Completed: false, Err: "编译图步骤失败", FirstToken: 0}, nil
}
}
rep := RunAB(context.Background(), tasks, direct, squad)
if len(rep.Records) != 2 {
t.Fatalf("records = %d", len(rep.Records))
}
r1 := rep.Records[0]
if r1.DirectPoints != 100 || r1.SquadPoints != 50 {
t.Fatalf("t1 points = %d/%d", r1.DirectPoints, r1.SquadPoints)
}
if !strings.Contains(strings.Join(r1.Diffs, ";"), "要点缺失") {
t.Fatalf("t1 diffs = %v", r1.Diffs)
}
r2 := rep.Records[1]
if r2.SquadPoints != 0 || !strings.Contains(strings.Join(r2.Diffs, ";"), "状态") {
t.Fatalf("t2 = %d %v", r2.SquadPoints, r2.Diffs)
}
// 分数汇总:direct 100,squad (50+0)/2=25 → 不达标,保持 direct。
if rep.Direct.Score != 100 || rep.Squad.Score != 25 {
t.Fatalf("scores = %d/%d", rep.Direct.Score, rep.Squad.Score)
}
if rep.Decision.Met || !strings.Contains(rep.Decision.Reason, "低于") {
t.Fatalf("decision = %+v", rep.Decision)
}
if !strings.Contains(rep.Text, "保持 direct") {
t.Fatalf("report = %s", rep.Text)
}
}
func TestRunABSwitchMetWhenEqual(t *testing.T) {
tasks := []ABTask{{ID: "t1", Goal: "g", Themes: []string{"要点"}}}
fn := stubRun(true, "覆盖要点的产出", 50*time.Millisecond, "")
rep := RunAB(context.Background(), tasks, fn, fn)
if rep.Direct.Score != 100 || rep.Squad.Score != 100 {
t.Fatalf("scores = %d/%d", rep.Direct.Score, rep.Squad.Score)
}
if !rep.Decision.Met {
t.Fatalf("同分同延迟应达标:%+v", rep.Decision)
}
if len(rep.Records[0].Diffs) != 0 {
t.Fatalf("同产出应无差异注记:%v", rep.Records[0].Diffs)
}
if rep.Direct.FirstTokenP50 != 50*time.Millisecond {
t.Fatalf("p50 = %s", rep.Direct.FirstTokenP50)
}
}
func TestRunABLatencyGate(t *testing.T) {
tasks := []ABTask{{ID: "t1", Goal: "g"}}
direct := stubRun(true, "ok", 100*time.Millisecond, "")
squad := stubRun(true, "ok", 4*time.Second, "") // 额外首字 > 3s
rep := RunAB(context.Background(), tasks, direct, squad)
if rep.Decision.Met || !strings.Contains(rep.Decision.Reason, "超过 3 秒") {
t.Fatalf("decision = %+v", rep.Decision)
}
if !strings.Contains(strings.Join(rep.Records[0].Diffs, ";"), "首字延迟差") {
t.Fatalf("diffs = %v", rep.Records[0].Diffs)
}
}
// RunFunc 返回 error:记为该侧失败,不中断整批。
func TestRunABToleratesRunFuncError(t *testing.T) {
tasks := []ABTask{{ID: "t1"}, {ID: "t2"}}
direct := stubRun(true, "ok", 0, "")
squad := func(_ context.Context, task ABTask) (ABOutcome, error) {
if task.ID == "t1" {
return ABOutcome{}, context.DeadlineExceeded
}
return ABOutcome{Completed: true, Summary: "ok"}, nil
}
rep := RunAB(context.Background(), tasks, direct, squad)
if rep.Records[0].SquadPoints != 0 || rep.Records[0].Squad.Err == "" {
t.Fatalf("t1 = %+v", rep.Records[0].Squad)
}
if rep.Records[1].SquadPoints != 100 {
t.Fatalf("t2 应不受 t1 失败影响:%+v", rep.Records[1])
}
}
func TestRunABEmptyBatch(t *testing.T) {
rep := RunAB(context.Background(), nil, nil, nil)
if rep.Decision.Met {
t.Fatal("空批不得达标(无实测分数)")
}
}
+457
View File
@@ -0,0 +1,457 @@
//go:build live
package live
import (
"archive/zip"
"bytes"
"encoding/json"
"encoding/xml"
"fmt"
"io"
"mime/multipart"
"net/http"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
"time"
"agent-system-backend/internal/runtime"
)
// upload 上传一个材料文件,返回 artifact id。
func (c *client) upload(t *testing.T, projectID, name string, content []byte) string {
t.Helper()
var buf bytes.Buffer
w := multipart.NewWriter(&buf)
fw, err := w.CreateFormFile("file", name)
if err != nil {
t.Fatalf("multipart field: %v", err)
}
if _, err := fw.Write(content); err != nil {
t.Fatalf("multipart write: %v", err)
}
w.Close()
req, err := http.NewRequest("POST", c.base+"/api/projects/"+projectID+"/uploads", &buf)
if err != nil {
t.Fatalf("upload request: %v", err)
}
req.Header.Set("Content-Type", w.FormDataContentType())
resp, err := c.http.Do(req)
if err != nil {
t.Fatalf("upload: %v", err)
}
defer resp.Body.Close()
raw, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 300 {
t.Fatalf("upload %s: %s: %s", name, resp.Status, truncate(string(raw), 300))
}
var out struct {
ID string `json:"artifactId"`
LegacyID string `json:"id"`
}
if err := json.Unmarshal(raw, &out); err != nil {
t.Fatalf("upload response: %v %s", err, truncate(string(raw), 300))
}
if out.ID == "" {
out.ID = out.LegacyID
}
if out.ID == "" {
t.Fatalf("upload response missing artifact id: %s", truncate(string(raw), 300))
}
return out.ID
}
// runTurn 发一轮消息并等到 run 终态;期间自动应答 HITL 卡。
func runTurn(t *testing.T, c *client, projectID, convID, text string, attachmentIDs []string, turnTimeout time.Duration) turnResult {
t.Helper()
started := time.Now()
var out struct {
Message message `json:"message"`
Run run `json:"run"`
}
body := map[string]any{"text": text, "requestId": fmt.Sprintf("eval-%d", time.Now().UnixNano())}
if len(attachmentIDs) > 0 {
body["attachmentIds"] = attachmentIDs
}
if err := c.do("POST", "/api/conversations/"+convID+"/messages", body, &out); err != nil {
t.Fatalf("post message: %v", err)
}
runID := out.Run.ID
deadline := time.Now().Add(turnTimeout)
var cur run
for {
var resp struct {
Run run `json:"run"`
}
if err := c.do("GET", "/api/runs/"+runID, nil, &resp); err != nil {
t.Fatalf("get run: %v", err)
}
cur = resp.Run
if isTerminal(cur.Status) {
break
}
autoResolveHITL(t, c, convID)
if time.Now().After(deadline) {
t.Fatalf("run %s timed out after %s (status=%s)", runID, turnTimeout, cur.Status)
}
time.Sleep(3 * time.Second)
}
detail := struct {
Messages []message `json:"messages"`
ActiveHitl []hitlCard `json:"activeHitl"`
}{}
if err := c.do("GET", "/api/conversations/"+convID, nil, &detail); err != nil {
t.Fatalf("conversation detail: %v", err)
}
var assistant *message
for i := range detail.Messages {
m := detail.Messages[i]
if m.RunID == runID && m.Role == "assistant" && !m.Excluded {
assistant = &detail.Messages[i]
}
}
tr := turnResult{User: text, RunID: runID, Status: cur.Status, DurationMs: time.Since(started).Milliseconds()}
if cur.Usage != nil {
tr.Usage = &cur
}
if assistant == nil {
tr.Issues = append(tr.Issues, "没有找到本回合的助手消息")
return tr
}
collectMetrics(&tr, assistant)
// 工件清单:本 run 产出的。
var arts []artifactMeta
if err := c.do("GET", "/api/projects/"+projectID+"/artifacts", nil, &arts); err == nil {
for _, a := range arts {
if a.SourceRun == runID {
tr.Artifacts = append(tr.Artifacts, a.Name)
}
}
}
if cur.Status != "completed" {
tr.Issues = append(tr.Issues, "run 终态="+cur.Status)
}
return tr
}
func isTerminal(status string) bool {
switch status {
case "completed", "failed", "cancelled":
return true
}
return false
}
// autoResolveHITL 无人值守:有活动确认卡时选第一个选项(没有选项给自由文本同意)。
func autoResolveHITL(t *testing.T, c *client, convID string) {
t.Helper()
detail := struct {
ActiveHitl []hitlCard `json:"activeHitl"`
}{}
if err := c.do("GET", "/api/conversations/"+convID, nil, &detail); err != nil {
return
}
for _, card := range detail.ActiveHitl {
resp := map[string]string{}
if len(card.Choices) > 0 {
resp["choiceId"] = card.Choices[0].ID
} else if card.AllowFreeInput {
resp["freeText"] = "同意继续(评测自动应答)"
} else {
continue
}
_ = c.do("POST", "/api/hitl/"+card.ID+"/resolve", resp, nil)
}
}
func collectMetrics(tr *turnResult, m *message) {
for _, p := range m.Parts {
switch p.Type {
case "text":
if !p.Process && strings.TrimSpace(p.Text) != "" {
tr.FinalText = p.Text
}
case "tool_call":
if p.ToolCall == nil {
continue
}
tr.Metrics.ToolCalls++
args := p.ToolCall.Arguments
switch p.ToolCall.Name {
case "update_plan", "update_todos":
tr.Metrics.PlanUpdates++
case "run_python":
if strings.Contains(args, "new_document(") || strings.Contains(args, "new_presentation(") || strings.Contains(args, "new_workbook(") {
tr.Metrics.FullRegens++
}
}
if strings.Contains(args, "dir(") || strings.Contains(args, "inspect.signature") || strings.Contains(args, "help(") {
tr.Metrics.APIDiscovery++
}
if strings.Contains(args, "inputs/") {
tr.Metrics.ReadsOriginal++
}
case "tool_result":
if p.ToolResult == nil {
continue
}
s := p.ToolResult.Summary
if p.ToolResult.Status == "error" || strings.HasPrefix(s, "exit=1") || strings.Contains(s, "Traceback") || strings.Contains(s, "ModuleNotFoundError") {
tr.Metrics.ToolFailures++
}
case "check":
if p.Check == nil {
continue
}
tr.Metrics.VerifierChecks++
tr.Checks = append(tr.Checks, checkRecord{Passed: p.Check.Passed, Reason: p.Check.Reason})
}
}
}
func lastArtifactTurn(tr taskResult) *turnResult {
for i := len(tr.Turns) - 1; i >= 0; i-- {
if len(tr.Turns[i].Artifacts) > 0 {
return &tr.Turns[i]
}
}
return nil
}
func downloadArtifactText(t *testing.T, c *client, projectID, name string) string {
t.Helper()
var arts []artifactMeta
if err := c.do("GET", "/api/projects/"+projectID+"/artifacts", nil, &arts); err != nil {
return ""
}
for _, a := range arts {
if a.Name != name {
continue
}
req, _ := http.NewRequest("GET", c.base+"/api/artifacts/"+a.ID+"/content", nil)
resp, err := c.http.Do(req)
if err != nil {
continue
}
raw, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if strings.HasSuffix(strings.ToLower(a.Name), ".docx") {
return docxTextIfDocx(t, a.Name, raw)
}
return string(raw)
}
return ""
}
// ---- 文本抽取与事实核查 ----
// docxTextIfDocx:docx 抽纯文本(标题行带 # 前缀);其他类型原样返回。
func docxTextIfDocx(t *testing.T, name string, raw []byte) string {
t.Helper()
if !strings.HasSuffix(strings.ToLower(name), ".docx") {
return string(raw)
}
zr, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw)))
if err != nil {
return ""
}
for _, f := range zr.File {
if f.Name != "word/document.xml" {
continue
}
rc, err := f.Open()
if err != nil {
return ""
}
defer rc.Close()
dec := xml.NewDecoder(rc)
var b strings.Builder
var para strings.Builder
style, inPara := "", false
flush := func() {
text := strings.TrimSpace(para.String())
if text != "" {
if lvl := docxHeadingLevel(style); lvl > 0 {
b.WriteString(strings.Repeat("#", lvl) + " ")
}
b.WriteString(text + "\n")
}
para.Reset()
style = ""
}
for {
tok, err := dec.Token()
if err != nil {
break
}
switch el := tok.(type) {
case xml.StartElement:
switch el.Name.Local {
case "p":
inPara, style = true, ""
para.Reset()
case "pStyle":
if inPara {
for _, at := range el.Attr {
if at.Name.Local == "val" {
style = at.Value
}
}
}
}
case xml.EndElement:
if el.Name.Local == "p" {
inPara = false
flush()
}
case xml.CharData:
if inPara {
para.Write(bytes.TrimSpace(el))
}
}
}
flush()
return b.String()
}
return ""
}
func docxHeadingLevel(style string) int {
switch {
case style == "Title":
return 1
case style == "Subtitle":
return 2
case strings.HasPrefix(style, "Heading"):
n := 0
for _, ch := range style[len("Heading"):] {
if ch < '0' || ch > '9' {
return 0
}
n = n*10 + int(ch-'0')
}
if n >= 1 && n <= 9 {
return n
}
}
return 0
}
var (
pctRe = regexp.MustCompile(`\d+(?:\.\d+)?%`)
dateRe = regexp.MustCompile(`\d{1,2}月\d{1,2}日`)
bigNumRe = regexp.MustCompile(`\d{4,}`)
claimRe = regexp.MustCompile(`(?:^|\n|\s)([一二三四五六七八九十]{1,2}、[^\s,。;|]{2,24})`)
)
// unverifiedFacts 抽取交付物里的百分比/日期/大数字,逐个回材料原文找出处。
// 找不到的列为未核实事实(模型编造或来自模型自带的「常识」)。
func unverifiedFacts(artifactText, materialText string) []string {
if strings.TrimSpace(artifactText) == "" || strings.TrimSpace(materialText) == "" {
return nil
}
seen := map[string]bool{}
var out []string
check := func(tok string) {
tok = strings.TrimSpace(tok)
if tok == "" || seen[tok] {
return
}
seen[tok] = true
if !strings.Contains(materialText, tok) {
out = append(out, tok)
}
}
for _, m := range pctRe.FindAllString(artifactText, -1) {
check(m)
}
for _, m := range dateRe.FindAllString(artifactText, -1) {
check(m)
}
for _, m := range bigNumRe.FindAllString(artifactText, -1) {
if len(m) >= 5 { // 4 位数大量是年份/编号,只报 5 位以上
check(m)
}
}
return out
}
// unmarkedFacts 从未核实清单里剔除已按「目标/预计/假设」标注的(与后端
// runtime.MarkedAsEstimate 同一口径,标注词邻域 ±32 字符)。
func unmarkedFacts(tokens []string, artifactText string) []string {
var out []string
for _, tok := range tokens {
if !runtime.MarkedAsEstimate(artifactText, tok) {
out = append(out, tok)
}
}
return out
}
// finalVsArtifact 终轮回复里声称的章节(「一、…」)必须能在工件里找到。
func finalVsArtifact(turn *turnResult, artifactText string) []string {
if turn == nil || turn.FinalText == "" || strings.TrimSpace(artifactText) == "" {
return nil
}
final := turn.FinalText
var out []string
seen := map[string]bool{}
for _, m := range claimRe.FindAllStringSubmatch(final, -1) {
claim := strings.TrimSpace(m[1])
if claim == "" || seen[claim] {
continue
}
seen[claim] = true
if !strings.Contains(artifactText, claim) {
out = append(out, claim)
}
}
return out
}
// ---- 报告 ----
func writeReport(t *testing.T, rep *report) {
t.Helper()
outPath := os.Getenv("AGENT_EVAL_OUT")
if outPath == "" {
outPath = filepath.Join("eval-live", "report-"+time.Now().Format("0102-150405")+".json")
}
if err := os.MkdirAll(filepath.Dir(outPath), 0o755); err != nil {
t.Fatalf("mkdir report dir: %v", err)
}
raw, _ := json.MarshalIndent(rep, "", " ")
if err := os.WriteFile(outPath, raw, 0o644); err != nil {
t.Fatalf("write report: %v", err)
}
fmt.Fprintf(os.Stderr, "\n[live-eval] report: %s\n%s\n", outPath, renderMarkdown(rep))
}
func renderMarkdown(rep *report) string {
var b strings.Builder
fmt.Fprintf(&b, "# Live eval — %s\n\n- target: %s\n- model: %s\n- generated: %s\n\n", rep.GeneratedAt, rep.Target, rep.Model, rep.GeneratedAt)
for _, tk := range rep.Tasks {
fmt.Fprintf(&b, "## %s\n\n| turn | status | tools | fails | regens | api探查 | 读原件 | 计划 | 检查 | 工件 | 时长s |\n|---|---|---|---|---|---|---|---|---|---|---|\n", tk.ID)
for _, turn := range tk.Turns {
fmt.Fprintf(&b, "| %s | %s | %d | %d | %d | %d | %d | %d | %d | %d | %.0f |\n",
truncate(turn.User, 18), turn.Status, turn.Metrics.ToolCalls, turn.Metrics.ToolFailures,
turn.Metrics.FullRegens, turn.Metrics.APIDiscovery, turn.Metrics.ReadsOriginal,
turn.Metrics.PlanUpdates, turn.Metrics.VerifierChecks, len(turn.Artifacts),
float64(turn.DurationMs)/1000)
}
if len(tk.UnverifiedFacts) > 0 {
fmt.Fprintf(&b, "\n未核实事实(工件里出现、材料里找不到):%s\n", strings.Join(tk.UnverifiedFacts, "、"))
if len(tk.UnverifiedUnmarkedFacts) > 0 {
fmt.Fprintf(&b, "其中未标注(无出处且无目标/预计/假设标注,E3 闸门口径):%s\n", strings.Join(tk.UnverifiedUnmarkedFacts, "、"))
}
}
if len(tk.FinalVsArtifact) > 0 {
fmt.Fprintf(&b, "\n终轮声称但工件不存在的章节:%s\n", strings.Join(tk.FinalVsArtifact, ";"))
}
fmt.Fprint(&b, "\n")
}
return b.String()
}
+318
View File
@@ -0,0 +1,318 @@
//go:build live
// Package live 是真实后端 + 真实模型的任务级评测(Phase 0 基线)。
//
// 用法(对准一个已配置好模型的运行实例):
//
// AGENT_EVAL_BASE=http://127.0.0.1:18799 \
// AGENT_EVAL_MATERIALS=~/.agent-system/eval/materials \
// AGENT_EVAL_OUT=/tmp/before.json \
// go test ./eval/live/ -tags live -run TestLiveSuite -v -timeout 90m
//
// 评测器只用公开 HTTP 面(上传→建会话→发消息→轮询),指标从消息 parts、
// run usage 与工件内容计算;不触碰目标实例数据库。HITL 卡自动选第一个选项,
// 保证无人值守跑完。产出 JSON + Markdown 报告;任务集见 tasks.json。
// 真实材料不进仓:materials 目录按 eval/README 的口径放本机。
package live
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
type task struct {
ID string `json:"id"`
Material string `json:"material,omitempty"`
Turns []string `json:"turns"`
}
type part struct {
Type string `json:"type"`
Text string `json:"text"`
Process bool `json:"process"`
ToolCall *struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
Duration float64 `json:"durationMs"`
} `json:"toolCall"`
ToolResult *struct {
CallID string `json:"callId"`
Status string `json:"status"`
Summary string `json:"summary"`
} `json:"toolResult"`
Check *struct {
Passed bool `json:"passed"`
Reason string `json:"reason"`
} `json:"check"`
Artifact *struct {
ArtifactID string `json:"artifactId"`
Name string `json:"name"`
Kind string `json:"kind"`
} `json:"artifact"`
}
type message struct {
ID string `json:"id"`
RunID string `json:"runId"`
Role string `json:"role"`
Parts []part `json:"parts"`
Excluded bool `json:"excluded"`
}
type run struct {
ID string `json:"id"`
Status string `json:"status"`
CreatedAt string `json:"createdAt"`
EndedAt string `json:"endedAt"`
Usage *struct {
PromptTokens int `json:"promptTokens"`
CompletionTokens int `json:"completionTokens"`
TotalTokens int `json:"totalTokens"`
DurationMs int `json:"durationMs"`
ToolCalls int `json:"toolCalls"`
} `json:"usage"`
Error any `json:"error"`
}
type artifactMeta struct {
ID string `json:"id"`
Name string `json:"name"`
Kind string `json:"kind"`
MIME string `json:"mime"`
SourceRun string `json:"sourceRun"`
CreatedAt string `json:"createdAt"`
}
type hitlCard struct {
ID string `json:"id"`
Question string `json:"question"`
Choices []struct {
ID string `json:"id"`
Label string `json:"label"`
} `json:"choices"`
AllowFreeInput bool `json:"allowFreeInput"`
}
type client struct {
base string
http *http.Client
t *testing.T
model string
}
func (c *client) do(method, path string, body any, out any) error {
var rd io.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return err
}
rd = bytes.NewReader(b)
}
req, err := http.NewRequest(method, c.base+path, rd)
if err != nil {
return err
}
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
raw, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 300 {
return fmt.Errorf("%s %s: %s: %s", method, path, resp.Status, truncate(string(raw), 300))
}
if out != nil {
return json.Unmarshal(raw, out)
}
return nil
}
func truncate(s string, n int) string {
r := []rune(s)
if len(r) <= n {
return s
}
return string(r[:n]) + "…"
}
func TestLiveSuite(t *testing.T) {
base := strings.TrimSpace(os.Getenv("AGENT_EVAL_BASE"))
if base == "" {
t.Skip("AGENT_EVAL_BASE not set; live eval requires a configured backend instance")
}
materialsDir := os.Getenv("AGENT_EVAL_MATERIALS")
if materialsDir == "" {
home, _ := os.UserHomeDir()
materialsDir = filepath.Join(home, ".agent-system", "eval", "materials")
}
if _, err := os.Stat(materialsDir); err != nil {
t.Skipf("materials dir not available: %v", err)
}
raw, err := os.ReadFile("tasks.json")
if err != nil {
t.Fatalf("read tasks.json: %v", err)
}
var tasks []task
if err := json.Unmarshal(raw, &tasks); err != nil {
t.Fatalf("parse tasks.json: %v", err)
}
if filter := os.Getenv("AGENT_EVAL_TASKS"); filter != "" {
want := map[string]bool{}
for _, id := range strings.Split(filter, ",") {
want[strings.TrimSpace(id)] = true
}
var kept []task
for _, tk := range tasks {
if want[tk.ID] {
kept = append(kept, tk)
}
}
tasks = kept
}
if len(tasks) == 0 {
t.Fatal("no tasks selected")
}
turnTimeout := 20 * time.Minute
if v := os.Getenv("AGENT_EVAL_TURN_TIMEOUT"); v != "" {
if d, err := time.ParseDuration(v); err == nil {
turnTimeout = d
}
}
c := &client{base: base, http: &http.Client{Timeout: 120 * time.Second}, t: t}
// /api/settings 的形状是 {defaults, project, schema, user};模型播在 user 作用域。
var settings struct {
User struct {
Models struct {
Chat struct {
Model string `json:"model"`
} `json:"chat"`
} `json:"models"`
} `json:"user"`
}
_ = c.do("GET", "/api/settings", nil, &settings)
c.model = settings.User.Models.Chat.Model
proj := struct {
ID string `json:"id"`
}{}
if err := c.do("POST", "/api/projects", map[string]string{"name": "eval-" + time.Now().Format("0102-150405")}, &proj); err != nil {
t.Fatalf("create project: %v", err)
}
rep := report{Target: base, Model: c.model, GeneratedAt: time.Now().UTC().Format(time.RFC3339)}
for _, tk := range tasks {
rep.Tasks = append(rep.Tasks, runTask(t, c, proj.ID, tk, materialsDir, turnTimeout))
}
writeReport(t, &rep)
}
type turnResult struct {
User string `json:"user"`
RunID string `json:"runId"`
Status string `json:"status"`
FinalText string `json:"finalText,omitempty"`
Usage *run `json:"usage,omitempty"`
Metrics turnMetrics `json:"metrics"`
Checks []checkRecord `json:"checks,omitempty"`
Artifacts []string `json:"artifacts,omitempty"`
Issues []string `json:"issues,omitempty"`
DurationMs int64 `json:"durationMs"`
}
type turnMetrics struct {
ToolCalls int `json:"toolCalls"`
ToolFailures int `json:"toolFailures"`
FullRegens int `json:"fullRegens"`
APIDiscovery int `json:"apiDiscovery"`
ReadsOriginal int `json:"readsOriginal"`
PlanUpdates int `json:"planUpdates"`
VerifierChecks int `json:"verifierChecks"`
}
type checkRecord struct {
Passed bool `json:"passed"`
Reason string `json:"reason"`
}
type taskResult struct {
ID string `json:"id"`
Turns []turnResult `json:"turns"`
Total turnMetrics `json:"total"`
UnverifiedFacts []string `json:"unverifiedFacts,omitempty"`
// UnverifiedUnmarkedFacts:无出处且未按「目标/预计/假设」标注的子集
// (E3 口径:严格口径 unverifiedFacts 保留原义以维持历史可比性)。
UnverifiedUnmarkedFacts []string `json:"unverifiedUnmarkedFacts,omitempty"`
FinalVsArtifact []string `json:"finalVsArtifact,omitempty"`
}
type report struct {
Target string `json:"target"`
Model string `json:"model,omitempty"`
GeneratedAt string `json:"generatedAt"`
Tasks []taskResult `json:"tasks"`
}
func runTask(t *testing.T, c *client, projectID string, tk task, materialsDir string, turnTimeout time.Duration) taskResult {
t.Helper()
conv := struct {
ID string `json:"id"`
}{}
if err := c.do("POST", "/api/projects/"+projectID+"/conversations", map[string]string{"title": "eval-" + tk.ID}, &conv); err != nil {
t.Fatalf("[%s] create conversation: %v", tk.ID, err)
}
var attachmentIDs []string
materialText := ""
if tk.Material != "" {
path := filepath.Join(materialsDir, tk.Material)
raw, err := os.ReadFile(path)
if err != nil {
t.Fatalf("[%s] open material: %v", tk.ID, err)
}
up := c.upload(t, projectID, tk.Material, raw)
attachmentIDs = append(attachmentIDs, up)
materialText = docxTextIfDocx(t, tk.Material, raw)
}
tr := taskResult{ID: tk.ID}
for i, turnText := range tk.Turns {
tr.Turns = append(tr.Turns, runTurn(t, c, projectID, conv.ID, turnText, attachmentIDs, turnTimeout))
attachmentIDs = nil // 材料只在第一轮附带;后续轮考验跨轮上下文
_ = i
}
// 汇总
for _, turn := range tr.Turns {
tr.Total.ToolCalls += turn.Metrics.ToolCalls
tr.Total.ToolFailures += turn.Metrics.ToolFailures
tr.Total.FullRegens += turn.Metrics.FullRegens
tr.Total.APIDiscovery += turn.Metrics.APIDiscovery
tr.Total.ReadsOriginal += turn.Metrics.ReadsOriginal
tr.Total.PlanUpdates += turn.Metrics.PlanUpdates
tr.Total.VerifierChecks += turn.Metrics.VerifierChecks
}
// 事实核查:最后一个带工件的回合 vs 材料
if materialText != "" {
last := lastArtifactTurn(tr)
if last != nil && len(last.Artifacts) > 0 {
artText := ""
for _, name := range last.Artifacts {
artText += downloadArtifactText(t, c, projectID, name)
}
tr.UnverifiedFacts = unverifiedFacts(artText, materialText)
tr.UnverifiedUnmarkedFacts = unmarkedFacts(tr.UnverifiedFacts, artText)
tr.FinalVsArtifact = finalVsArtifact(last, artText)
}
}
return tr
}
+26
View File
@@ -0,0 +1,26 @@
[
{
"id": "minutes-basic",
"material": "meeting-rd-agent-asr.docx",
"turns": ["帮我总结一份会议纪要"]
},
{
"id": "plan-followup",
"material": "meeting-rd-agent-asr.docx",
"turns": [
"帮我总结一份会议纪要",
"根据这个纪要,制定一个方案",
"方案太简要了,给我一个详细的方案和计划"
]
},
{
"id": "qa-citation",
"material": "meeting-rd-agent-asr.docx",
"turns": ["这次会议最主要的事情是什么"]
},
{
"id": "minutes-pm-tool",
"material": "meeting-pm-tool-asr.docx",
"turns": ["生成会议纪要"]
}
]
+108
View File
@@ -0,0 +1,108 @@
package app
// EXT-19 消费测试:inprocHost.ListAccounts——最小目录(owner/valid)、游标分页、
// limit 钳制(1-200,缺省 50)。数据来自 users 表(真实 store,非桩)。
import (
"testing"
"agent-system-backend/internal/store"
)
func TestInprocHostListAccounts(t *testing.T) {
dir := t.TempDir()
st, err := store.OpenSQLite(dir + "/test.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
for _, o := range []string{"carol", "alice", "bob", "dave"} {
if _, err := st.CreateAPIKey(o, "e2e"); err != nil {
t.Fatal(err)
}
}
h := inprocHost{caps: hostcaps{st: st}}
// 首页(limit=2):alice/bob,next=bob。
page, next, err := h.ListAccounts("", 2)
if err != nil {
t.Fatal(err)
}
if len(page) != 2 || page[0].Owner != "alice" || page[1].Owner != "bob" {
t.Fatalf("首页不符: %+v", page)
}
for _, e := range page {
if !e.Valid {
t.Fatalf("现有账号应 valid: %+v", e)
}
}
if next != "bob" {
t.Fatalf("next 应为 bob, got %q", next)
}
// 次页:carol/dave,next=""(末页)。
page2, next2, _ := h.ListAccounts(next, 2)
if len(page2) != 2 || page2[0].Owner != "carol" || page2[1].Owner != "dave" || next2 != "" {
t.Fatalf("次页不符: %+v next=%q", page2, next2)
}
// limit 钳制:0 → 50 缺省;超 200 → 200(不炸)。
if p, _, _ := h.ListAccounts("", 0); len(p) != 4 {
t.Fatalf("limit=0 应取缺省(4 账号全量), got %d", len(p))
}
if _, _, err := h.ListAccounts("", 999); err != nil {
t.Fatalf("limit 超限应钳制不报错: %v", err)
}
}
// TestInprocHostSearchAccounts:EXT-19 扩展 SearchAccounts——账号/姓名子串
// 检索、大小写不敏感、空 query 首页、limit 钳制(P3,2026-10-13)。
func TestInprocHostSearchAccounts(t *testing.T) {
dir := t.TempDir()
st, err := store.OpenSQLite(dir + "/test.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
hash, err := store.HashPassword("init-pass-01")
if err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("alice", "王小明", hash, false); err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("bob", "李大明", hash, false); err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("carol", "", hash, false); err != nil {
t.Fatal(err)
}
h := inprocHost{caps: hostcaps{st: st}}
// 姓名子串(同名区分场景):两个"大明/小明"都以"明"命中。
got, err := h.SearchAccounts("明", 50)
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[0].Owner != "alice" || got[1].Owner != "bob" {
t.Fatalf("姓名子串命中不符: %+v", got)
}
if got[0].Name != "王小明" {
t.Fatalf("目录行应携带姓名: %+v", got[0])
}
// 账号子串、大小写不敏感。
got, _ = h.SearchAccounts("AL", 50)
if len(got) != 1 || got[0].Owner != "alice" {
t.Fatalf("账号子串应命中 alice: %+v", got)
}
// 未命中。
if got, _ = h.SearchAccounts("不存在", 50); len(got) != 0 {
t.Fatalf("未命中应为空: %+v", got)
}
// 空 query=首页候选(owner 升序);姓名为空的账号 Name 为空串。
if got, _ = h.SearchAccounts("", 2); len(got) != 2 || got[0].Owner != "alice" || got[1].Name != "李大明" {
t.Fatalf("空 query 首页不符: %+v", got)
}
// ListAccounts 回归:分页行同样携带姓名。
page, _, _ := h.ListAccounts("", 2)
if page[0].Name != "王小明" {
t.Fatalf("ListAccounts 应携带姓名: %+v", page[0])
}
}
+479 -55
View File
@@ -30,12 +30,15 @@ import (
"agent-native/runtime/telemetry"
agenttools "agent-native/runtime/tools"
nativeadp "agent-squad/adapters/native"
"agent-system-backend/internal/blobstore"
"agent-system-backend/internal/bus"
"agent-system-backend/internal/cardsvc"
"agent-system-backend/internal/cmdbus"
"agent-system-backend/internal/config"
"agent-system-backend/internal/genui"
"agent-system-backend/internal/hostapi"
"agent-system-backend/internal/intentgate"
"agent-system-backend/internal/intentsem"
"agent-system-backend/internal/pgbus"
"agent-system-backend/internal/plugins"
"agent-system-backend/internal/profiles"
@@ -49,8 +52,10 @@ import (
"agent-system-backend/internal/skills"
"agent-system-backend/internal/squadadapter"
"agent-system-backend/internal/store"
"agent-system-backend/internal/surfacesvc"
"agent-system-backend/internal/tools"
"agent-system-backend/internal/workflow"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
@@ -65,6 +70,78 @@ type Options struct {
InstanceID string // 空 = 主机名:监听地址;写入 runs.instance_id,启动对账只收口本实例
InsecureListen bool
AllowedHosts []string
// DisableUploadIndexing EXT-9:true 时上传端点不自动写检索索引(发布决定
// 移交下游命令层);零值=false=现状。由 link 层自 extension.Options 透传。
DisableUploadIndexing bool
// DisableConvCardMirror EXT-12:true 时会话卡不镜像进项目公共流(不发布
// 即不持久,重放不含会话卡);零值=false=现状全量镜像。
DisableConvCardMirror bool
// DisableExecTools 启动级硬关闭执行原语(run_shell / run_python /
// register_artifact 与小队内核执行配置):用户设置 runtime.exec 无法覆盖;
// 零值=false=现状。
DisableExecTools bool
// ExecBackend 执行后端(EXT-16):空/direct=现状直启;sandbox=bwrap 隔离档。
// 硬关闭 DisableExecTools 优先级高于后端选择。
ExecBackend string
// ExecDepsDir sandbox 档只读依赖层(site-packages 等,空=不挂):修复
// --tmpfs /home 遮蔽 ~/.local 导致交付物生成链路不可用(F3)。
ExecDepsDir string
// ExecWorkspace = "overlay" 时 sandbox 档启用 S0 工作区(冻结基底+私有
// upper+CAS 回收);空 = EXT-16 现状直写会话目录。
ExecWorkspace string
// Tier 部署档(协作存储 KTD1):空=auto(DSN→PG 否则 SQLite);local=SQLite;
// team=PG fail-closed(缺 DSN/未认证拒绝启动;PG 故障拒绝就绪,不回落 SQLite)。
Tier string
// PGPool PG 连接预算(KTD12;零值走默认)。仅 PG 档生效。
PGPool store.PoolOptions
}
// instanceTag 稳定执行身份:tier=team 时强制 host:addr(多实例互异),
// 本地档退化为 local(单实例语义)。
func instanceTag(opt Options) string {
if opt.Tier == "team" || opt.StorageDSN != "" {
return hostTag() + ":" + opt.Addr
}
return "local"
}
func hostTag() string {
h, err := os.Hostname()
if err != nil || h == "" {
return "host"
}
return h
}
// tierName 输出部署档名(诊断统一拼写:空 = auto)。
func tierName(opt Options) string {
if opt.Tier == "" {
return "auto"
}
return opt.Tier
}
// TierRefusal 部署档门禁(KTD1):返回阻止启动的问题(空 = 放行)。
// team 档 fail-closed:缺 DSN、未开认证拒绝;PG 故障的拒绝发生在其后的
// OpenPostgres(Run 对 team 档的错误显式声明不回落 SQLite)。
func TierRefusal(opt Options) []string {
return config.Resolved{Config: config.Config{
Addr: opt.Addr, AuthMode: opt.AuthMode, StorageDSN: opt.StorageDSN, Tier: opt.Tier,
PGMaxOpen: opt.PGPool.MaxOpenConn, PGMaxIdle: opt.PGPool.MaxIdleConn,
PGConnMaxIdleSec: opt.PGPool.ConnMaxIdleTimeSec, PGConnMaxLifetimeSec: opt.PGPool.ConnMaxLifetimeSec,
PGQueryTimeoutSec: opt.PGPool.QueryTimeoutSec,
}}.TierProblems()
}
// latestVersion 返回已应用迁移的最高版本(0 = 空账本)。
func latestVersion(versions []int) int {
max := 0
for _, v := range versions {
if v > max {
max = v
}
}
return max
}
// Run 装配并阻塞服务直至 SIGINT/SIGTERM(优雅停机:中断在途 run、冲净事件缓冲)。
@@ -73,17 +150,28 @@ func Run(opt Options) error {
Service: "agent-system", Module: "app", Level: logsys.LevelInfo,
Sinks: []logsys.Sink{&sink.Console{Out: os.Stderr, Format: "json"}},
})
logger.Info("agent-system starting", "addr", opt.Addr, "data", opt.DataDir)
logger.Info("agent-system starting", "addr", opt.Addr, "data", opt.DataDir, "tier", tierName(opt))
resolved := config.Resolved{Config: config.Config{
Addr: opt.Addr, AuthMode: opt.AuthMode, StorageDSN: opt.StorageDSN, InsecureListen: opt.InsecureListen,
Tier: opt.Tier, PGMaxOpen: opt.PGPool.MaxOpenConn, PGMaxIdle: opt.PGPool.MaxIdleConn,
PGConnMaxIdleSec: opt.PGPool.ConnMaxIdleTimeSec, PGConnMaxLifetimeSec: opt.PGPool.ConnMaxLifetimeSec,
PGQueryTimeoutSec: opt.PGPool.QueryTimeoutSec,
}}
if msg := resolved.ListenRefusal(); msg != "" {
return fmt.Errorf("listen guard: %s", msg)
}
// 部署档门禁(KTD1 fail-closed):team 档缺 DSN/未开认证直接拒绝启动;
// local 档带 DSN 是配置错误。此处与 config.TierProblems 同规则(纵深防御,
// 服务入口不依赖 CLI 是否校验)。
if msgs := TierRefusal(opt); len(msgs) > 0 {
return fmt.Errorf("tier guard: %s", strings.Join(msgs, ";"))
}
if opt.Tier == "team" && !opt.DisableExecTools && opt.ExecBackend == "" {
log.Printf("warning: team 档未配置执行隔离(disableExecTools 或 execBackend);向真实多用户开放前必须补齐(方案 KTD1)")
}
if opt.InsecureListen && !config.IsLoopbackListen(opt.Addr) && !resolved.AuthEnabled() {
log.Printf("warning: listening on %s without authentication because insecureListen is set", opt.Addr)
}
traceFn := telemetry.StartFunc(logger)
var st *store.Store
var err error
if opt.StorageDSN != "" {
@@ -91,7 +179,20 @@ func Run(opt Options) error {
if artifacts == "" {
artifacts = filepath.Join(opt.DataDir, "artifacts")
}
st, err = store.OpenPostgres(opt.StorageDSN, artifacts)
// 共享工件(U5/KTD9):配置了 S3 兼容对象存储时注入(多实例共享);
// 未配置回落本地目录(单实例/共享卷)。凭据只经环境(不入库不入报告)。
blobs, berr := blobstore.NewS3FromEnv()
if berr != nil {
return fmt.Errorf("blobstore s3: %w", berr)
}
if blobs != nil {
log.Printf("artifacts: shared via S3-compatible object storage (prefix %q)", blobs.Prefix)
}
st, err = store.OpenPostgresWithOptions(opt.StorageDSN, artifacts, opt.PGPool, blobs)
if err != nil && opt.Tier == "team" {
// team 档拒绝就绪(R2):PG 故障不回落 SQLite,错误必须明说。
return fmt.Errorf("team 档打开 PostgreSQL 失败,拒绝就绪(不回落 SQLite): %w", err)
}
} else {
st, err = store.OpenSQLite(filepath.Join(opt.DataDir, "agent-system.db"))
}
@@ -99,7 +200,30 @@ func Run(opt Options) error {
return fmt.Errorf("open store: %w", err)
}
defer st.Close()
log.Printf("storage tier: %s", st.Dialect())
// U1 I3:svc 注入的服务实例身份(manifest.instance_id)与库内 marker 对齐;
// 未设置则跳过(兼容未接线旧调用方)。与 --instance(runs 进程标识)无关。
if sysID := strings.TrimSpace(os.Getenv("AGENT_SYSTEM_INSTANCE_ID")); sysID != "" {
dbref := strings.TrimSpace(os.Getenv("AGENT_SYSTEM_DBREF"))
if dbref == "" {
if opt.StorageDSN != "" {
dbref = "pg"
} else {
dbref = "sqlite"
}
}
if err := st.VerifySystemIdentity(sysID, dbref); err != nil {
return fmt.Errorf("system identity: %w", err)
}
}
pool := st.Pool()
if versions, verr := st.MigrationStatus(); verr == nil {
log.Printf("storage tier: %s (dialect=%s pool=%d/%d idleSec=%d lifetimeSec=%d schema=v%d)",
tierName(opt), st.Dialect(), pool.MaxOpenConn, pool.MaxIdleConn, pool.ConnMaxIdleTimeSec,
pool.ConnMaxLifetimeSec, latestVersion(versions))
} else {
log.Printf("storage tier: %s (dialect=%s pool=%d/%d schema=? migration status unavailable: %v)",
tierName(opt), st.Dialect(), pool.MaxOpenConn, pool.MaxIdleConn, verr)
}
b := bus.New()
reg := tools.NewRegistry()
@@ -112,7 +236,33 @@ func Run(opt Options) error {
}
// 执行原语(agent-native 交付):shell/python 在会话工作区执行 + 产物登记桥,
// runtime.exec=false 可整体禁用(fail-closed)。生成文档/表格/演示稿的方法
// 在提示词库 base.deliverables,不在系统工具里。
// 在提示词库 base.deliverables,不在系统工具里。启动参数 DisableExecTools
// 为不受用户设置影响的硬关闭(多用户部署的默认安全档)。
tools.SetExecDisabled(opt.DisableExecTools)
if opt.DisableExecTools {
log.Printf("exec tools: disabled by startup option (runtime.exec settings are ignored)")
}
// EXT-16:执行后端选择(direct|sandbox);未知名字启动即失败(fail-closed)。
if err := tools.SetExecBackend(opt.ExecBackend); err != nil {
return fmt.Errorf("exec backend: %w", err)
}
if tools.ExecBackendName() == "sandbox" {
log.Printf("exec tools: backend=sandbox (bwrap: no network, ro root, session-only writes, proc limit)")
}
if opt.ExecDepsDir != "" {
tools.SetExecDepsDir(opt.ExecDepsDir)
log.Printf("exec tools: sandbox deps layer=%s", opt.ExecDepsDir)
}
if opt.ExecWorkspace == "overlay" {
tools.SetWorkspaceOverlay(true)
if tools.ExecBackendName() == "sandbox" {
log.Printf("exec tools: workspace=overlay (frozen base + private upper + CAS apply)")
} else {
log.Printf("exec tools: workspace=overlay ignored (backend is not sandbox)")
}
} else if opt.ExecWorkspace != "" {
return fmt.Errorf("exec workspace: unknown mode %q (overlay)", opt.ExecWorkspace)
}
if err := tools.RegisterExecTools(reg); err != nil {
return fmt.Errorf("exec tools: %w", err)
}
@@ -123,6 +273,10 @@ func Run(opt Options) error {
return fmt.Errorf("code orchestration tool: %w", err)
}
setSvc := settings.NewService(st)
effBoot := setSvc.Effective(store.OwnerDefault, "")
traceFn := telemetry.StartFunc(logger,
telemetry.WithServiceName(effBoot.GetString("telemetry.serviceName", "agent-system-backend")),
telemetry.WithEndpoint(effBoot.GetString("telemetry.otlpEndpoint", "")))
setSvc.SetSystemOverride(map[string]any{"storage": map[string]any{"dataDir": opt.DataDir}})
// 点路径键展开必须在读 tools.tavily.apiKeys 之前,否则启动仍注册旧池。
norm, err := setSvc.NormalizeStored()
@@ -142,23 +296,32 @@ func Run(opt Options) error {
if owner == "" {
owner = store.OwnerDefault
}
return st.ActiveSecret(owner, service)
if v, ok := st.ActiveSecret(owner, service); ok {
return v, true
}
// P2 受管:密钥统一迁入 secrets 表后,系统层受管密钥挂在系统行
//(owner=settings.OwnerSystemRow);租户未单独配置时回退系统密钥。
return st.ActiveSecret(settings.OwnerSystemRow, service)
}
rag.SecretLookup = func(service, fallback string) string {
return runtime.LookupSecret(store.OwnerDefault, service, fallback)
}
// 网络搜索(Tavily):配置密钥池才注册(对齐 agent-native 语义:未配置不静默降级)。
eff := setSvc.Effective(store.OwnerDefault, "")
tavilyRaw := eff.GetString("tools.tavily.apiKeys", "")
if active, ok := st.ActiveSecret(store.OwnerDefault, "tools.tavily"); ok {
tavilyRaw = active
}
if keys := splitKeyPool(tavilyRaw); len(keys) > 0 {
provider := agenttools.NewTavilyProvider(eff.GetString("tools.tavily.endpoint", ""), keys)
if err := tools.RegisterSearchTools(reg, provider); err != nil {
return fmt.Errorf("search tools: %w", err)
// 网络搜索(Tavily):始终注册,密钥按调用时读取(设置页/svc 播种改了
// 即刻生效——此前启动时快照密钥,播种后必须重启工具才注册)。
// 未配置时调用返回明确的「未配置」提示,不静默降级为空结果。
if err := tools.RegisterLazySearchTools(reg, func() agenttools.SearchProvider {
effNow := setSvc.Effective(store.OwnerDefault, "")
tavilyNow := effNow.GetString("tools.tavily.apiKeys", "")
if active, ok := st.ActiveSecret(store.OwnerDefault, "tools.tavily"); ok {
tavilyNow = active
}
log.Printf("tools: tavily_search/web_search registered (%d key(s))", len(keys))
keys := splitKeyPool(tavilyNow)
if len(keys) == 0 {
return nil
}
return agenttools.NewTavilyProvider(effNow.GetString("tools.tavily.endpoint", ""), keys)
}); err != nil {
return fmt.Errorf("search tools: %w", err)
}
promptLib, err := prompts.DefaultLibrary()
if err != nil {
@@ -255,6 +418,9 @@ func Run(opt Options) error {
}
}
cfg.Trace = traceFn
// N1 WP4:小队成员的内核 ask 决策经 GateOptions.PermissionAsk 到达
// runCtx 的 HITL 确认卡(宿主无实现则 nil,内核保持 ask=拒绝旧行为)。
cfg.PermissionAsk = runtime.PermissionAskOf(rc)
if host, ok := rc.(interface {
OpenCheckpoints() (*checkpoint.Log, error)
CheckpointAnchor() string
@@ -272,7 +438,10 @@ func Run(opt Options) error {
return fmt.Errorf("default profile: %w", err)
}
if opt.ForceMock {
if err := setSvc.MergeUser(store.OwnerDefault, map[string]any{"runtime": map[string]any{"mock": true}}); err != nil {
// runtime.* 自 #188 起为系统受管项:启动选项属受信操作员输入,
// 播种走系统层(对所有租户生效);用户层写入会被受管拦截拒绝,
// 导致 serve --mock 直接启动失败(multi-instance 集成回归)。
if err := setSvc.MergeSystem(map[string]any{"runtime": map[string]any{"mock": true}}); err != nil {
return err
}
}
@@ -294,6 +463,8 @@ func Run(opt Options) error {
if err := mgr.ReconcileInterrupted(); err != nil {
return fmt.Errorf("reconcile interrupted runs: %w", err)
}
// 后台子代理通知重启重投(N2 WP5):未投递结果装回队列并按空闲语义投递。
mgr.LoadPendingNotices()
mgr.SetSquadRunner(&runtime.SquadRunner{})
// 意图闸门:插件意图 overlay → agent-intent 注册表(未装载意图 = 行为回退)。
@@ -324,29 +495,131 @@ func Run(opt Options) error {
logger.Error("plugin load failed", perr, "plugin", id, "code", string(errcode.SysUnavailable))
}
}
// P2 受管:插件声明里带 scope=system 的设置项同样纳入受管路径
//(user/project 写入拒绝、生效链剥除)。
setSvc.SetExtraSchema(pluginHost.SettingsSchema())
// 检索管线(4.12):分块+全文+(配置时)向量混合与 rerank;未配置自动退化为全文。
pipe := rag.NewPipeline(st, setSvc, nil, nil)
pipe.SetEmbedder(rag.SwitchingEmbedder{HTTP: rag.NewHTTPEmbedder(pipe.EmbeddingRoute), Provider: func() string {
embedder := rag.SwitchingEmbedder{HTTP: rag.NewHTTPEmbedder(pipe.EmbeddingRoute), Provider: func() string {
return setSvc.Effective(store.OwnerDefault, "").GetString("models.embedding.provider", "")
}})
}}
pipe.SetEmbedder(embedder)
pipe.SetReranker(rag.NewHTTPReranker(pipe.RerankRoute))
mgr.SetRetriever(pipe)
mgr.SetIndexer(pipe)
// 意图慢通道(I2):语义向量裁决 + 轻量模型命令生成。
// 两个开关都默认关——此时识别行为与 I1 完全一致(fail-closed 分层);
// 开启语义需已配置 embedding 端点,开启生成需可用的轻量/主模型档。
if setSvc.Effective(store.OwnerDefault, "").GetBool("runtime.intentSemantic", false) {
gate.SetSemantic(intentsem.EmbeddingScorer{Provider: embedder}, 0)
}
if setSvc.Effective(store.OwnerDefault, "").GetBool("runtime.intentGenerator", false) {
effNow := setSvc.Effective(store.OwnerDefault, "")
light := runtime.ResolveLightweightModel(core.AgentProfile{}, effNow)
client := &agentllm.Client{BaseURL: light.Base, APIKey: light.Key, HTTPClient: &http.Client{}}
gate.SetGenerator(intentsem.LightGenerator{Stream: client.Stream, Model: light.Model})
}
// fs 预留能力沙箱根:<DataDir>/workspaces/<owner>/<conversationID>/。
mgr.SetWorkspaceRoot(filepath.Join(opt.DataDir, "workspaces"))
// 中断子代理续跑(N2 WP5):依赖转录目录(workspaceRoot)与派生请求。
mgr.ResumeInterruptedChildren()
// 收件箱任务卡:事件走会话事件管道(持久化/seq/重放);关闭即 T0 基线。
cardSvc := cardsvc.New(st, mgr.EmitCardEvent)
cardSvc.SetProjectSharedEmitter(mgr.EmitProjectSharedCardEvent)
surfaceSvc := surfacesvc.New(st, mgr.EmitCardEvent)
go func() { // 启动恢复:重发「快照已落盘、事件未发出」的待恢复投影(KTD)。
if n, err := surfaceSvc.RecoverPendingEvents(context.Background()); err == nil && n > 0 {
log.Printf("surface: recovered %d pending events", n)
}
}()
mgr.SetCards(cardSvc)
// 执行身份与命令消费(协作存储 U5):PG 档启动跨实例命令轮询;
// SQLite 本地档无跨实例命令(StartCommandConsumer 自行短路)。
mgr.SetInstanceTag(instanceTag(opt), opt.Addr)
stopCommands := mgr.StartCommandConsumer(context.Background())
defer stopCommands()
// 执行预算(协作存储 U3,R8):runtime.execBudgetGlobal / runtime.execBudgetPerUser
// 全零=不限(本地档默认,行为不变);开启后执行许可共享账本生效,
// 待执行队列上限 runtime.execQueueMax(默认 200,容量合同数值待 U7 实测调整)。
mgr.SetExecutionBudgetFn(func() store.ExecutionBudget {
eff := setSvc.Effective(store.OwnerDefault, "")
return store.ExecutionBudget{
Global: int(eff.GetNumber("runtime.execBudgetGlobal", 0)),
PerUser: int(eff.GetNumber("runtime.execBudgetPerUser", 0)),
}
}, int(setSvc.Effective(store.OwnerDefault, "").GetNumber("runtime.execQueueMax", 200)))
// 命令总线(提案 T3/D27/D30):四入口同源分发 + 审计 + 危险内联确认卡。
bus := cmdbus.New(st, cardSvc, func() time.Time { return time.Now().UTC() })
// U5:页面动作分发口——surfacesvc.SubmitAction 经命令总线四入口同源分发
//(审计信封、参数校验、DangerHigh 内联确认卡全复用;Confirmed 不透传)。
surfaceSvc.SetDispatcher(func(ctx context.Context, d surfacesvc.ActionDispatch) (surfacesvc.ActionOutcome, error) {
resp := bus.Dispatch(ctx, cmdbus.Request{ID: d.CommandID, Params: d.Params,
Entrance: "button", Owner: d.Owner, ConversationID: d.ConversationID})
state := "committed"
if resp.Receipt.State != "" {
state = string(resp.Receipt.State)
}
cardID := ""
if resp.Card != nil {
cardID = string(resp.Card.ID)
}
return surfacesvc.ActionOutcome{State: state, CardID: cardID, Receipt: resp.Receipt, Payload: resp.Payload}, nil
})
// U3:generative-ui 工具注入闸——activation 台账关闭时命令不投影给该 owner
// 的模型(禁用即不注入能力;执行端守卫仍在 surfacesvc)。
bus.SetToolGate(func(owner, commandID string) bool {
if !strings.HasPrefix(commandID, genui.PluginID+".") {
return true
}
row, err := surfaceSvc.Activation(context.Background(), owner, genui.PluginID)
return err == nil && row.Enabled
})
// 自动建议页面 opt-in:settings ∧ activation;预算/冷却在 genui.Auto。
mgr.SetGenUIAuto(genui.NewAuto(
func(owner string) bool {
row, err := surfaceSvc.Activation(context.Background(), owner, genui.PluginID)
return err == nil && row.Enabled
},
func(owner string) bool {
return setSvc.Effective(owner, "").GetBool(genui.SettingAutoPresent, false)
},
func(owner, kind, detail string) {
_ = st.AppendAuditEvent(store.AuditEvent{Owner: owner, Action: kind, Evidence: detail})
},
))
bus.SetPendingStore(st)
pluginHost.OnCommands = bus.RegisterPluginCommands
// hostcaps:hostapi 闭包与进程内插件句柄共用的 owner 作用域能力(EXT-1 批注 1)。
caps := hostcaps{st: st, pipe: pipe, set: setSvc}
applyCompiled(skillsRoot, bus, caps)
authzSlot := &projectAuthz{}
projectSlot := &projectLifecycleSlot{}
caps := hostcaps{projects: projectSlot, st: st, pipe: pipe, set: setSvc, authz: authzSlot, surfaces: surfaceSvc}
// EXT-13:回合内检索(knowledge_search / @knowledge 注入)与 HTTP/Host 检索面
// 同一适配器授权;槽为共享指针,扩展在 applyCompiled 期间注册后即生效。
mgr.SetSharedKnowledgeAllow(func(owner string, projectID core.ID) bool {
ok, err := authzSlot.Authorize(owner, projectID)
return err == nil && ok
})
// EXT-14:命令即工具——带 exposeAsTool 的命令投影进回合工具清单,模型调用
// 经命令总分发(Entrance=agent,同审计信封与高危内联确认卡)。
mgr.SetCommandSource(busAdapter{bus: bus})
// EXT-11:公开受限 Host 工厂(下游恢复路径经 extension.NewScopedHost 重建句柄)。
wireScopedHostFactory(caps)
compiledSummary := applyCompiled(skillsRoot, bus, caps)
mgr.SetProjectWorkAuthorization(projectSlot.work())
mgr.SetKnowledgeProjectScope(func(ctx context.Context, actor string, id core.ID) (core.ID, error) {
return projectSlot.resolve(ctx, caps, actor, id)
})
if authzSlot.hasV2() {
mgr.SetKnowledgeResourceAllow(func(actor string, d core.KnowledgeDoc) bool {
ok, err := authzSlot.AuthorizeResource(extension.ResourceAuthzQuery{Actor: actor, Action: "model-context", ResourceType: "knowledge", ResourceID: d.ID, Owner: d.Owner, ProjectID: d.ProjectID, Source: d.Source})
return err == nil && ok
})
}
mgr.OnUsage = func(owner string, tokens int) {
budget := int(setSvc.Effective(owner, "").GetNumber("llm.dailyTokenBudget", 0))
status, aerr := st.AccountTokens(owner, tokens, budget)
@@ -369,14 +642,34 @@ func Run(opt Options) error {
hostSrv.Search = func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error) {
return caps.searchDefault(ctx, owner, text)
}
// EXT-8 余项:hostapi 检索面同样吃适配器共享(本人 + 放行的他人项目知识)。
hostSrv.SharedSearch = func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error) {
return caps.searchDefault(ctx, owner, text)
}
hostSrv.Complete = func(ctx context.Context, owner, prompt string) (string, error) {
return caps.complete(ctx, owner, prompt)
}
srv := server.New(st, mgr, reg, profSvc, setSvc, pluginHost)
projectSlot.wire(srv, caps)
// EXT-8:装载期注册的资源授权适配器注入读取入口(未注册=默认拒绝,行为不变)。
srv.SetProjectAuthorizer(authzSlot.Authorize)
// EXT-8b:资源级适配器注入读面(未注册=server 回落 V1,行为不变)。
if authzSlot.hasV2() {
srv.SetResourceAuthorizerV2(authzSlot.AuthorizeResource)
}
// EXT-9:宿主可关闭上传自动索引(发布决定移交下游命令层)。
srv.DisableUploadIndexing = opt.DisableUploadIndexing
// EXT-12 最小开关:会话卡不镜像进项目公共流(cardSvc 在上方已建)。
cardSvc.MirrorConvCards = !opt.DisableConvCardMirror
srv.Workflows = wfReg
mgr.SetWorkflows(wfReg)
srv.Cards = cardSvc
srv.Surfaces = surfaceSvc
// WP3-I4:编译插件摘要进 /api/plugins 管理清单(注册面即清单面)。
srv.CompiledPlugins = compiledSummary
srv.Bus = bus
srv.Gate = gate
srv.Prompts = promptSvc
@@ -445,38 +738,88 @@ func Run(opt Options) error {
return nil
}
// CreateUserAccount CLI 建号(管理员发放制):未给密码则生成随机 12 位并返回明文(仅此一次输出)。
func CreateUserAccount(storageDSN, dataDir, owner, name, password string, isAdmin bool) (string, error) {
if password == "" {
password = server.RandomPassword()
}
hash, err := store.HashPassword(password)
if err != nil {
return "", err
}
if err := withStore(storageDSN, dataDir, func(st *store.Store) error {
return st.CreateAccount(owner, name, hash, isAdmin)
}); err != nil {
return "", err
}
return password, nil
}
// ResetUserPassword CLI 重置密码:must_change=1(首登强制改密),吊销全部会话。
func ResetUserPassword(storageDSN, dataDir, owner, password string) (string, error) {
if password == "" {
password = server.RandomPassword()
}
hash, err := store.HashPassword(password)
if err != nil {
return "", err
}
if err := withStore(storageDSN, dataDir, func(st *store.Store) error {
if err := st.SetUserPassword(owner, hash); err != nil {
// CreateUserAccount CLI 建号。
// password 空且 generatePassword=false:无初始密码,同事务签发 activate Key,返回明文 Key。
// generatePassword=true 或显式 password:兼容旧路径(随机/给定密码 + must_change)。
func CreateUserAccount(storageDSN, dataDir, owner, name, password string, isAdmin, openExisting, generatePassword bool) (string, error) {
var out string
err := withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
if password != "" || generatePassword {
pw := password
if pw == "" {
pw = server.RandomPassword()
}
hash, err := store.HashPassword(pw)
if err != nil {
return err
}
if err := st.CreateAccount(owner, name, hash, isAdmin); err != nil {
return err
}
out = pw
return nil
}
if err := st.CreateAccount(owner, name, "", isAdmin); err != nil {
return err
}
st.DeleteUserSessions(owner, "")
plain, _, err := st.IssueLoginKey(owner, store.LoginKeyPurposeActivate, "cli", 0)
if err != nil {
return err
}
out = plain
return nil
})
return out, err
}
// IssueUserLoginKey CLI 签发激活/恢复 Key。
func IssueUserLoginKey(storageDSN, dataDir, owner, purpose string, openExisting, revokeAllCreds bool, ttl time.Duration) (string, error) {
var plain string
err := withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
if revokeAllCreds {
if err := st.RevokeAllAPIKeys(owner); err != nil {
return err
}
st.DeleteUserSessions(owner, "")
}
p, _, err := st.IssueLoginKey(owner, purpose, "cli", ttl)
plain = p
return err
})
return plain, err
}
// RevokeUserLoginKey CLI 撤销未使用登录 Key。
func RevokeUserLoginKey(storageDSN, dataDir, id string, openExisting bool) error {
return withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
return st.RevokeLoginKey(id)
})
}
// BootstrapAdmin 首位管理员引导:单事务串行(KTD12——PG 咨询锁/SQLite 插入后复查),
// 建号、激活 Key 与审计共同提交;库内已有用户即拒绝接管。
func BootstrapAdmin(storageDSN, dataDir, owner string, openExisting bool) (string, error) {
var plain string
err := withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
p, err := st.BootstrapFirstAdmin(owner, owner, "cli-bootstrap")
plain = p
return err
})
return plain, err
}
// ResetUserPassword CLI 重置密码:单事务写 must_change=1、bump 认证版本、
// 作废未用登录 Key、吊销全部会话并审计。
func ResetUserPassword(storageDSN, dataDir, owner, password string, openExisting bool) (string, error) {
if password == "" {
password = server.RandomPassword()
}
hash, err := store.HashPassword(password)
if err != nil {
return "", err
}
if err := withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
return st.AdminResetUserPassword(owner, hash, "cli", "cli")
}); err != nil {
return "", err
}
@@ -484,9 +827,9 @@ func ResetUserPassword(storageDSN, dataDir, owner, password string) (string, err
}
// ListUserAccounts CLI 用户列表。
func ListUserAccounts(storageDSN, dataDir string) ([]store.AdminUser, error) {
func ListUserAccounts(storageDSN, dataDir string, openExisting bool) ([]store.AdminUser, error) {
var out []store.AdminUser
err := withStore(storageDSN, dataDir, func(st *store.Store) error {
err := withStoreMode(storageDSN, dataDir, openExisting, func(st *store.Store) error {
var e error
out, e = st.ListUsersAdmin()
return e
@@ -495,12 +838,39 @@ func ListUserAccounts(storageDSN, dataDir string) ([]store.AdminUser, error) {
}
func withStore(storageDSN, dataDir string, fn func(*store.Store) error) error {
return withStoreMode(storageDSN, dataDir, false, fn)
}
// withIdentityStore 只打开既有库:不建文件、不建目录、不迁移。
func withIdentityStore(storageDSN, dataDir string, fn func(*store.Store) error) error {
st, err := store.OpenIdentity(storageDSN, dataDir)
if err != nil {
return err
}
defer st.Close()
return fn(st)
}
func withStoreMode(storageDSN, dataDir string, openExisting bool, fn func(*store.Store) error) error {
var st *store.Store
var err error
// DSN 等价安全输入(KTD14):argv 缺省时回落环境变量——凭据不落 shell history。
if storageDSN == "" {
storageDSN = os.Getenv("AGENT_SYSTEM_DSN")
}
if storageDSN != "" {
st, err = store.OpenPostgres(storageDSN, filepath.Join(dataDir, "artifacts"))
if openExisting {
st, err = store.OpenExistingPostgres(storageDSN, filepath.Join(dataDir, "artifacts"))
} else {
st, err = store.OpenPostgres(storageDSN, filepath.Join(dataDir, "artifacts"))
}
} else {
st, err = store.OpenSQLite(filepath.Join(dataDir, "agent-system.db"))
dbPath := filepath.Join(dataDir, "agent-system.db")
if openExisting {
st, err = store.OpenExistingSQLite(dbPath)
} else {
st, err = store.OpenSQLite(dbPath)
}
}
if err != nil {
return err
@@ -560,6 +930,32 @@ func RevokeAPIKey(storageDSN, dataDir, owner, name string) (bool, error) {
return st.RevokeAPIKeyByName(owner, name)
}
// EnsureSystemIdentity CLI:显式建立库内服务身份(svc init / I2→I3 引导)。
// 只打开既有库,不建库、不迁移。
func EnsureSystemIdentity(storageDSN, dataDir, instanceID, dbref string) error {
return withIdentityStore(storageDSN, dataDir, func(st *store.Store) error {
return st.EnsureSystemIdentity(instanceID, dbref)
})
}
// VerifySystemIdentity CLI:校验库内服务身份(svc start 前;失败不插入)。
// 只打开既有库,不建库、不迁移。schemaVersion 取身份行上的值。
func VerifySystemIdentity(storageDSN, dataDir, instanceID, dbref string) (int, error) {
var schemaVersion int
err := withIdentityStore(storageDSN, dataDir, func(st *store.Store) error {
if err := st.VerifySystemIdentity(instanceID, dbref); err != nil {
return err
}
row, err := st.GetSystemIdentity()
if err != nil {
return err
}
schemaVersion = row.SchemaVersion
return nil
})
return schemaVersion, err
}
// cachedInt 带短 TTL 的整型缓存读(限流值热读取免打库)。
func cachedInt(ttl time.Duration, fn func() int) func() int {
var mu sync.Mutex
@@ -632,3 +1028,31 @@ func splitKeyPool(s string) []string {
}
return out
}
// busAdapter EXT-14:cmdbus.Bus 适配 runmgr.CommandSource。视图类型转换在此收口
// (runmgr 不依赖 cmdbus,边不进架构基线);Entrance 由 runmgr 固定为 agent。
type busAdapter struct{ bus *cmdbus.Bus }
func (a busAdapter) CommandTools() []runmgr.CommandTool {
return a.CommandToolsFor("")
}
func (a busAdapter) CommandToolsFor(owner string) []runmgr.CommandTool {
src := a.bus.CommandToolsFor(owner)
out := make([]runmgr.CommandTool, len(src))
for i, ct := range src {
out[i] = runmgr.CommandTool{ID: ct.ID, Description: ct.Description, Params: ct.Params, Risk: ct.Risk}
}
return out
}
func (a busAdapter) DispatchCommand(ctx context.Context, req runmgr.CommandRequest) runmgr.CommandResponse {
resp := a.bus.Dispatch(ctx, cmdbus.Request{
ID: req.ID, Params: req.Params, Entrance: req.Entrance,
Actor: req.Actor, Owner: req.Owner, ConversationID: req.ConversationID,
})
return runmgr.CommandResponse{
State: string(resp.Receipt.State), Reason: resp.Receipt.Reason,
Payload: resp.Payload, Card: resp.Card != nil,
}
}
+140 -4
View File
@@ -3,6 +3,7 @@ package app
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"os"
@@ -10,6 +11,7 @@ import (
agentcommand "agent-command"
"agent-system-backend/internal/cmdbus"
"agent-system-backend/internal/server"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
@@ -25,6 +27,29 @@ type pluginRegistry struct {
compiledRegistry
pluginID string
caps hostcaps
// summary 是本插件的注册面摘要(WP3-I4):AddAdminZone 声明记录于此,
// server 经 CompiledPlugins 供给分区清单端点(WP3-I3)。
summary *server.CompiledPlugin
}
// AddAdminZone 记录管理区分区声明(WP3-I3):声明即展示元数据,权威记录
// 就是注册期摘要;规格非法(非对象、id/title 缺失)fail-closed 拒绝,
// 使 register() 整体中止(与命令注册同一失败语义)。
func (r pluginRegistry) AddAdminZone(spec json.RawMessage) error {
if r.summary == nil {
return errors.New("extension: " + r.pluginID + ": admin zone outside applyCompiled")
}
var s extension.AdminZoneSpec
if err := json.Unmarshal(spec, &s); err != nil {
return fmt.Errorf("extension: admin zone spec: %w", err)
}
if s.ID == "" || s.Title == "" {
return fmt.Errorf("extension: %s: admin zone requires id and title", r.pluginID)
}
r.summary.AdminZones = append(r.summary.AdminZones, server.AdminZoneSection{
ID: s.ID, Title: s.Title, Icon: s.Icon, Desc: s.Desc, Order: s.Order,
})
return nil
}
func (r pluginRegistry) AddCommandHandler(spec json.RawMessage, h extension.Handler) error {
@@ -39,14 +64,30 @@ func (r pluginRegistry) AddCommandHandler(spec json.RawMessage, h extension.Hand
return fmt.Errorf("extension: command spec: %w", err)
}
return r.bus.RegisterPluginCommand(s, func(ctx context.Context, req cmdbus.Request) (any, error) {
host := inprocHost{caps: r.caps, pluginID: r.pluginID, owner: req.Owner}
host := inprocHost{caps: r.caps, pluginID: r.pluginID, owner: req.Owner, actor: req.Actor}
return h(ctx, host, extension.Request{
ID: req.ID, Params: req.Params, Entrance: req.Entrance,
Actor: req.Actor, Owner: req.Owner, ConversationID: req.ConversationID,
HostProjectID: req.HostProjectID,
})
})
}
// AddIntent 意图声明(G2-2):汇入总线意图表;插件身份随声明归因。
func (r pluginRegistry) AddIntent(spec json.RawMessage) error {
if r.bus == nil {
return nil
}
var is extension.IntentSpec
if err := json.Unmarshal(spec, &is); err != nil {
return fmt.Errorf("extension: intent spec: %w", err)
}
return r.bus.Intents().Declare(cmdbus.Intent{
ID: is.ID, Phrases: is.Phrases, Command: is.Command,
Params: is.Params, Description: is.Description, Plugin: r.pluginID,
})
}
func (r compiledRegistry) AddSkill(name, markdown string) error {
if r.skillsRoot == "" || name == "" {
return nil
@@ -65,22 +106,117 @@ func (r compiledRegistry) AddCommand(spec json.RawMessage) error {
return r.bus.RegisterPluginCommands([]json.RawMessage{spec})
}
// RegisterResourceAuthorizer 声明式扩展无宿主进程内代码,不能提供回调(fail-closed)。
// RegisterResourceAuthorizerV2 声明式扩展同样不能提供资源级回调(fail-closed)。
func (r compiledRegistry) RegisterResourceAuthorizerV2(extension.ResourceAuthorizerV2) error {
return errors.New("extension: declarative plugins cannot register resource authorizers")
}
func (r compiledRegistry) RegisterResourceAuthorizer(extension.ResourceAuthorizer) error {
return errors.New("extension: declarative plugins cannot register resource authorizers")
}
// RegisterResourceAuthorizer 进程内扩展(EXT-1):填充 EXT-8 适配器槽
// (hostcaps.authz 共享指针;首个注册者生效,组合根已装配进 Server/runmgr)。
// 此前 pluginRegistry 未覆盖该方法——进程内插件注册适配器会命中声明式分支的
// 拒绝,register() 整体中止,插件命令全部未注册(RDM 二进制 rdm.* 全部
// 「unknown command」;MS1 重验收烟测实测发现)。
func (r pluginRegistry) RegisterResourceAuthorizer(fn extension.ResourceAuthorizer) error {
if fn == nil {
return errors.New("extension: " + r.pluginID + ": nil resource authorizer")
}
r.caps.authz.set(r.pluginID, fn)
return nil
}
// RegisterResourceAuthorizerV2 进程内扩展:填充 EXT-8b 资源级适配器槽(首注册生效)。
func (r pluginRegistry) RegisterResourceAuthorizerV2(fn extension.ResourceAuthorizerV2) error {
if fn == nil {
return errors.New("extension: " + r.pluginID + ": nil resource authorizer v2")
}
r.caps.authz.setV2(r.pluginID, fn)
return nil
}
func (r compiledRegistry) Note(kind, name string, payload json.RawMessage) error {
log.Printf("extension: note %s %s (%d bytes)", kind, name, len(payload))
return nil
}
func applyCompiled(skillsRoot string, bus *cmdbus.Bus, caps hostcaps) {
// recordRegistry 包装 per-plugin 注册表,把成功注册的声明资产名记入摘要
// (WP3-I4:注册面即清单面——编译插件的管理条目不另设声明口)。
type recordRegistry struct {
extension.Registry
sum *server.CompiledPlugin
}
func (r recordRegistry) AddCommand(spec json.RawMessage) error {
if err := r.Registry.AddCommand(spec); err != nil {
return err
}
if id := specID(spec); id != "" {
r.sum.Commands = append(r.sum.Commands, id)
}
return nil
}
func (r recordRegistry) AddCommandHandler(spec json.RawMessage, h extension.Handler) error {
if err := r.Registry.AddCommandHandler(spec, h); err != nil {
return err
}
if id := specID(spec); id != "" {
r.sum.Commands = append(r.sum.Commands, id)
}
return nil
}
func (r recordRegistry) AddIntent(spec json.RawMessage) error {
if err := r.Registry.AddIntent(spec); err != nil {
return err
}
if id := specID(spec); id != "" {
r.sum.Intents = append(r.sum.Intents, id)
}
return nil
}
func (r recordRegistry) AddSkill(name, markdown string) error {
if err := r.Registry.AddSkill(name, markdown); err != nil {
return err
}
r.sum.Skills = append(r.sum.Skills, name)
return nil
}
// specID 宽松提取 spec 顶层 id:仅服务清单摘要展示,解析失败返回空串
// 不记录,不影响注册结果本身。
func specID(spec json.RawMessage) string {
var s struct {
ID string `json:"id"`
}
if err := json.Unmarshal(spec, &s); err != nil {
return ""
}
return s.ID
}
func applyCompiled(skillsRoot string, bus *cmdbus.Bus, caps hostcaps) []server.CompiledPlugin {
base := compiledRegistry{skillsRoot: skillsRoot, bus: bus}
var out []server.CompiledPlugin
for _, e := range extension.Compiled() {
if e.Register == nil {
continue
}
reg := pluginRegistry{compiledRegistry: base, pluginID: e.ID, caps: caps}
if err := e.Register(reg); err != nil {
sum := server.CompiledPlugin{ID: e.ID}
reg := pluginRegistry{compiledRegistry: base, pluginID: e.ID, caps: caps, summary: &sum}
// Register 失败的插件保留已成功注册部分(与命令总线一致:部分注册
// 生效),摘要如实反映实际注册面。
if err := e.Register(recordRegistry{Registry: reg, sum: &sum}); err != nil {
log.Printf("extension: %s: %v", e.ID, err)
}
out = append(out, sum)
}
return out
}
func mergeMCPServers(raw string, extra []core.PluginMCPServer) string {
+55
View File
@@ -0,0 +1,55 @@
package app
import (
"context"
"testing"
"time"
"agent-system-backend/internal/cmdbus"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
// fakeExtAuthz 进程内扩展:注册 EXT-8 适配器+一条命令——修复前适配器注册被
// 声明式分支拒绝,register() 中止导致命令全部丢失。
type fakeExtAuthz struct{}
func (f *fakeExtAuthz) Register(reg extension.Registry) error {
if err := reg.RegisterResourceAuthorizer(func(owner string, projectID core.ID) (bool, error) {
return owner == "alice", nil
}); err != nil {
return err
}
return reg.AddCommandHandler([]byte(`{"id":"t.authz.ping","title":"t"}`),
func(ctx context.Context, h extension.Host, req extension.Request) (any, error) {
return map[string]any{"ok": true}, nil
})
}
func TestInprocPluginRegistersAuthorizerAndCommands(t *testing.T) {
extension.ResetCompiled()
extension.RegisterCompiled(extension.InProc{ID: "t", Register: (&fakeExtAuthz{}).Register})
defer extension.ResetCompiled()
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
bus := cmdbus.New(st, nil, func() time.Time { return time.Unix(1700000000, 0).UTC() })
caps := hostcaps{st: st, authz: &projectAuthz{}}
applyCompiled("", bus, caps)
// ① 命令已注册且可分发(修复前:适配器注册被拒 → register() 中止 →
// 命令全部丢失,rdm.* 全部 unknown command)。
if resp := bus.Dispatch(context.Background(), cmdbus.Request{
ID: "t.authz.ping", Entrance: "button", Owner: "alice"}); resp.Receipt.State != "committed" {
t.Fatalf("命令分发 = %+v(注册整体中止)", resp.Receipt)
}
// ② 适配器经共享槽生效:alice 放行、bob 拒绝。
if ok, _ := caps.authz.Authorize("alice", "p1"); !ok {
t.Fatal("alice 应放行")
}
if ok, _ := caps.authz.Authorize("bob", "p1"); ok {
t.Fatal("bob 应被拒绝")
}
}
+92
View File
@@ -0,0 +1,92 @@
package app
import (
"context"
"encoding/json"
"path/filepath"
"testing"
"time"
agentcommand "agent-command"
"agent-system-backend/internal/cmdbus"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
)
// WP3-I4:applyCompiled 的注册面摘要——成功注册的命令/意图/技能记入
// CompiledPlugin,失败注册不记录,Register 整体失败保留已成功部分。
func TestApplyCompiledRecordsSummary(t *testing.T) {
extension.ResetCompiled()
t.Cleanup(extension.ResetCompiled)
extension.RegisterCompiled(extension.InProc{ID: "demo", Register: func(reg extension.Registry) error {
if err := reg.AddCommandHandler(json.RawMessage(`{"id":"demo.one","title":"一"}`), func(ctx context.Context, h extension.Host, req extension.Request) (any, error) {
return "ok", nil
}); err != nil {
return err
}
// 重复 id 注册失败:总线拒绝,摘要不记录。
if err := reg.AddCommand(json.RawMessage(`{"id":"demo.one","title":"重复"}`)); err == nil {
t.Fatal("duplicate command should fail")
}
if err := reg.AddIntent(json.RawMessage(`{"id":"demo.go","phrases":["演示"],"command":"demo.one"}`)); err != nil {
return err
}
_ = reg.AddSkill("demo-skill", "# 演示\n技能正文")
// WP3-I3:管理区分区声明记入摘要;缺 id/title 的非法规格 fail-closed。
if err := reg.AddAdminZone(json.RawMessage(`{"id":"demo.audit","title":"演示审计","icon":"search","order":20}`)); err != nil {
return err
}
if err := reg.AddAdminZone(json.RawMessage(`{"title":"缺 id"}`)); err == nil {
t.Fatal("admin zone without id should fail")
}
return nil
}})
extension.RegisterCompiled(extension.InProc{ID: "broken", Register: func(reg extension.Registry) error {
if err := reg.AddCommand(json.RawMessage(`{"id":"broken.cmd","title":"先成功"}`)); err != nil {
return err
}
return reg.AddCommandHandler(json.RawMessage(`not-json`), nil)
}})
// 无 Register 的登记不产生条目。
extension.RegisterCompiled(extension.InProc{ID: "empty"})
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
bus := cmdbus.New(st, nil, func() time.Time { return time.Unix(1700000000, 0).UTC() })
sums := applyCompiled(t.TempDir(), bus, hostcaps{st: st, set: settings.NewService(st)})
if len(sums) != 2 {
t.Fatalf("want 2 summaries, got %d: %+v", len(sums), sums)
}
if sums[0].ID != "demo" {
t.Fatalf("sums[0] = %+v", sums[0])
}
if len(sums[0].Commands) != 1 || sums[0].Commands[0] != "demo.one" {
t.Fatalf("demo commands = %v", sums[0].Commands)
}
if len(sums[0].Intents) != 1 || sums[0].Intents[0] != "demo.go" {
t.Fatalf("demo intents = %v", sums[0].Intents)
}
if len(sums[0].Skills) != 1 || sums[0].Skills[0] != "demo-skill" {
t.Fatalf("demo skills = %v", sums[0].Skills)
}
// WP3-I3:合法声明记入,非法规格(缺 id)被拒后摘要不受污染。
if len(sums[0].AdminZones) != 1 {
t.Fatalf("demo admin zones = %+v", sums[0].AdminZones)
}
if z := sums[0].AdminZones[0]; z.ID != "demo.audit" || z.Title != "演示审计" || z.Icon != "search" || z.Order != 20 {
t.Fatalf("demo admin zone = %+v", z)
}
// broken:Register 报错但先成功的命令仍在总线,摘要如实保留。
if sums[1].ID != "broken" || len(sums[1].Commands) != 1 || sums[1].Commands[0] != "broken.cmd" {
t.Fatalf("broken summary = %+v", sums[1])
}
// 摘要与总线一致:记录的命令真实可分发(不是空头衔)。
resp := bus.Dispatch(context.Background(), cmdbus.Request{ID: "demo.one", Owner: "alice", Entrance: "test"})
if resp.Receipt.State != agentcommand.StateCommitted {
t.Fatalf("demo.one dispatch = %s (%s)", resp.Receipt.State, resp.Receipt.Reason)
}
}
+69
View File
@@ -0,0 +1,69 @@
package app
// EXT-13 红测(提案 2026-10-09-ext-g1-b1-b3 §1):进程内 Host.KnowledgeSearch
// 的检索面在适配器放行后吃到他人项目的共享知识;未放行保持本人两路、零泄露。
import (
"context"
"path/filepath"
"testing"
"agent-system-backend/internal/rag"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
core "agent-system-core"
)
// newExt13Host EXT-13 装配:真实 store + rag 管线 + authz 槽 + inprocHost(bob 视角)。
func newExt13Host(t *testing.T) (interface {
KnowledgeSearch(ctx context.Context, projectID core.ID, text string, topK int) ([]core.RetrievalHit, error)
}, *projectAuthz) {
t.Helper()
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
set := settings.NewService(st)
pipe := rag.NewPipeline(st, set, nil, nil)
authz := &projectAuthz{}
h := inprocHost{caps: hostcaps{st: st, pipe: pipe, set: set, authz: authz}, pluginID: "rdm", owner: "bob"}
// alice 已发布的项目知识(带 ProjectID;模拟发布状态机产物)。
if err := st.AddKnowledge(core.KnowledgeDoc{Owner: "alice", ProjectID: "p-1",
Source: "plugin:rdm", Ref: "ext13-1", Chunk: "EXT13共享独占令牌陈述"}); err != nil {
t.Fatal(err)
}
return h, authz
}
func TestInprocHostKnowledgeSearchShared(t *testing.T) {
h, authz := newExt13Host(t)
ctx := context.Background()
q := core.RetrievalQuery{Text: "EXT13共享独占令牌", TopK: 5}
if hits, err := h.KnowledgeSearch(ctx, "", q.Text, q.TopK); err != nil || len(hits) != 0 {
t.Fatalf("before grant: hits=%d err=%v, want 0(未放行零泄露)", len(hits), err)
}
// 适配器放行 bob × p-1 → Host 检索面吃到 alice 的共享命中。
// (槽为首个注册者生效——切换授权改闭包内变量,不重注册。)
allowBob := true
authz.set("rdm", func(owner string, projectID core.ID) (bool, error) {
return allowBob && owner == "bob" && string(projectID) == "p-1", nil
})
hits, err := h.KnowledgeSearch(ctx, "", q.Text, q.TopK)
if err != nil {
t.Fatal(err)
}
if len(hits) == 0 {
t.Fatal("after grant: host search got 0 shared hits, want ≥1(EXT-13)")
}
// 指定他人 ProjectID 时同样经适配器(发布边界在下游状态机)。
if hits, err := h.KnowledgeSearch(ctx, "p-1", q.Text, q.TopK); err != nil || len(hits) == 0 {
t.Fatalf("scoped to shared project: hits=%d err=%v, want ≥1", len(hits), err)
}
// 撤销(同一注册内翻转授权)→ 共享命中即停、零泄露。
allowBob = false
if hits, err := h.KnowledgeSearch(ctx, "", q.Text, q.TopK); err != nil || len(hits) != 0 {
t.Fatalf("after revoke: hits=%d err=%v, want 0", len(hits), err)
}
}
+62
View File
@@ -0,0 +1,62 @@
package app
// EXT-18 AdminProvider 正负例:admin 主体 → true;非 admin / 空 actor(scoped
// host)→ false;store 查询失败 → false(fail-closed)。
import (
"testing"
"agent-system-backend/internal/store"
)
func TestExt18AdminProviderPositive(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("boss", "Boss", "pw", true); err != nil {
t.Fatal(err)
}
h := inprocHost{caps: hostcaps{st: st}, pluginID: "tst", owner: "boss", actor: "boss"}
if !h.IsAdmin() {
t.Fatal("admin 主体应返回 true")
}
}
func TestExt18AdminProviderNegative(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("user", "User", "pw", false); err != nil {
t.Fatal(err)
}
h := inprocHost{caps: hostcaps{st: st}, pluginID: "tst", owner: "user", actor: "user"}
if h.IsAdmin() {
t.Fatal("非 admin 主体应返回 false")
}
}
func TestExt18AdminProviderEmptyActor(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
if err := st.CreateAccount("boss", "Boss", "pw", true); err != nil {
t.Fatal(err)
}
h := inprocHost{caps: hostcaps{st: st}, pluginID: "tst", owner: "boss", actor: ""} // scoped host(无 dispatch)
if h.IsAdmin() {
t.Fatal("空 actor(非 dispatch 上下文)恒 false")
}
}
func TestExt18RequestHostProjectIDField(t *testing.T) {
// 编译期断言:Request 有 HostProjectID 字段。
_ = ""
var r struct{ HostProjectID string }
r.HostProjectID = "proj-1"
if r.HostProjectID != "proj-1" {
t.Fatal("字段赋值失败")
}
}
+81
View File
@@ -0,0 +1,81 @@
package app
import (
"context"
"path/filepath"
"testing"
"agent-system-backend/internal/rag"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
core "agent-system-core"
)
// newExt9Host EXT-9 测试装配:真实 store + rag 管线 + inprocHost(无 embedder,
// embed=false 走全文档路径)。
func newExt9Host(t *testing.T) (interface {
KnowledgePublish(ctx context.Context, docs []core.KnowledgeDoc, embed bool) error
KnowledgeDelete(ctx context.Context, sourcePrefix string) error
}, *store.Store, *rag.Pipeline) {
t.Helper()
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
set := settings.NewService(st)
pipe := rag.NewPipeline(st, set, nil, nil)
h := inprocHost{caps: hostcaps{st: st, pipe: pipe, set: set}, pluginID: "rdm", owner: "alice"}
return h, st, pipe
}
func searchAll(t *testing.T, pipe *rag.Pipeline, owner, text string) []core.RetrievalHit {
t.Helper()
hits, err := pipe.Retrieve(context.Background(), core.ID(owner), core.RetrievalQuery{Text: text, TopK: 50})
if err != nil {
t.Fatalf("retrieve: %v", err)
}
return hits
}
// EXT-9 验收:publish 按 ref 幂等(重试不重复);delete 后检索即停且幂等;
// manual 来源不经 host 可删(前缀强制 plugin:<自身>)。
func TestKnowledgePublishIdempotentAndRevoke(t *testing.T) {
h, st, pipe := newExt9Host(t)
ctx := context.Background()
docs := []core.KnowledgeDoc{
{Ref: "batch-001", Chunk: "EXT9甲 独占令牌之一陈述"},
{Ref: "batch-001", Chunk: "EXT9乙 独占令牌之二陈述"},
}
if err := h.KnowledgePublish(ctx, docs, false); err != nil {
t.Fatalf("publish#1: %v", err)
}
if err := h.KnowledgePublish(ctx, docs, false); err != nil { // 重试幂等
t.Fatalf("publish#2: %v", err)
}
hits := searchAll(t, pipe, "alice", "独占令牌")
if len(hits) != 2 {
t.Fatalf("after double publish: hits=%d, want 2(幂等不重复;got %+v)", len(hits), hits)
}
if err := h.KnowledgeDelete(ctx, "plugin:rdm"); err != nil {
t.Fatalf("delete: %v", err)
}
if hits = searchAll(t, pipe, "alice", "独占令牌"); len(hits) != 0 {
t.Fatalf("after delete: hits=%d, want 0(撤销即停)", len(hits))
}
if err := h.KnowledgeDelete(ctx, "plugin:rdm"); err != nil { // 幂等
t.Fatalf("delete#2: %v", err)
}
// manual 来源:host 前缀强制拒绝,且内容不受插件删除影响。
if err := st.AddKnowledge(core.KnowledgeDoc{Owner: "alice", Source: "manual", Ref: "m-1", Chunk: "manual 手册独占词"}); err != nil {
t.Fatal(err)
}
if err := h.KnowledgeDelete(ctx, "manual"); err == nil {
t.Fatal("delete(manual) must be rejected: plugins may only revoke their own prefix")
}
if hits = searchAll(t, pipe, "alice", "手册独占词"); len(hits) != 1 {
t.Fatalf("manual row must survive: hits=%d", len(hits))
}
}
+333 -5
View File
@@ -1,13 +1,19 @@
package app
import (
"agent-system-backend/pkg/extension"
"context"
"fmt"
"sort"
"strings"
"sync"
"agent-system-backend/internal/ingest"
"agent-system-backend/internal/rag"
"agent-system-backend/internal/runtime"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
"agent-system-backend/internal/surfacesvc"
core "agent-system-core"
)
@@ -15,14 +21,119 @@ import (
// 同一实现同时服务 hostapi.Server 的 Search/Complete 闭包与进程内
// inprocHost(EXT-1 批注 1:复用现有 owner 作用域函数,不新造句柄)。
type hostcaps struct {
st *store.Store
pipe *rag.Pipeline
set *settings.Service
projects *projectLifecycleSlot
st *store.Store
pipe *rag.Pipeline
set *settings.Service
authz *projectAuthz // EXT-8 适配器槽(共享指针;首个注册者生效)
surfaces *surfacesvc.Service // EXT-17 Surface 能力槽(nil = 未装配,明确拒绝)
}
// projectAuthz EXT-8 资源授权适配器槽:进程内扩展注册,组合根装配进 Server。
type projectAuthz struct {
mu sync.Mutex
holder string // 持有者 pluginID(首个注册者生效)
fn extension.ResourceAuthorizer
holder2 string // EXT-8b 资源级持有者(首个注册者生效)
fn2 extension.ResourceAuthorizerV2
}
func (a *projectAuthz) set(pluginID string, fn extension.ResourceAuthorizer) {
a.mu.Lock()
defer a.mu.Unlock()
if a.fn != nil {
return // 首个注册者生效;后来者忽略(调用方经日志可查)
}
a.holder, a.fn = pluginID, fn
}
func (a *projectAuthz) setV2(pluginID string, fn extension.ResourceAuthorizerV2) {
a.mu.Lock()
defer a.mu.Unlock()
if a.fn2 != nil {
return
}
a.holder2, a.fn2 = pluginID, fn
}
// AuthorizeResource EXT-8b 资源级判定(U9):V2 注册时按查询六要素咨询(含
// owner 自有命中与共享读取,fail-closed);未注册回落 V1 的 owner×project
// 布尔(兼容现行行为)。查询字段由执行端受信构造。
func (a *projectAuthz) AuthorizeResource(q extension.ResourceAuthzQuery) (bool, error) {
a.mu.Lock()
fn2, fn1 := a.fn2, a.fn
a.mu.Unlock()
if fn2 != nil {
if q.Actor == "" || q.Action == "" || q.ResourceType == "" || q.ResourceID == "" {
return false, nil // 受信上下文字段缺失=不可判定=拒绝
}
return fn2(q)
}
if fn1 == nil || q.Owner == "" || q.ProjectID == "" {
return false, nil
}
return fn1(q.Actor, q.ProjectID)
}
// Authorize owner×project 共享读取判定;未注册一律 false(默认拒绝)。
func (a *projectAuthz) hasV2() bool {
a.mu.Lock()
defer a.mu.Unlock()
return a.fn2 != nil
}
func (a *projectAuthz) Authorize(owner string, projectID core.ID) (bool, error) {
a.mu.Lock()
fn := a.fn
a.mu.Unlock()
if fn == nil || owner == "" || projectID == "" {
return false, nil
}
return fn(owner, projectID)
}
// wireScopedHostFactory EXT-11:把「按 owner+pluginID 构造受限 Host」注入公开面
// (extension.SetScopedHostFactory)。组合根在 hostcaps 就绪后调用;重建句柄与
// 分发期同构(owner 作用域、来源前缀强制、KV/任务台账隔离)。
func wireScopedHostFactory(caps hostcaps) {
extension.SetScopedHostFactory(func(owner, pluginID string) extension.Host {
return inprocHost{caps: caps, pluginID: pluginID, owner: owner}
})
}
// search owner 作用域混合检索;projectID/topK 为空/0 时用管线默认值。
// sharedRetriever EXT-13:rag.Pipeline.RetrieveShared 形状(避免 app 直接 import rag
// 的具体类型断言散落)。
type sharedRetriever interface {
RetrieveShared(ctx context.Context, owner core.ID, q core.RetrievalQuery, allow func(docOwner, projectID core.ID) bool) ([]core.RetrievalHit, error)
}
func (c hostcaps) search(ctx context.Context, owner string, projectID core.ID, text string, topK int) ([]core.RetrievalHit, error) {
return c.pipe.Retrieve(ctx, core.ID(owner), core.RetrievalQuery{ProjectID: projectID, Text: text, TopK: topK})
q := core.RetrievalQuery{ProjectID: projectID, Text: text, TopK: topK}
if c.projects != nil && projectID != "" {
id, err := c.projects.resolve(ctx, c, owner, projectID)
if err != nil {
return nil, err
}
q.ProjectID = id
}
if c.authz != nil && c.authz.hasV2() {
return c.pipe.RetrieveAuthorized(ctx, core.ID(owner), q, func(d core.KnowledgeDoc) bool {
ok, err := c.authz.AuthorizeResource(extension.ResourceAuthzQuery{Actor: owner, Action: "search", ResourceType: "knowledge", ResourceID: d.ID, Owner: d.Owner, ProjectID: d.ProjectID, Source: d.Source})
return err == nil && ok
})
}
// EXT-13:进程内 Host 检索面与回合内检索同一授权口径——适配器已注册时走
// 共享通道(本人两路如常 + 放行的他人项目知识,与 HTTP 检索面一致)。
if c.authz != nil {
if rs, ok := interface{}(c.pipe).(sharedRetriever); ok {
return rs.RetrieveShared(ctx, core.ID(owner), q, func(docOwner, pid core.ID) bool {
ok, err := c.authz.Authorize(owner, pid)
return err == nil && ok
})
}
}
return c.pipe.Retrieve(ctx, core.ID(owner), q)
}
// searchDefault 与 hostapi /host/v1 检索端点同口径(不带 projectID/topK)。
@@ -39,17 +150,118 @@ func (c hostcaps) complete(ctx context.Context, owner, prompt string) (string, e
return completeChat(ctx, route.Base, route.Key, route.Model, prompt)
}
// maxParseInput 与上传端点的读取上限同口径(server.upload 的 100<<20)。
const maxParseInput = 100 << 20
// parse 走基座内置解析器(EXT-5):纯计算、不落 owner 数据;不可解析由
// Warnings 显式表达,只有取消与超限是 error。
func (c hostcaps) parse(ctx context.Context, filename, mime string, content []byte) (core.ParsedDocument, error) {
if err := ctx.Err(); err != nil {
return core.ParsedDocument{}, err
}
if len(content) > maxParseInput {
return core.ParsedDocument{}, fmt.Errorf("parse input %d bytes exceeds %d limit", len(content), maxParseInput)
}
return ingest.Parse(filename, mime, content), nil
}
// inprocHost 是进程内插件命令的 owner 作用域宿主句柄(extension.Host 实现)。
// 每次分发按请求 Owner 新建;所有能力调用都以该 owner 为第一参数,插件无法越权。
// actor 是本次分发主体(通常等于 owner);AdminProvider 能力经 actor 查询
// (EXT-18)。非 dispatch 上下文(EXT-11 scoped host)actor 为空——IsAdmin 恒 false。
type inprocHost struct {
caps hostcaps
pluginID string
owner string
actor string // EXT-18:分发主体(空=非 dispatch 上下文)
}
func (h inprocHost) PluginID() string { return h.pluginID }
func (h inprocHost) Owner() string { return h.owner }
// IsAdmin 实现 EXT-18 AdminProvider:查询本次分发主体的基座平台管理角色。
// actor 为空(非 dispatch 上下文)恒 false;store 查询失败按非管理员处理
// (fail-closed——不因存储故障放行超管面)。
func (h inprocHost) IsAdmin() bool {
if h.actor == "" {
return false
}
state, err := h.caps.st.GetUserAuthState(h.actor)
if err != nil {
return false
}
return state.IsAdmin
}
// ListAccounts 实现 EXT-19 AccountLister(最小账号目录):仅授权管理读面用。
// 有效态=账号存在且非停用(当前 users 表无停用位——must_change_password 不影响
// 有效性;后续加停用位在此映射)。P3(2026-10-13):携带 Name 供同名区分。
// 游标=owner 升序偏移。
func (h inprocHost) ListAccounts(cursor string, limit int) ([]extension.AccountEntry, string, error) {
if limit <= 0 || limit > 200 {
limit = 50
}
users, err := h.caps.st.ListUsersAdmin()
if err != nil {
return nil, "", err
}
entries := make([]extension.AccountEntry, 0, len(users))
for _, u := range users {
entries = append(entries, extension.AccountEntry{Owner: u.Owner, Name: u.Name, Valid: true})
}
// 排序+游标(owner 升序;cursor 为上一页末 owner,空=首页)。
for i := 0; i < len(entries); i++ {
for j := i + 1; j < len(entries); j++ {
if entries[j].Owner < entries[i].Owner {
entries[i], entries[j] = entries[j], entries[i]
}
}
}
start := 0
if cursor != "" {
for start < len(entries) && entries[start].Owner <= cursor {
start++
}
}
end := start + limit
if end > len(entries) {
end = len(entries)
}
next := ""
if end < len(entries) {
next = entries[end-1].Owner
}
return entries[start:end], next, nil
}
// SearchAccounts 实现 EXT-19 扩展 AccountSearcher(P3,2026-10-13):账号或
// 姓名子串检索(大小写不敏感;空 query = 首页候选)。结果按 owner 升序,
// 有效态口径与 ListAccounts 一致。
func (h inprocHost) SearchAccounts(query string, limit int) ([]extension.AccountEntry, error) {
if limit <= 0 || limit > 200 {
limit = 50
}
users, err := h.caps.st.ListUsersAdmin()
if err != nil {
return nil, err
}
q := strings.ToLower(strings.TrimSpace(query))
entries := make([]extension.AccountEntry, 0, len(users))
for _, u := range users {
if q != "" &&
!strings.Contains(strings.ToLower(u.Owner), q) &&
!strings.Contains(strings.ToLower(u.Name), q) {
continue
}
entries = append(entries, extension.AccountEntry{Owner: u.Owner, Name: u.Name, Valid: true})
}
sort.Slice(entries, func(i, j int) bool { return entries[i].Owner < entries[j].Owner })
if len(entries) > limit {
entries = entries[:limit]
}
return entries, nil
}
func (h inprocHost) KVPut(key, value string) error {
return h.caps.st.PutPluginKV(h.pluginID, h.owner, key, value)
}
@@ -73,9 +285,23 @@ func (h inprocHost) KnowledgeSearch(ctx context.Context, projectID core.ID, text
return h.caps.search(ctx, h.owner, projectID, text, topK)
}
// knowledgeSource preserves ordinary host Source semantics. A lifecycle provider
// introduces shared publication provenance, so plugins may only assert their own
// namespace; another plugin cannot impersonate an RDM published row.
func (h inprocHost) knowledgeSource(source string) error {
if h.caps.projects == nil || !h.caps.projects.active() {
return nil
}
prefix := "plugin:" + h.pluginID
if source == prefix || strings.HasPrefix(source, prefix+":") {
return nil
}
return fmt.Errorf("extension: knowledge source must belong to plugin %q", h.pluginID)
}
// KnowledgeWrite 强制 owner 归属:Owner 改写为本次分发 owner,Source 为空补
// "plugin:<id>";embed=false 只写全文行,embed=true 走 IndexChunks 建向量
//(embedding 失败降级纯全文,不阻断写入)。
// (embedding 失败降级纯全文,不阻断写入)。
func (h inprocHost) KnowledgeWrite(ctx context.Context, docs []core.KnowledgeDoc, embed bool) error {
if len(docs) == 0 {
return nil
@@ -86,6 +312,9 @@ func (h inprocHost) KnowledgeWrite(ctx context.Context, docs []core.KnowledgeDoc
if docs[i].Source == "" {
docs[i].Source = src
}
if err := h.knowledgeSource(docs[i].Source); err != nil {
return err
}
}
if !embed {
return h.caps.st.AddKnowledge(docs...)
@@ -93,6 +322,53 @@ func (h inprocHost) KnowledgeWrite(ctx context.Context, docs []core.KnowledgeDoc
return h.caps.pipe.IndexChunks(ctx, core.ID(h.owner), docs)
}
// KnowledgePublish replaces refs and their vectors within the publication scope.
// A registered lifecycle scopes replacement to the exact owner/project/plugin;
// ordinary unregistered Hosts retain the legacy cross-source ref semantics.
func (h inprocHost) KnowledgePublish(ctx context.Context, docs []core.KnowledgeDoc, embed bool) error {
if len(docs) == 0 {
return nil
}
type publicationRef struct{ projectID, ref string }
refs := map[publicationRef]bool{}
for i := range docs {
docs[i].Owner = core.ID(h.owner)
if docs[i].Source == "" {
docs[i].Source = "plugin:" + h.pluginID
}
if err := h.knowledgeSource(docs[i].Source); err != nil {
return err
}
if docs[i].Ref != "" {
refs[publicationRef{docs[i].ProjectID, docs[i].Ref}] = true
}
}
scoped := h.caps.projects != nil && h.caps.projects.active()
for ref := range refs {
var err error
if scoped {
err = h.caps.st.DeletePluginProjectKnowledgeByRef(ctx, h.owner, ref.projectID, ref.ref, h.pluginID)
} else {
err = h.caps.st.DeleteProjectKnowledgeByRef(ctx, h.owner, ref.projectID, ref.ref)
}
if err != nil {
return err
}
}
return h.KnowledgeWrite(ctx, docs, embed)
}
// KnowledgeDelete 撤销本插件来源(EXT-9):前缀强制 plugin:<自身ID>——manual
// 或他人来源拒绝(fail-closed);删行+向量,幂等。
func (h inprocHost) KnowledgeDelete(ctx context.Context, sourcePrefix string) error {
if sourcePrefix != "plugin:"+h.pluginID {
return fmt.Errorf("ext-9: plugins may only delete their own source prefix, want %q", "plugin:"+h.pluginID)
}
_, err := h.caps.st.DeleteKnowledgeBySource(ctx, h.owner, sourcePrefix)
return err
}
// TaskCreate/TaskUpdate 后台任务台账:基座只记进度/状态,执行体由下游自跑(H5)。
func (h inprocHost) TaskCreate(kind, title, conversationID string) (string, error) {
t, err := h.caps.st.CreateBackgroundTask(h.owner, kind, title, conversationID)
@@ -114,3 +390,55 @@ func (h inprocHost) ArtifactGet(artifactID string) (core.Artifact, error) {
func (h inprocHost) ArtifactContent(artifactID string) ([]byte, error) {
return h.caps.st.ArtifactContent(h.owner, artifactID)
}
// ParseDocument 内置文档解析(EXT-5):委托 hostcaps.parse;纯计算不经存储,
// owner 作用域由调用方经 ArtifactContent 取字节时保证。
func (h inprocHost) ParseDocument(ctx context.Context, filename, mime string, content []byte) (core.ParsedDocument, error) {
return h.caps.parse(ctx, filename, mime, content)
}
// ---- EXT-17 Surface 能力(可选):nil 槽=能力缺失,明确拒绝不降级 ----
func (h inprocHost) surface() (*surfacesvc.Service, error) {
if h.caps.surfaces == nil {
return nil, surfacesvc.ErrUnavailable
}
return h.caps.surfaces, nil
}
func (h inprocHost) SurfaceSave(ctx context.Context, req extension.SurfaceSaveRequest) (core.SurfaceSnapshot, error) {
svc, err := h.surface()
if err != nil {
return core.SurfaceSnapshot{}, err
}
return svc.Submit(ctx, h.owner, h.pluginID, surfacesvc.SubmitRequest{
SurfaceID: req.SurfaceID, ConversationID: req.ConversationID, SourceRunID: req.SourceRunID,
BaseRevision: req.BaseRevision, ObservedEpoch: req.ObservedEpoch,
ContextVersion: req.ContextVersion, SemanticText: req.SemanticText, Spec: req.Spec,
})
}
func (h inprocHost) SurfaceGet(ctx context.Context, surfaceID core.ID) (core.SurfaceSnapshot, error) {
svc, err := h.surface()
if err != nil {
return core.SurfaceSnapshot{}, err
}
return svc.Get(ctx, h.owner, surfaceID)
}
func (h inprocHost) SurfaceList(ctx context.Context, conversationID core.ID) ([]core.SurfaceMeta, error) {
svc, err := h.surface()
if err != nil {
return nil, err
}
return svc.List(ctx, h.owner, conversationID)
}
func (h inprocHost) SurfaceActivation(ctx context.Context) (bool, int64, error) {
svc, err := h.surface()
if err != nil {
return false, 0, err
}
row, err := svc.Activation(ctx, h.owner, h.pluginID)
return row.Enabled, row.Epoch, err
}
+111
View File
@@ -1,6 +1,8 @@
package app
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"path/filepath"
@@ -193,3 +195,112 @@ func TestInprocHostKnowledgeWriteEmbedHash(t *testing.T) {
t.Fatalf("vector owner leak: %+v", bhits)
}
}
// TestInprocHostParseDocumentOwnerScope:EXT-5 解析方法的 owner 作用域——
// alice 经句柄读自己的工件并解析;bob 用同一工件 ID 在 ArtifactContent 处
// 即被拒(解析是纯计算,作用域边界在取原件一环)。
func TestInprocHostParseDocumentOwnerScope(t *testing.T) {
extension.ResetCompiled()
t.Cleanup(extension.ResetCompiled)
extension.RegisterCompiled(extension.InProc{
ID: "tst",
Register: func(reg extension.Registry) error {
return reg.AddCommandHandler(json.RawMessage(`{"id":"tst.parse","title":"解析"}`),
func(ctx context.Context, h extension.Host, req extension.Request) (any, error) {
id, _ := req.Params["artifactId"].(string)
content, err := h.ArtifactContent(id)
if err != nil {
return nil, err
}
doc, err := h.ParseDocument(ctx, "报价.xlsx", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", content)
if err != nil {
return nil, err
}
return map[string]any{
"parser": doc.Meta.Parser,
"warnings": len(doc.Warnings),
"blocks": len(doc.Blocks),
}, nil
})
},
})
bus, st := newInprocTestApp(t, nil)
art := core.Artifact{ProjectID: "p1", Owner: "alice", Name: "报价.xlsx", Kind: "document"}
if err := st.CreateArtifact(&art, testXLSXBytes(t)); err != nil {
t.Fatal(err)
}
artID := art.ID
resp := bus.Dispatch(context.Background(), cmdbus.Request{
ID: "tst.parse", Params: map[string]any{"artifactId": artID}, Owner: "alice", Entrance: "chat",
})
if resp.Receipt.State != agentcommand.StateCommitted {
t.Fatalf("alice dispatch: %+v %s", resp.Receipt, resp.Receipt.Reason)
}
m, _ := resp.Payload.(map[string]any)
if m["parser"] != "builtin:xlsx" || m["warnings"] != 0 || m["blocks"] == 0 {
t.Fatalf("alice payload: %+v", m)
}
// bob 同一工件 ID:取原件即被拒(跨 owner 不可见)。
resp = bus.Dispatch(context.Background(), cmdbus.Request{
ID: "tst.parse", Params: map[string]any{"artifactId": artID}, Owner: "bob", Entrance: "chat",
})
if resp.Receipt.State == agentcommand.StateCommitted {
t.Fatalf("bob must not read alice's artifact: %+v", resp.Payload)
}
}
// TestInprocHostParseDocumentLimits:仅有的两条 error 路径——ctx 取消与输入
// 超过 100MiB 上限;不可解析格式仍是带 Warnings 的显式失败。
func TestInprocHostParseDocumentLimits(t *testing.T) {
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "t.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
h := inprocHost{caps: hostcaps{st: st}, pluginID: "tst", owner: "alice"}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := h.ParseDocument(ctx, "a.xlsx", "", []byte("x")); err == nil {
t.Fatal("canceled ctx must error")
}
if _, err := h.ParseDocument(context.Background(), "a.xlsx", "", make([]byte, maxParseInput+1)); err == nil {
t.Fatal("oversized input must error")
}
doc, err := h.ParseDocument(context.Background(), "旧.xls", "", []byte{0xD0, 0xCF})
if err != nil {
t.Fatalf("legacy format is a visible failure, not an error: %v", err)
}
if doc.Meta.Parser != "builtin:legacy-office" || len(doc.Warnings) != 1 {
t.Fatalf("legacy doc: %+v %v", doc.Meta, doc.Warnings)
}
}
// testXLSXBytes 构造最小可用 xlsx(单表两行),供句柄测试解析。
func testXLSXBytes(t *testing.T) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
files := map[string]string{
"xl/workbook.xml": `<workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"><sheets><sheet name="报价" sheetId="1" r:id="rId1"/></sheets></workbook>`,
"xl/_rels/workbook.xml.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet1.xml"/></Relationships>`,
"xl/worksheets/sheet1.xml": `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData>
<row r="1"><c r="A1" t="inlineStr"><is><t>物料</t></is></c><c r="B1" t="inlineStr"><is><t>单价</t></is></c></row>
<row r="2"><c r="A2" t="inlineStr"><is><t>支架</t></is></c><c r="B2"><v>12.5</v></c></row>
</sheetData></worksheet>`,
}
for _, name := range []string{"xl/workbook.xml", "xl/_rels/workbook.xml.rels", "xl/worksheets/sheet1.xml"} {
w, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
w.Write([]byte(files[name]))
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
+44
View File
@@ -0,0 +1,44 @@
package app
import (
"testing"
"agent-system-backend/internal/settings"
core "agent-system-core"
)
// 慢通道开关默认关(行为与 I1 一致的机器证明):
// 两条 runtime.intentSemantic / runtime.intentGenerator 必须注册且默认 false。
func TestIntentSlowLaneFlagsDefaultOff(t *testing.T) {
seen := map[string]core.SettingDesc{}
for _, d := range settings.Schema() {
seen[d.Key] = d
}
for _, key := range []string{"runtime.intentSemantic", "runtime.intentGenerator"} {
d, ok := seen[key]
if !ok {
t.Fatalf("设置未注册:%s", key)
}
if d.Type != core.SettingBool {
t.Fatalf("%s 类型应为 bool,得到 %v", key, d.Type)
}
if def, isBool := d.Default.(bool); !isBool || def {
t.Fatalf("%s 必须默认 false,得到 %v", key, d.Default)
}
}
}
// 遥测开关默认安全(N3 WP4):otlpEndpoint 默认空(不导出,保持现状),
// serviceName 有非空默认。两处都注册进 Schema。
func TestTelemetryFlagsDefaults(t *testing.T) {
seen := map[string]core.SettingDesc{}
for _, d := range settings.Schema() {
seen[d.Key] = d
}
if d, ok := seen["telemetry.otlpEndpoint"]; !ok || d.Default != "" {
t.Fatalf("otlpEndpoint 必须注册且默认空:%+v", d)
}
if d, ok := seen["telemetry.serviceName"]; !ok || d.Default != "agent-system-backend" {
t.Fatalf("serviceName 必须注册且默认 agent-system-backend:%+v", d)
}
}
@@ -0,0 +1,166 @@
package app
import (
"agent-system-backend/internal/rag"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
core "agent-system-core"
"context"
"testing"
)
func TestHostKnowledgeV2AndProjectReplacement(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
pipe := rag.NewPipeline(st, settings.NewService(st), nil, nil)
auth := &projectAuthz{}
alice := inprocHost{owner: "alice", pluginID: "rdm", caps: hostcaps{st: st, pipe: pipe, authz: auth}}
first := []core.KnowledgeDoc{{ID: "slice-1", ProjectID: "domain-1", Ref: "notes.md", Chunk: "sharedneedle one"}}
second := []core.KnowledgeDoc{{ID: "slice-2", ProjectID: "domain-2", Ref: "notes.md", Chunk: "sharedneedle two"}}
for _, ds := range [][]core.KnowledgeDoc{first, second, second} {
if err := alice.KnowledgePublish(context.Background(), ds, false); err != nil {
t.Fatal(err)
}
}
own, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "sharedneedle"})
if err != nil || len(own) != 2 {
t.Fatalf("cross project publish replaced first: %+v %v", own, err)
}
auth.set("rdm", func(string, core.ID) (bool, error) { return true, nil })
auth.setV2("rdm", func(q extension.ResourceAuthzQuery) (bool, error) {
return q.ResourceType == "project" || (q.ResourceType == "knowledge" && q.ResourceID == "slice-1" && q.ProjectID == "domain-1" && q.Actor == "bob" && q.Owner == "alice"), nil
})
bob := inprocHost{owner: "bob", pluginID: "rdm", caps: alice.caps}
hits, err := bob.KnowledgeSearch(context.Background(), "", "sharedneedle", 5)
if err != nil || len(hits) != 1 || hits[0].Doc.ID != "slice-1" {
t.Fatalf("host V2=%+v %v", hits, err)
}
}
func TestHostKnowledgeResolvesTrustedProjectScope(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if err := st.AddKnowledge(core.KnowledgeDoc{ID: "slice-current", Owner: "alice", ProjectID: "domain-current", Chunk: "scopeneedle current"}, core.KnowledgeDoc{ID: "slice-other", Owner: "alice", ProjectID: "domain-other", Chunk: "scopeneedle other"}); err != nil {
t.Fatal(err)
}
slot := &projectLifecycleSlot{}
provider := completeLifecycle()
provider.List = func(_ context.Context, h extension.Host) ([]core.Project, error) {
if h.Owner() != "bob" {
t.Fatalf("wrong scope actor %q", h.Owner())
}
return []core.Project{{ID: "host-current", Settings: map[string]any{"rdm.projectId": "domain-current", "rdm.state": "ready"}}}, nil
}
caps := hostcaps{st: st, pipe: rag.NewPipeline(st, settings.NewService(st), nil, nil), authz: &projectAuthz{}, projects: slot}
if err := (pluginRegistry{pluginID: "rdm", caps: caps}).RegisterProjectLifecycle(provider); err != nil {
t.Fatal(err)
}
caps.authz.setV2("rdm", func(extension.ResourceAuthzQuery) (bool, error) { return true, nil })
h := inprocHost{owner: "bob", pluginID: "rdm", caps: caps}
hits, err := h.KnowledgeSearch(context.Background(), "host-current", "scopeneedle", 5)
if err != nil || len(hits) != 1 || hits[0].Doc.ID != "slice-current" {
t.Fatalf("host scope: %+v %v", hits, err)
}
}
func TestRegisteredProjectHostRejectsForeignKnowledgeSource(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
slot := &projectLifecycleSlot{}
caps := hostcaps{st: st, projects: slot}
if err := (pluginRegistry{pluginID: "rdm", caps: caps}).RegisterProjectLifecycle(completeLifecycle()); err != nil {
t.Fatal(err)
}
h := inprocHost{owner: "alice", pluginID: "other", caps: caps}
for _, publish := range []bool{false, true} {
docs := []core.KnowledgeDoc{{ID: "claimed-published", ProjectID: "domain", Source: "plugin:rdm:published", Ref: "ref", Chunk: "spoofneedle"}}
var err error
if publish {
err = h.KnowledgePublish(context.Background(), docs, false)
} else {
err = h.KnowledgeWrite(context.Background(), docs, false)
}
if err == nil {
t.Fatalf("foreign namespace accepted publish=%v", publish)
}
}
if hits, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "spoofneedle"}); err != nil || len(hits) != 0 {
t.Fatalf("foreign source wrote data: %+v %v", hits, err)
}
// Existing ordinary Hosts preserve their documented caller-provided Source.
h.caps.projects = nil
if err := h.KnowledgeWrite(context.Background(), []core.KnowledgeDoc{{Source: "manual", Chunk: "ordinaryneedle"}}, false); err != nil {
t.Fatalf("ordinary source semantics changed: %v", err)
}
}
func TestProjectPublishPreservesForeignPluginAndScope(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
slot := &projectLifecycleSlot{}
caps := hostcaps{st: st, projects: slot}
if err := (pluginRegistry{pluginID: "rdm", caps: caps}).RegisterProjectLifecycle(completeLifecycle()); err != nil {
t.Fatal(err)
}
original := core.KnowledgeDoc{ID: "rdm-published", Owner: "alice", ProjectID: "domain", Source: "plugin:rdm:published", Ref: "notes.md", Chunk: "originalneedle"}
if err := st.AddKnowledge(original); err != nil {
t.Fatal(err)
}
if err := st.UpsertKnowledgeVector(context.Background(), "alice", original.ID, original.Ref, []float32{1, 0}); err != nil {
t.Fatal(err)
}
h := inprocHost{owner: "alice", pluginID: "other", caps: caps}
replacement := []core.KnowledgeDoc{{ID: "other", ProjectID: "domain", Source: "plugin:other", Ref: "notes.md", Chunk: "replacementneedle"}}
for i := 0; i < 2; i++ {
if err := h.KnowledgePublish(context.Background(), replacement, false); err != nil {
t.Fatal(err)
}
}
if hits, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "originalneedle"}); err != nil || len(hits) != 1 {
t.Fatalf("foreign plugin deleted RDM publication: %+v %v", hits, err)
}
if hits, err := st.SearchKnowledgeVectors(context.Background(), "alice", []float32{1, 0}, 5, core.RetrievalQuery{ProjectID: "domain"}); err != nil || len(hits) != 1 || hits[0].Doc.ID != original.ID {
t.Fatalf("foreign plugin deleted RDM vector: %+v %v", hits, err)
}
h.pluginID = "rdm"
if err := h.KnowledgePublish(context.Background(), []core.KnowledgeDoc{{ID: "unscoped", Source: "plugin:rdm", Ref: "notes.md", Chunk: "privateneedle"}}, false); err != nil {
t.Fatal(err)
}
if hits, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "originalneedle", ProjectID: "domain"}); err != nil || len(hits) != 1 {
t.Fatalf("unscoped publication deleted scoped RDM data: %+v %v", hits, err)
}
}
func TestUnregisteredPublishRetainsLegacyRefScope(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
if err := st.AddKnowledge(core.KnowledgeDoc{ID: "legacy", Owner: "alice", ProjectID: "old-project", Source: "manual", Ref: "same", Chunk: "legacyneedle"}); err != nil {
t.Fatal(err)
}
h := inprocHost{owner: "alice", pluginID: "other", caps: hostcaps{st: st}}
if err := h.KnowledgePublish(context.Background(), []core.KnowledgeDoc{{ID: "replacement", Source: "manual", Ref: "same", Chunk: "replacementneedle"}}, false); err != nil {
t.Fatal(err)
}
if hits, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "legacyneedle"}); err != nil || len(hits) != 0 {
t.Fatalf("unregistered legacy scope changed: %+v %v", hits, err)
}
if hits, err := st.SearchKnowledge("alice", core.RetrievalQuery{Text: "replacementneedle"}); err != nil || len(hits) != 1 {
t.Fatalf("unregistered replacement missing: %+v %v", hits, err)
}
}
+124
View File
@@ -0,0 +1,124 @@
package app
import (
"agent-system-backend/internal/projectscope"
"agent-system-backend/internal/server"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
core "agent-system-core"
"context"
"errors"
"sync"
)
type projectLifecycleSlot struct {
mu sync.Mutex
holder string
provider *extension.ProjectLifecycle
}
func (r compiledRegistry) RegisterProjectLifecycle(extension.ProjectLifecycle) error {
return errors.New("extension: declarative plugins cannot register project lifecycle")
}
func (r pluginRegistry) RegisterProjectLifecycle(p extension.ProjectLifecycle) error {
if r.caps.projects == nil || r.pluginID == "" {
return errors.New("extension: project lifecycle slot unavailable")
}
if p.List == nil || p.Create == nil || p.Rename == nil || p.Archive == nil || p.Work == nil {
return errors.New("extension: project lifecycle requires all callbacks")
}
slot := r.caps.projects
slot.mu.Lock()
defer slot.mu.Unlock()
if slot.provider != nil {
return errors.New("extension: project lifecycle already registered")
}
slot.holder, slot.provider = r.pluginID, &p
return nil
}
func (r recordRegistry) RegisterProjectLifecycle(p extension.ProjectLifecycle) error {
reg, ok := r.Registry.(extension.ProjectLifecycleRegistry)
if !ok {
return errors.New("extension: project lifecycle registration unavailable")
}
return reg.RegisterProjectLifecycle(p)
}
func (slot *projectLifecycleSlot) wire(srv *server.Server, caps hostcaps) {
slot.mu.Lock()
defer slot.mu.Unlock()
if slot.provider == nil {
return
}
holder := slot.holder
srv.SetProjectLifecycle(*slot.provider, func(owner string) extension.Host {
return inprocHost{caps: caps, owner: owner, actor: owner, pluginID: holder}
})
}
func (slot *projectLifecycleSlot) active() bool {
slot.mu.Lock()
defer slot.mu.Unlock()
return slot.provider != nil
}
func (slot *projectLifecycleSlot) work() func(context.Context, string, core.ID) (bool, error) {
slot.mu.Lock()
defer slot.mu.Unlock()
if slot.provider == nil {
return nil
}
return slot.provider.Work
}
func (slot *projectLifecycleSlot) resolve(ctx context.Context, caps hostcaps, owner string, id core.ID) (core.ID, error) {
slot.mu.Lock()
if slot.provider == nil {
slot.mu.Unlock()
return id, nil
}
p, holder := *slot.provider, slot.holder
slot.mu.Unlock()
return projectscope.Resolve(ctx, inprocHost{caps: caps, owner: owner, actor: owner, pluginID: holder}, p, id)
}
var _ extension.ProjectHost = inprocHost{}
func (h inprocHost) OwnedProjects() ([]core.Project, error) {
ps, err := h.caps.st.ListProjects(h.owner)
for i := range ps {
ps[i].Owner = h.owner
}
return ps, err
}
func (h inprocHost) OwnedProject(id string) (core.Project, error) {
return h.caps.st.GetProject(h.owner, id)
}
func (h inprocHost) ProjectMetadata(id string) (core.Project, error) {
p, err := h.caps.st.ProjectMetadata(id)
if err != nil {
return core.Project{}, err
}
if h.caps.projects != nil && h.caps.projects.active() {
p.Settings = h.caps.set.MaskedProjectMetadata(p.Settings)
}
if p.Owner == h.owner {
return p, nil
}
if h.caps.authz == nil {
return core.Project{}, store.ErrNotFound
}
ok, err := h.caps.authz.AuthorizeResource(extension.ResourceAuthzQuery{Actor: h.owner, Action: "read", ResourceType: "project", ResourceID: core.ID(id), Owner: p.Owner, ProjectID: core.ID(id)})
if err != nil || !ok {
return core.Project{}, store.ErrNotFound
}
return p, nil
}
func (h inprocHost) EnsureProject(id, name string) (core.Project, error) {
return h.caps.st.EnsureProject(h.owner, id, name)
}
func (h inprocHost) RenameOwnedProject(id, name string) (core.Project, error) {
return h.caps.st.RenameOwnedProject(h.owner, id, name)
}
func (h inprocHost) ProjectNameAvailable(name, exceptID string) (bool, error) {
return h.caps.st.ProjectNameAvailable(name, exceptID)
}
+163
View File
@@ -0,0 +1,163 @@
package app
import (
"agent-system-backend/internal/server"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
core "agent-system-core"
"bytes"
"context"
"encoding/json"
"errors"
"net/http/httptest"
"testing"
)
func completeLifecycle() extension.ProjectLifecycle {
return extension.ProjectLifecycle{
List: func(context.Context, extension.Host) ([]core.Project, error) { return nil, nil },
Create: func(context.Context, extension.Host, string) (core.Project, error) { return core.Project{}, nil },
Rename: func(context.Context, extension.Host, string, string) (core.Project, error) {
return core.Project{}, nil
},
Archive: func(context.Context, extension.Host, string) error { return nil },
Work: func(context.Context, string, core.ID) (bool, error) { return true, nil },
}
}
func TestProjectLifecycleOptionalRegistration(t *testing.T) {
slot := &projectLifecycleSlot{}
reg := pluginRegistry{pluginID: "rdm", caps: hostcaps{projects: slot}}
if err := reg.RegisterProjectLifecycle(extension.ProjectLifecycle{}); err == nil {
t.Fatal("incomplete registration accepted")
}
if err := reg.RegisterProjectLifecycle(completeLifecycle()); err != nil {
t.Fatal(err)
}
if err := reg.RegisterProjectLifecycle(completeLifecycle()); err == nil {
t.Fatal("duplicate registration accepted")
}
if err := (compiledRegistry{}).RegisterProjectLifecycle(completeLifecycle()); err == nil {
t.Fatal("declarative callback accepted")
}
rr := recordRegistry{Registry: pluginRegistry{pluginID: "other", caps: hostcaps{projects: &projectLifecycleSlot{}}}}
optional, ok := any(rr).(extension.ProjectLifecycleRegistry)
if !ok {
t.Fatal("record registry hides optional registration")
}
if err := optional.RegisterProjectLifecycle(completeLifecycle()); err != nil {
t.Fatal(err)
}
}
func TestProjectHostMetadataDoesNotGrantMutation(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
alice := inprocHost{owner: "alice", caps: hostcaps{st: st}}
p, err := alice.EnsureProject("stable", "Shared")
if err != nil {
t.Fatal(err)
}
auth := &projectAuthz{}
auth.set("rdm", func(owner string, id core.ID) (bool, error) { return owner == "bob" && id == core.ID(p.ID), nil })
bob := inprocHost{owner: "bob", caps: hostcaps{st: st, authz: auth}}
if _, err := bob.ProjectMetadata(p.ID); err != nil {
t.Fatal(err)
}
if _, err := bob.OwnedProject(p.ID); !errors.Is(err, store.ErrNotFound) {
t.Fatalf("owned lookup=%v", err)
}
if _, err := bob.RenameOwnedProject(p.ID, "Other"); !errors.Is(err, store.ErrNotFound) {
t.Fatalf("shared rename=%v", err)
}
if _, err := bob.EnsureProject(p.ID, p.Name); !errors.Is(err, store.ErrNotFound) {
t.Fatalf("shared ensure=%v", err)
}
auth.setV2("rdm", func(q extension.ResourceAuthzQuery) (bool, error) { return false, nil })
if _, err := bob.ProjectMetadata(p.ID); !errors.Is(err, store.ErrNotFound) {
t.Fatalf("V2 denial ignored=%v", err)
}
}
func TestProjectHostMetadataMasksSharedSecretsWithoutMutation(t *testing.T) {
st, err := store.OpenSQLite(t.TempDir() + "/t.db")
if err != nil {
t.Fatal(err)
}
defer st.Close()
p, err := st.CreateProject("alice", "Shared")
if err != nil {
t.Fatal(err)
}
raw := map[string]any{"models": map[string]any{"chat": map[string]any{"apiKey": "fixture-nested-secret"}}, "models.embedding.apiKey": "fixture-flat-secret", "legacy.token": "fixture-legacy-secret", "plugins.private": map[string]any{"credential": "fixture-schema-secret"}, "display": map[string]any{"caption": "Shared configuration"}, "rdm.projectId": "canonical", "rdm.state": "ready"}
if err := st.UpdateProjectSettings("alice", p.ID, raw); err != nil {
t.Fatal(err)
}
svc := settings.NewService(st)
svc.SetExtraSchema([]core.SettingDesc{{Key: "plugins.private.credential", Secret: true}})
slot := &projectLifecycleSlot{}
auth := &projectAuthz{}
caps := hostcaps{st: st, set: svc, projects: slot, authz: auth}
provider := completeLifecycle()
provider.List = func(_ context.Context, h extension.Host) ([]core.Project, error) {
project, err := h.(extension.ProjectHost).ProjectMetadata(p.ID)
return []core.Project{project}, err
}
if err := (pluginRegistry{pluginID: "rdm", caps: caps}).RegisterProjectLifecycle(provider); err != nil {
t.Fatal(err)
}
auth.setV2("rdm", func(q extension.ResourceAuthzQuery) (bool, error) {
return q.Actor == "bob" && q.ResourceType == "project" && q.ResourceID == p.ID, nil
})
for _, actor := range []string{"bob", "alice"} {
h := inprocHost{owner: actor, caps: caps}
view, err := h.ProjectMetadata(p.ID)
if err != nil {
t.Fatal(err)
}
encoded, _ := json.Marshal(view.Settings)
for _, value := range []string{"fixture-nested-secret", "fixture-flat-secret", "fixture-legacy-secret", "fixture-schema-secret"} {
if bytes.Contains(encoded, []byte(value)) {
t.Fatalf("%s metadata exposed a fixture credential", actor)
}
}
if view.Settings["rdm.projectId"] != "canonical" || view.Settings["rdm.state"] != "ready" || view.Settings["display"].(map[string]any)["caption"] != "Shared configuration" {
t.Fatal("allowed service/configuration metadata changed")
}
}
// Exercise the real registry -> actor-scoped Host -> public HTTP list chain.
srv := server.New(st, nil, nil, nil, svc, nil)
srv.Auth = server.NewAuthenticator(st, true)
slot.wire(srv, caps)
key, err := st.CreateAPIKey("bob", "test")
if err != nil {
t.Fatal(err)
}
request := httptest.NewRequest("GET", "/api/projects", nil)
request.Header.Set("Authorization", "Bearer "+key)
response := httptest.NewRecorder()
srv.Handler().ServeHTTP(response, request)
if response.Code != 200 {
t.Fatalf("member list=%d", response.Code)
}
for _, value := range []string{"fixture-nested-secret", "fixture-flat-secret", "fixture-legacy-secret", "fixture-schema-secret"} {
if bytes.Contains(response.Body.Bytes(), []byte(value)) {
t.Fatal("actor-scoped lifecycle list exposed a fixture credential")
}
}
if !bytes.Contains(response.Body.Bytes(), []byte(`"rdm.projectId":"canonical"`)) {
t.Fatal("actor-scoped list lost canonical metadata")
}
owned, err := (inprocHost{owner: "alice", caps: caps}).OwnedProject(p.ID)
if err != nil {
t.Fatal(err)
}
before, _ := json.Marshal(raw)
after, _ := json.Marshal(owned.Settings)
if !bytes.Equal(before, after) {
t.Fatal("metadata read mutated persisted owner settings")
}
}
+73
View File
@@ -0,0 +1,73 @@
package app
// EXT-8b 资源级授权槽位测试(U9 首切片):
// 1. V2 未注册 → AuthorizeResource 回落 V1(owner×project 兼容现行);
// 2. V2 注册 → 按查询六要素咨询;受信字段缺失(actor/action/type/id 空)=拒绝;
// 3. V2 返回 error = 拒绝(fail-closed);
// 4. 首注册者生效(后来者忽略);
// 5. compiledRegistry 的 V2 注册被拒(声明式无宿主回调)。
import (
"errors"
"testing"
"agent-system-backend/pkg/extension"
)
func TestAuthorizeResourceV2FallbackAndQuery(t *testing.T) {
var a projectAuthz
// 未注册任何适配器 → 拒绝。
if ok, err := a.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u1", Action: "download", ResourceType: "artifact", ResourceID: "a1",
Owner: "u2", ProjectID: "p1"}); ok || err != nil {
t.Fatalf("未注册应拒绝, got %v %v", ok, err)
}
// 仅 V1:回落 owner×project 布尔。
a.set("p1", func(owner string, projectID string) (bool, error) { return true, nil })
if ok, _ := a.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u1", Action: "download", ResourceType: "artifact", ResourceID: "a1",
Owner: "u2", ProjectID: "p1"}); !ok {
t.Fatal("V1 允许时回落应放行")
}
// V1 路径要求 owner/project 非空。
if ok, _ := a.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u1", Action: "download", ResourceType: "artifact", ResourceID: "a1"}); ok {
t.Fatal("V1 回落在无项目私域应拒绝")
}
// 注册 V2 → 按六要素咨询。
a.setV2("p1", func(q extension.ResourceAuthzQuery) (bool, error) {
return q.Actor == "u1" && q.Action == "download" && q.ResourceType == "artifact" &&
q.ResourceID == "a1" && q.Owner == "u2" && q.ProjectID == "p1", nil
})
if ok, _ := a.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u1", Action: "download", ResourceType: "artifact", ResourceID: "a1",
Owner: "u2", ProjectID: "p1"}); !ok {
t.Fatal("V2 六要素齐备应放行")
}
// 六要素缺一(action 空)=拒绝(不可判定)。
if ok, _ := a.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u1", ResourceType: "artifact", ResourceID: "a1", Owner: "u2", ProjectID: "p1"}); ok {
t.Fatal("受信字段缺失应拒绝")
}
// V2 出错=拒绝(fail-closed)。
a2 := projectAuthz{}
a2.setV2("p1", func(extension.ResourceAuthzQuery) (bool, error) { return false, errors.New("boom") })
if ok, err := a2.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u", Action: "read", ResourceType: "x", ResourceID: "1"}); ok || err == nil {
t.Fatal("V2 error 应拒绝")
}
// 首注册者生效。
a3 := projectAuthz{}
a3.setV2("first", func(extension.ResourceAuthzQuery) (bool, error) { return true, nil })
a3.setV2("second", func(extension.ResourceAuthzQuery) (bool, error) { return false, nil })
if ok, _ := a3.AuthorizeResource(extension.ResourceAuthzQuery{
Actor: "u", Action: "read", ResourceType: "x", ResourceID: "1"}); !ok {
t.Fatal("首个 V2 注册者应生效")
}
}
func TestCompiledRegistryRejectsV2(t *testing.T) {
r := compiledRegistry{}
if err := r.RegisterResourceAuthorizerV2(func(extension.ResourceAuthzQuery) (bool, error) { return true, nil }); err == nil {
t.Fatal("声明式扩展不得注册资源级授权回调(fail-closed)")
}
}
+57
View File
@@ -0,0 +1,57 @@
package app
import (
"context"
"path/filepath"
"testing"
"agent-system-backend/internal/rag"
"agent-system-backend/internal/settings"
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
)
// EXT-11 红测:注入前 NewScopedHost fail-closed;注入后重建句柄与分发期同构
// (owner 作用域、PluginID、KV 隔离、知识删除前缀强制)。
func TestScopedHostFactory(t *testing.T) {
// 注入前:fail-closed。
if _, err := extension.NewScopedHost("alice", "rdm"); err == nil {
t.Fatal("before wiring, NewScopedHost must fail closed(工厂未注入)")
}
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "t.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
set := settings.NewService(st)
pipe := rag.NewPipeline(st, set, nil, nil)
caps := hostcaps{st: st, pipe: pipe, set: set}
// 组合根同款注入(app.Run 内以 wireScopedHostFactory 接线;此处直接调内部函数)。
wireScopedHostFactory(caps)
t.Cleanup(func() { extension.SetScopedHostFactory(nil) })
h, err := extension.NewScopedHost("alice", "rdm")
if err != nil {
t.Fatalf("NewScopedHost: %v", err)
}
if h.Owner() != "alice" || h.PluginID() != "rdm" {
t.Fatalf("Owner/PluginID = %s/%s, want alice/rdm", h.Owner(), h.PluginID())
}
// KV 往返落 owner 作用域。
if err := h.KVPut("k1", "v1"); err != nil {
t.Fatal(err)
}
if v, ok, err := h.KVGet("k1"); err != nil || !ok || v != "v1" {
t.Fatalf("KVGet = %q ok=%v err=%v", v, ok, err)
}
// 同构性:知识删除前缀强制(manual 不可及)。
if err := h.KnowledgeDelete(context.Background(), "manual"); err == nil {
t.Fatal("scoped host must keep the plugin:<id> prefix enforcement")
}
// 空 owner/pluginID:拒绝。
if _, err := extension.NewScopedHost("", "rdm"); err == nil {
t.Fatal("empty owner must be rejected")
}
}
+31
View File
@@ -0,0 +1,31 @@
package app
import (
"strings"
"testing"
)
// 协作存储方案 U2(KTD1/R2):服务入口的部署档门禁。
// team 档缺 DSN / 未开认证必须拒绝启动——不创建 SQLite、不进入装配。
func TestTierRefusalTeam(t *testing.T) {
msgs := TierRefusal(Options{Tier: "team", Addr: "127.0.0.1:8788"})
if len(msgs) != 2 {
t.Fatalf("team without dsn/auth: %v", msgs)
}
joined := strings.Join(msgs, ";")
if !strings.Contains(joined, "PostgreSQL DSN") || !strings.Contains(joined, "认证") {
t.Fatalf("refusal wording: %v", msgs)
}
msgs = TierRefusal(Options{Tier: "team", Addr: "127.0.0.1:8788", StorageDSN: "postgres://u:p@h/db", AuthMode: "on"})
if len(msgs) != 0 {
t.Fatalf("team with dsn+auth should pass: %v", msgs)
}
msgs = TierRefusal(Options{Tier: "local", Addr: "127.0.0.1:8788", StorageDSN: "postgres://u:p@h/db"})
if len(msgs) != 1 || !strings.Contains(msgs[0], "local 档不使用") {
t.Fatalf("local with dsn: %v", msgs)
}
msgs = TierRefusal(Options{Addr: "127.0.0.1:8788"})
if len(msgs) != 0 {
t.Fatalf("auto default: %v", msgs)
}
}
+22
View File
@@ -7,6 +7,7 @@ import (
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
)
@@ -19,6 +20,21 @@ type Store interface {
Delete(ref string) error
}
// IsNil 判断对象存储是否未配置。nil 接口,以及放进接口后的带类型空指针
// (例如未设置端点时的 (*S3)(nil)),都算未配置。已构造的实现返回 false。
func IsNil(s Store) bool {
if s == nil {
return true
}
v := reflect.ValueOf(s)
switch v.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
default:
return false
}
}
// Local 是本地目录实现(data/artifacts;多实例时把目录指向共享卷即可)。
type Local struct {
dir string
@@ -32,6 +48,12 @@ func NewLocal(dir string) (*Local, error) {
return &Local{dir: dir}, nil
}
// OpenLocalExisting 绑定既有目录,不 MkdirAll。目录缺失时仍返回可用句柄
// (账户运维只碰 DB;Put 时若目录不存在会失败)。
func OpenLocalExisting(dir string) (*Local, error) {
return &Local{dir: dir}, nil
}
// validID 只接受纯文件名形态的对象 id:非空、不含路径分隔符、非 . / ..。
// Put 的 id 可能是上层透传的用户输入(如工件路径参数),先在此挡住路径穿越。
func validID(id string) bool {
+29
View File
@@ -0,0 +1,29 @@
package blobstore
import "testing"
func TestIsNil(t *testing.T) {
var unset Store
if !IsNil(unset) {
t.Fatal("nil interface must count as unset")
}
var typed *S3
var asStore Store = typed
if !IsNil(asStore) {
t.Fatal("typed nil *S3 must count as unset")
}
local, err := NewLocal(t.TempDir())
if err != nil {
t.Fatal(err)
}
if IsNil(local) {
t.Fatal("local store must count as configured")
}
s3, err := NewS3("http://127.0.0.1:9/bucket", "us-east-1", "access", "secret", "art/")
if err != nil {
t.Fatal(err)
}
if IsNil(s3) {
t.Fatal("configured S3 must count as configured")
}
}
+121
View File
@@ -0,0 +1,121 @@
package blobstore
import (
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
)
// U5/KTD9:S3 兼容对象存储实现——针对嵌入式测试服务器验证
// 签名请求、Put/Get/Delete 语义与预签名 URL 生成。
// (真实 S3 兼容服务的端到端验收随 U7 压测环境执行;此处锁定协议正确性。)
func newTestS3(t *testing.T, base string) *S3 {
t.Helper()
s, err := NewS3(base, "us-east-1", "AKIDEXAMPLE", "secret", "art/")
if err != nil {
t.Fatal(err)
}
return s
}
func TestS3PutGetDeleteRoundTrip(t *testing.T) {
var gotPath, gotMethod, gotAuth string
var gotBody []byte
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath, gotMethod, gotAuth = r.URL.Path, r.Method, r.Header.Get("Authorization")
buf := make([]byte, 4096)
n, _ := r.Body.Read(buf)
gotBody = buf[:n]
switch r.Method {
case http.MethodPut:
w.WriteHeader(200)
case http.MethodGet:
if strings.Contains(gotPath, "obj404") {
w.WriteHeader(404)
return
}
w.Write([]byte("payload"))
case http.MethodDelete:
w.WriteHeader(204)
}
}))
defer srv.Close()
s := newTestS3(t, srv.URL+"/bucket")
ref, err := s.Put("doc1", []byte("payload"))
if err != nil {
t.Fatal(err)
}
if ref != "art/doc1" {
t.Fatalf("ref=%s(对象键应带前缀)", ref)
}
if gotPath != "/bucket/art/doc1" || gotMethod != http.MethodPut || string(gotBody) != "payload" {
t.Fatalf("put 请求:%s %s body=%q", gotMethod, gotPath, gotBody)
}
if len(gotAuth) == 0 || gotAuth[:17] != "AWS4-HMAC-SHA256 " {
t.Fatalf("Authorization 头:%q", gotAuth)
}
got, err := s.Get(ref)
if err != nil || string(got) != "payload" {
t.Fatalf("get: %q err=%v", got, err)
}
if err := s.Delete(ref); err != nil {
t.Fatalf("delete: %v", err)
}
// 404 → 哨兵错误(服务端对 art/obj404 的 GET 返回 404)。
if _, err := s.Get("art/obj404"); err == nil || err.Error() != ErrObjectNotFound.Error() {
t.Fatalf("404 应 ErrObjectNotFound:%v", err)
}
// Delete 幂等(404 视为成功)。
srv2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(404)
}))
defer srv2.Close()
s2 := newTestS3(t, srv2.URL+"/b")
if err := s2.Delete("art/none"); err != nil {
t.Fatalf("delete 404 应幂等:%v", err)
}
}
func TestS3PresignGet(t *testing.T) {
s := newTestS3(t, "https://s3.example.com/bucket")
u, err := s.PresignGet("art/doc1", 15*time.Minute)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"X-Amz-Algorithm=AWS4-HMAC-SHA256", "X-Amz-Signature=", "X-Amz-Expires=900", "/bucket/art/doc1"} {
if !contains(u, want) {
t.Fatalf("预签名 URL 缺 %s:%s", want, u)
}
}
}
func contains(s, sub string) bool {
return len(s) >= len(sub) && (func() bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
})()
}
func TestNewS3FromEnvFallback(t *testing.T) {
os.Unsetenv("BLOBSTORE_S3_ENDPOINT")
s, err := NewS3FromEnv()
if s != nil || err != nil {
t.Fatalf("未配置端点应回落 (nil,nil):s=%v err=%v", s, err)
}
t.Setenv("BLOBSTORE_S3_ENDPOINT", "http://x/b")
t.Setenv("BLOBSTORE_S3_ACCESS_KEY", "a")
t.Setenv("BLOBSTORE_S3_SECRET_KEY", "b")
s, err = NewS3FromEnv()
if err != nil || s == nil {
t.Fatalf("已配置应构造:s=%v err=%v", s, err)
}
}
+242
View File
@@ -0,0 +1,242 @@
package blobstore
import (
"bytes"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
)
// S3 是 S3 兼容对象存储实现(协作存储 U5/KTD9:多实例共享工件)。
// 认证:AWS Signature V4(静态 AccessKey/SecretKey;凭据只经环境或受保护
// 配置注入,引用形态为对象键,不含凭据)。
//
// 语义(KTD9):
// - Put:先完整写对象再由上层提交引用(失败孤儿可清理);
// - Get:按对象键读取;
// - Delete:立即删除(标记/延迟 GC 由桶生命周期策略承担)。
type S3 struct {
Base string // 端点(含 bucket 路径,如 https://s3.example.com/bucket)
Region string
Access string
Secret string
Prefix string // 对象键前缀(租户/应用隔离)
client *http.Client
nowFunc func() time.Time
}
// NewS3 从显式参数构造;nil client 用默认。
func NewS3(base, region, access, secret, prefix string) (*S3, error) {
if base == "" || access == "" || secret == "" {
return nil, errors.New("blobstore: s3 requires base/access/secret")
}
if !strings.HasSuffix(base, "/") {
base += "/"
}
return &S3{Base: base, Region: region, Access: access, Secret: secret, Prefix: prefix,
client: &http.Client{Timeout: 30 * time.Second}, nowFunc: time.Now}, nil
}
// NewS3FromEnv 按环境构造:BLOBSTORE_S3_ENDPOINT/REGION/ACCESS_KEY/SECRET_KEY/PREFIX。
// 未配置端点返回 (nil, nil)(调用方回落本地档)。
func NewS3FromEnv() (*S3, error) {
if os.Getenv("BLOBSTORE_S3_ENDPOINT") == "" {
return nil, nil
}
return NewS3(os.Getenv("BLOBSTORE_S3_ENDPOINT"),
envOr("BLOBSTORE_S3_REGION", "us-east-1"),
os.Getenv("BLOBSTORE_S3_ACCESS_KEY"),
os.Getenv("BLOBSTORE_S3_SECRET_KEY"),
os.Getenv("BLOBSTORE_S3_PREFIX"))
}
func envOr(k, d string) string {
if v := os.Getenv(k); v != "" {
return v
}
return d
}
func (s *S3) key(id string) (string, error) {
if !validID(id) {
return "", fmt.Errorf("blobstore: invalid object id %q", id)
}
return s.Prefix + id, nil
}
// Put 上传对象;ref=对象键。
func (s *S3) Put(id string, content []byte) (string, error) {
key, err := s.key(id)
if err != nil {
return "", err
}
resp, err := s.do(http.MethodPut, key, content, "")
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
return "", fmt.Errorf("blobstore: s3 put %s: %s %s", key, resp.Status, body)
}
return key, nil
}
// Get 下载对象。
func (s *S3) Get(ref string) ([]byte, error) {
resp, err := s.do(http.MethodGet, ref, nil, "")
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotFound {
return nil, ErrObjectNotFound
}
if resp.StatusCode/100 != 2 {
return nil, fmt.Errorf("blobstore: s3 get %s: %s", ref, resp.Status)
}
return io.ReadAll(resp.Body)
}
// ErrObjectNotFound 对象不存在(与本地档 ErrNotExist 对齐的哨兵)。
var ErrObjectNotFound = errors.New("blobstore: object not found")
// Delete 删除对象(幂等:404 视为成功)。
func (s *S3) Delete(ref string) error {
resp, err := s.do(http.MethodDelete, ref, nil, "")
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode/100 == 2 || resp.StatusCode == http.StatusNotFound {
return nil
}
return fmt.Errorf("blobstore: s3 delete %s: %s", ref, resp.Status)
}
// do 签名并执行请求(AWS SigV4,service=s3)。
func (s *S3) do(method, key string, body []byte, contentType string) (*http.Response, error) {
u, err := url.Parse(s.Base + key)
if err != nil {
return nil, err
}
now := s.nowFunc().UTC()
amzDate := now.Format("20060102T150405Z")
dateStamp := now.Format("20060102")
payloadHash := sha256.Sum256(body)
payloadHex := hex.EncodeToString(payloadHash[:])
req, err := http.NewRequest(method, u.String(), bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("x-amz-date", amzDate)
req.Header.Set("x-amz-content-sha256", payloadHex)
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
// 规范请求与签名(SigV4 单 chunk)。
canonicalURI := "/" + strings.TrimPrefix(u.Path, "/")
canonicalQuery := u.RawQuery
var ch bytes.Buffer
ch.WriteString(method + "\n" + canonicalURI + "\n" + canonicalQuery + "\n")
ch.WriteString("host:" + u.Host + "\n")
ch.WriteString("x-amz-content-sha256:" + payloadHex + "\n")
ch.WriteString("x-amz-date:" + amzDate + "\n")
ch.WriteString("\nhost;x-amz-content-sha256;x-amz-date\n" + payloadHex)
cr := ch.Bytes()
scope := dateStamp + "/" + s.Region + "/s3/aws4_request"
stringToSign := "AWS4-HMAC-SHA256\n" + amzDate + "\n" + scope + "\n" + hex.EncodeToString(sha256sum(cr))
kDate := hmacSHA256([]byte("AWS4"+s.Secret), dateStamp)
kRegion := hmacSHA256(kDate, s.Region)
kService := hmacSHA256(kRegion, "s3")
kSigning := hmacSHA256(kService, "aws4_request")
sig := hex.EncodeToString(hmacSHA256(kSigning, stringToSign))
req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential="+s.Access+"/"+scope+
", SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature="+sig)
return s.client.Do(req)
}
func sha256sum(b []byte) []byte {
h := sha256.Sum256(b)
return h[:]
}
func hmacSHA256(key []byte, data string) []byte {
m := hmac.New(sha256.New, key)
m.Write([]byte(data))
return m.Sum(nil)
}
// PresignGet 生成限时 GET 预签名 URL(下载入口;有效期由调用方指定)。
// 用于把大对象下载直接指向对象存储,应用实例不做数据中继。
func (s *S3) PresignGet(ref string, ttl time.Duration) (string, error) {
u, err := url.Parse(s.Base + ref)
if err != nil {
return "", err
}
now := s.nowFunc().UTC()
amzDate := now.Format("20060102T150405Z")
dateStamp := now.Format("20060102")
expiry := int64(ttl / time.Second)
payloadHex := hex.EncodeToString(sha256sum(nil))
q := u.Query()
q.Set("X-Amz-Algorithm", "AWS4-HMAC-SHA256")
q.Set("X-Amz-Credential", s.Access+"/"+dateStamp+"/"+s.Region+"/s3/aws4_request")
q.Set("X-Amz-Date", amzDate)
q.Set("X-Amz-Expires", fmt.Sprint(expiry))
q.Set("X-Amz-SignedHeaders", "host")
u.RawQuery = q.Encode()
canonicalQuery := canonicalQueryEncode(u)
var ch bytes.Buffer
ch.WriteString("GET\n/" + strings.TrimPrefix(u.Path, "/") + "\n" + canonicalQuery + "\n")
ch.WriteString("host:" + u.Host + "\n")
ch.WriteString("x-amz-content-sha256:" + payloadHex + "\n")
ch.WriteString("x-amz-date:" + amzDate + "\n")
ch.WriteString("\nhost;x-amz-content-sha256;x-amz-date\n" + payloadHex)
scope := dateStamp + "/" + s.Region + "/s3/aws4_request"
stringToSign := "AWS4-HMAC-SHA256\n" + amzDate + "\n" + scope + "\n" + hex.EncodeToString(sha256sum(ch.Bytes()))
kDate := hmacSHA256([]byte("AWS4"+s.Secret), dateStamp)
kRegion := hmacSHA256(kDate, s.Region)
kService := hmacSHA256(kRegion, "s3")
kSigning := hmacSHA256(kService, "aws4_request")
sig := hex.EncodeToString(hmacSHA256(kSigning, stringToSign))
return u.String() + "&X-Amz-Signature=" + sig, nil
}
// canonicalQueryEncode 按 AWS 规则重排查询串(键排序 + URL 编码)。
func canonicalQueryEncode(u *url.URL) string {
q := u.Query()
var keys []string
for k := range q {
keys = append(keys, k)
}
// 排序(简单实现足够:键集固定且互不冲突)
for i := 0; i < len(keys); i++ {
for j := i + 1; j < len(keys); j++ {
if keys[j] < keys[i] {
keys[i], keys[j] = keys[j], keys[i]
}
}
}
var sb strings.Builder
for i, k := range keys {
if i > 0 {
sb.WriteString("&")
}
sb.WriteString(url.QueryEscape(k) + "=" + url.QueryEscape(q[k][0]))
}
return sb.String()
}
var _ Store = (*S3)(nil)
+73 -14
View File
@@ -27,15 +27,55 @@ var (
const cardAppendWindow = 250 * time.Millisecond
type Service struct {
st *store.Store
emit Emitter
now func() time.Time
emitMu sync.Mutex
lastEmit map[string]time.Time // 流式追加的最近发布时刻(按卡)
st *store.Store
emit Emitter
projectSharedEmit ProjectSharedEmitter
// MirrorConvCards EXT-12:会话卡是否镜像进项目公共流(缺省 true=现状全量
// 镜像;false 时会话卡只进本人会话流,项目级卡不受影响)。
MirrorConvCards bool
now func() time.Time
emitMu sync.Mutex
lastEmit map[string]time.Time // 流式追加的最近发布时刻(按卡)
}
func New(st *store.Store, emit Emitter) *Service {
return &Service{st: st, emit: emit, now: time.Now, lastEmit: map[string]time.Time{}}
return &Service{st: st, emit: emit, now: time.Now, lastEmit: map[string]time.Time{}, MirrorConvCards: true}
}
// ProjectSharedEmitter is an internal trusted publication adapter. It is never
// bound to model arguments or exposed as a mandatory extension capability.
type ProjectSharedEmitter func(owner, producer, channel, runID string, typ core.StreamEventType, payload any) error
func (s *Service) SetProjectSharedEmitter(emit ProjectSharedEmitter) { s.projectSharedEmit = emit }
// PublishProjectCard explicitly publishes a project-level structural snapshot.
// Ordinary Create/Update remain private unless this entry issues a new proof.
func (s *Service) PublishProjectCard(owner, producer string, card core.ActivityCard, ref string) (core.ActivityCard, error) {
if s.projectSharedEmit == nil || owner == "" || producer == "" || card.ProjectID == "" || card.ConversationID != "" {
return card, errors.New("card: trusted project publisher required")
}
if card.ID != "" {
current, err := s.st.GetCard(owner, card.ID)
if err != nil {
return card, err
}
if current.ProjectID != card.ProjectID || current.ConversationID != "" {
return card, store.ErrNotFound
}
} else {
card.ID = store.NewID("card")
}
now := s.now().UTC()
card.CreatedAt = now
card.UpdatedAt = now
if card.Status == "" {
card.Status = core.CardActive
}
if err := s.st.SaveCard(owner, &card, ref); err != nil {
return card, err
}
err := s.projectSharedEmit(owner, producer, string(core.ProjectChannelKey(card.ProjectID)), card.SourceRunID, core.EvCardCreated, core.CardPayload{Card: card})
return card, err
}
// emitCard 按卡归属发布卡片事件(D50):
@@ -52,6 +92,11 @@ func (s *Service) emitCard(c core.ActivityCard, typ core.StreamEventType, payloa
if err := s.emit(c.ConversationID, c.SourceRunID, typ, payload); err != nil {
return err
}
// EXT-12 最小开关:会话卡不镜像进项目公共流(不发布即不持久,
// 重放天然不含会话卡);项目级卡(无会话归属)不受影响。
if !s.MirrorConvCards {
return nil
}
}
if c.ProjectID != "" {
return s.emit(string(core.ProjectChannelKey(c.ProjectID)), c.SourceRunID, typ, payload)
@@ -87,6 +132,7 @@ func (s *Service) List(owner, conversationID string) ([]core.ActivityCard, error
}
// Update 保存整卡变更(流式 Body 增量等)并发布 card.updated;ref 保持原值。
// 系统镜像路径(同会话 run 的顺序行为)沿用盲写;用户态裁决走 UpdateVersioned。
func (s *Service) Update(owner string, card core.ActivityCard) error {
card.UpdatedAt = s.now().UTC()
if err := s.st.SaveCard(owner, &card, ""); err != nil {
@@ -95,6 +141,16 @@ func (s *Service) Update(owner string, card core.ActivityCard) error {
return s.emitCard(card, core.EvCardUpdated, core.CardPayload{Card: card})
}
// UpdateVersioned 乐观并发保存(U3,KTD7):版本不匹配透传 store.ErrConflict,
// HTTP 层映射 409;客户端重读卡后重试。
func (s *Service) UpdateVersioned(owner string, card core.ActivityCard, expectedVersion int) error {
card.UpdatedAt = s.now().UTC()
if err := s.st.SaveCardVersioned(owner, &card, "", expectedVersion); err != nil {
return err
}
return s.emitCard(card, core.EvCardUpdated, core.CardPayload{Card: card})
}
// Append 流式追加正文(card_append 工具;D45):文本并入 body 尾部 markdown
// 节点(尾节点非 markdown 则新增),逐次持久化;card.updated 按 cardAppendWindow
// 节流——窗口内的追加只落库不发布(投影语义:事件携带整卡,后续事件自然覆盖
@@ -133,7 +189,7 @@ func (s *Service) Append(owner, id, text string) error {
// SetStatus 状态迁移:更新并发布 card.status。
func (s *Service) SetStatus(owner, id string, status core.CardStatus, tone, note string) (core.ActivityCard, error) {
c, err := s.st.GetCard(owner, id)
c, ver, err := s.st.GetCardVersion(owner, id)
if err != nil {
return c, err
}
@@ -141,7 +197,7 @@ func (s *Service) SetStatus(owner, id string, status core.CardStatus, tone, note
if tone != "" {
c.Tone = tone
}
if err := s.Update(owner, c); err != nil {
if err := s.UpdateVersioned(owner, c, ver); err != nil {
return c, err
}
return c, s.emitCard(c, core.EvCardStatus, core.CardStatusPayload{CardID: c.ID, Status: status, Tone: c.Tone, Note: note})
@@ -149,12 +205,12 @@ func (s *Service) SetStatus(owner, id string, status core.CardStatus, tone, note
// SetPinned 置顶/取消(收件箱"不沉没"语义)。
func (s *Service) SetPinned(owner, id string, pinned bool) (core.ActivityCard, error) {
c, err := s.st.GetCard(owner, id)
c, ver, err := s.st.GetCardVersion(owner, id)
if err != nil {
return c, err
}
c.Pinned = pinned
return c, s.Update(owner, c)
return c, s.UpdateVersioned(owner, c, ver)
}
// Dismiss 关闭卡(非确认语义,留痕)。
@@ -165,7 +221,7 @@ func (s *Service) Dismiss(owner, id string) (core.ActivityCard, error) {
// Action 处理卡上按钮点击:仅 action_required 卡可裁决;结果 resolved 并发布
// card.resolved(ActionID 即按钮语义)。
func (s *Service) Action(owner, id, actionID, freeText string) (core.ActivityCard, string, error) {
c, err := s.st.GetCard(owner, id)
c, ver, err := s.st.GetCardVersion(owner, id)
if err != nil {
return c, "", err
}
@@ -189,7 +245,7 @@ func (s *Service) Action(owner, id, actionID, freeText string) (core.ActivityCar
return c, "", ErrNoAction
}
c.Status = core.CardResolved
if err := s.Update(owner, c); err != nil {
if err := s.UpdateVersioned(owner, c, ver); err != nil {
return c, "", err
}
return c, label, s.emitCard(c, core.EvCardResolved, core.CardResolvedPayload{CardID: c.ID, ActionID: actionID, Answer: freeText})
@@ -242,7 +298,7 @@ func (s *Service) ResolveByRef(owner, ref, actionID, answer string) error {
}
return err
}
c, err := s.st.GetCard(owner, id)
c, ver, err := s.st.GetCardVersion(owner, id)
if err != nil {
return err
}
@@ -250,7 +306,10 @@ func (s *Service) ResolveByRef(owner, ref, actionID, answer string) error {
return nil // 已决(双现任一侧先到)
}
c.Status = core.CardResolved
if err := s.Update(owner, c); err != nil {
if err := s.UpdateVersioned(owner, c, ver); err != nil {
if errors.Is(err, store.ErrConflict) {
return nil // 双现任一侧先到(版本竞争败者):不重试,卡以先到侧为准
}
return err
}
if c.ConversationID != "" && s.emit != nil {
+56
View File
@@ -0,0 +1,56 @@
package cardsvc
import (
"testing"
"agent-system-backend/internal/store"
core "agent-system-core"
)
// EXT-12 最小开关(红测):MirrorConvCards=false 时会话卡不镜像进项目公共流
// (不发布即不持久,重放天然不含会话卡);项目级卡不受影响;缺省 true 保持现状。
func TestConvCardMirrorSwitch(t *testing.T) {
newSvc := func(mirror bool) (*Service, *[]string) {
chans := []string{}
emit := func(convID, runID string, typ core.StreamEventType, payload any) error {
chans = append(chans, convID)
return nil
}
st, err := store.OpenSQLite(t.TempDir() + "/cards.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
svc := New(st, emit)
svc.MirrorConvCards = mirror
return svc, &chans
}
card := core.ActivityCard{ConversationID: "c-priv", ProjectID: "p-1", Kind: core.CardKindProgress, Title: "私有过程卡"}
// 开(缺省现状):会话流 + 项目流双发。
svc, chans := newSvc(true)
if _, err := svc.Create("alice", card, ""); err != nil {
t.Fatal(err)
}
if len(*chans) != 2 {
t.Fatalf("mirror on: emitted %v, want 会话流+项目流", *chans)
}
// 关:只进本人会话流,项目公共流零事件。
svc2, chans2 := newSvc(false)
if _, err := svc2.Create("alice", card, ""); err != nil {
t.Fatal(err)
}
if len(*chans2) != 1 || (*chans2)[0] != "c-priv" {
t.Fatalf("mirror off: emitted %v, want 仅会话流 c-priv(项目流不得发布)", *chans2)
}
// 项目级卡(无会话归属)不受开关影响。
pc := core.ActivityCard{ProjectID: "p-1", Kind: core.CardKindProgress, Title: "项目级卡"}
if _, err := svc2.Create("alice", pc, ""); err != nil {
t.Fatal(err)
}
if len(*chans2) != 2 || (*chans2)[1] != string(core.ProjectChannelKey("p-1")) {
t.Fatalf("project-level card must still publish: %v", *chans2)
}
}
+69 -3
View File
@@ -18,9 +18,11 @@ import (
"agent-system-core"
)
// AuditStore 审计落盘端口(store 实现;v7 command_audit)。
// AuditStore 审计落盘端口(store 实现;v7 建表、v18 加 owner 列)。
// owner 是请求租户(req.Owner):审计行按租户归属,查询按 owner 过滤,
// 跨租户审计不可见(信封 Actor 是发起主体,不作归属依据)。
type AuditStore interface {
AppendAudit(env agentcommand.AuditEnvelope) error
AppendAudit(owner string, env agentcommand.AuditEnvelope) error
}
// CardPort 确认卡所需卡片能力(cardsvc.Service 实现)。
@@ -32,6 +34,53 @@ type CardPort interface {
// Handler 宿主执行适配器:同步内置用例返回 payload;异步执行自行返回 accepted 回执。
type Handler func(ctx context.Context, req Request) (any, error)
// CommandTool EXT-14:带 exposeAsTool 声明的命令的工具投影视图(runmgr 消费)。
// Risk 取 low|medium|high(工具权限声明口径);ID 即工具名(命令 id)。
type CommandTool struct {
ID string
Title string
Description string
Params json.RawMessage
Risk string
}
// CommandTools 全部带 ExposeAsTool 的已启用命令(EXT-14 工具投影源)。
// ToolGate 是 owner 级工具注入闸(U3:禁用即不注入能力):返回 false 时该命令
// 不投影给该 owner 的模型。组合根按需装配(如 generative-ui 按 activation 台账)。
type ToolGate func(owner, commandID string) bool
// SetToolGate 装配 owner 级注入闸(组合根;启动期一次)。
func (b *Bus) SetToolGate(g ToolGate) { b.toolGate = g }
func (b *Bus) CommandTools() []CommandTool {
return b.CommandToolsFor("")
}
// CommandToolsFor 按 owner 过滤工具投影(gate 为 nil 时不过滤;owner 为空时
// gated 命令一律不投影——fail-closed)。
func (b *Bus) CommandToolsFor(owner string) []CommandTool {
out := []CommandTool{}
for _, r := range b.reg.List() {
if !r.Enabled || r.Spec.ExposeAsTool == nil {
continue
}
if b.toolGate != nil && !b.toolGate(owner, r.Spec.ID) {
continue
}
risk := r.Spec.ExposeAsTool.Risk
if risk == "" {
risk = "low"
}
desc := r.Spec.Description
if desc == "" {
desc = r.Spec.Title
}
out = append(out, CommandTool{ID: r.Spec.ID, Title: r.Spec.Title,
Description: desc, Params: r.Spec.ParamsSchema, Risk: risk})
}
return out
}
// Request 一次分发请求(入口归一后的统一形态)。
type Request struct {
ID string
@@ -42,6 +91,7 @@ type Request struct {
ConversationID string
AnchorMessageID string // 确认卡锚定的发起消息(内联渲染)
Confirmed bool // 危险命令经确认卡放行后的执行
HostProjectID string // EXT-18:分发所属基座项目(空=无项目上下文)
}
// Response 分发结果:回执 + 处理负载 + 可能的确认卡。
@@ -67,6 +117,8 @@ type Bus struct {
pendMu sync.Mutex
pending map[string]pending
pendingStore PendingStore
intents *IntentStore // G2-2:意图声明表(自然语言路由)
toolGate ToolGate // U3:owner 级工具注入闸(nil = 不过滤)
}
// PendingStore 把危险命令的确认挂起态落盘。进程内 map 仍是热路径;重启后从存储取回。
@@ -82,6 +134,7 @@ func New(audit AuditStore, cards CardPort, now func() time.Time) *Bus {
return &Bus{
reg: agentcommand.NewRegistry(), audit: audit, cards: cards, now: now,
handlers: map[string]Handler{}, slash: map[string]string{}, pending: map[string]pending{},
intents: NewIntentStore(),
}
}
@@ -136,10 +189,19 @@ func (b *Bus) RegisterPluginCommands(overlay []json.RawMessage) error {
if err := b.reg.Register(spec, agentcommand.SourcePlugin); err != nil {
return fmt.Errorf("cmdbus: register %s: %w", spec.ID, err)
}
b.bindPluginSlash(spec)
}
return nil
}
// bindPluginSlash EXT-10②:插件命令规格自带的 slash 别名绑定(与内置命令同一张
// 别名表——四入口同源:slash 命中即同一命令 id、同一审计信封)。
func (b *Bus) bindPluginSlash(spec agentcommand.CommandSpec) {
if spec.Slash != "" {
b.slash[strings.ToLower(spec.Slash)] = spec.ID
}
}
// RegisterPluginCommand 注册进程内插件命令(EXT-1):登记规格并绑定宿主处理器。
// 仅登记规格的声明式路径仍走 RegisterPluginCommands(无 handler,分发即 rejected)。
func (b *Bus) RegisterPluginCommand(spec agentcommand.CommandSpec, h Handler) error {
@@ -149,9 +211,13 @@ func (b *Bus) RegisterPluginCommand(spec agentcommand.CommandSpec, h Handler) er
if h != nil {
b.handlers[spec.ID] = h
}
b.bindPluginSlash(spec)
return nil
}
// Intents 意图声明表(G2-2:注册面与路由查询共用)。
func (b *Bus) Intents() *IntentStore { return b.intents }
// ResolveSlash `/` 入口解析:返回命令 id 与原始字段(参数由宿主处理器按命令语义解析)。
func (b *Bus) ResolveSlash(cmd string) (string, bool) {
id, ok := b.slash[strings.ToLower(strings.TrimSpace(cmd))]
@@ -275,7 +341,7 @@ func (b *Bus) auditReceipt(req Request, state agentcommand.ResultState, op, reas
if _, src, err := b.reg.Resolve(req.ID); err == nil {
env.Source = src
}
if err := b.audit.AppendAudit(env); err != nil {
if err := b.audit.AppendAudit(req.Owner, env); err != nil {
// 审计不可用:结果标记未知,不臆断成功(四态语义)。
return agentcommand.ExecReceipt{CommandID: req.ID, State: agentcommand.StateUnknown, Reason: "audit unavailable: " + err.Error(), Time: b.now().UTC()}
}
+1 -1
View File
@@ -20,7 +20,7 @@ type memAudit struct {
fail bool
}
func (m *memAudit) AppendAudit(env agentcommand.AuditEnvelope) error {
func (m *memAudit) AppendAudit(owner string, env agentcommand.AuditEnvelope) error {
if m.fail {
return errors.New("audit down")
}
+102
View File
@@ -0,0 +1,102 @@
package cmdbus
// EXT-14:Bus.CommandTools 只投影带 ExposeAsTool 的已启用命令;risk 缺省 low、
// description 缺省取 title;未暴露/未启用的命令不出现。
import (
"context"
"encoding/json"
"strings"
"testing"
"time"
agentcommand "agent-command"
)
func newToolBus(t *testing.T) *Bus {
t.Helper()
return New(nopAudit{}, nil, func() time.Time { return time.Now().UTC() })
}
func TestCommandToolsFiltering(t *testing.T) {
b := newToolBus(t)
specs := []json.RawMessage{
[]byte(`{"id":"rdm.trace","title":"追溯查询","description":"查网","paramsSchema":{"type":"object"},"exposeAsTool":{"risk":"medium"}}`),
[]byte(`{"id":"rdm.baseline","title":"基线视图","exposeAsTool":{}}`),
[]byte(`{"id":"rdm.hidden","title":"未暴露","description":"不出现"}`),
}
if err := b.RegisterPluginCommands(specs); err != nil {
t.Fatal(err)
}
tools := b.CommandTools()
byID := map[string]CommandTool{}
for _, ct := range tools {
byID[ct.ID] = ct
}
if len(byID) != 2 {
t.Fatalf("projected = %+v, want 2(rdm.trace + rdm.baseline)", tools)
}
if ct := byID["rdm.trace"]; ct.Risk != "medium" || ct.Description != "查网" || string(ct.Params) != `{"type":"object"}` {
t.Fatalf("rdm.trace = %+v", ct)
}
if ct := byID["rdm.baseline"]; ct.Risk != "low" || ct.Description != "基线视图" {
t.Fatalf("rdm.baseline = %+v(risk 缺省 low、description 缺省 title)", ct)
}
if _, ok := byID["rdm.hidden"]; ok {
t.Fatal("未声明 exposeAsTool 的命令不得投影")
}
}
func TestCommandToolDispatchEnvelope(t *testing.T) {
b := newToolBus(t)
// 注册一个带处理器的暴露命令,经 Dispatch 分发验证完整回执链路。
var spec agentcommand.CommandSpec
if err := json.Unmarshal([]byte(`{"id":"rdm.echo","title":"回声","exposeAsTool":{"risk":"low"}}`), &spec); err != nil {
t.Fatal(err)
}
if err := b.RegisterPluginCommand(spec, func(ctx context.Context, req Request) (any, error) {
return map[string]any{"echo": req.Params["v"], "actor": req.Actor, "entrance": req.Entrance}, nil
}); err != nil {
t.Fatal(err)
}
resp := b.Dispatch(context.Background(), Request{ID: "rdm.echo", Params: map[string]any{"v": 1},
Entrance: "agent", Actor: "bob", Owner: "bob", ConversationID: "cnv-1"})
if resp.Receipt.State != "committed" {
t.Fatalf("receipt = %+v (%s)", resp.Receipt, resp.Receipt.Reason)
}
m, _ := resp.Payload.(map[string]any)
if m["entrance"] != "agent" || m["actor"] != "bob" {
t.Fatalf("payload = %+v(Entrance=agent 原样入审计)", m)
}
}
type nopAudit struct{}
func (nopAudit) AppendAudit(owner string, env agentcommand.AuditEnvelope) error { return nil }
// U3:owner 级注入闸——gate 拒绝的命令不投影(禁用即不注入能力);fail-closed
// (owner 为空且 gate 判否时不投影)。
func TestCommandToolsForGate(t *testing.T) {
bus := New(nil, nil, func() time.Time { return time.Now().UTC() })
if err := bus.RegisterPluginCommands([]json.RawMessage{
[]byte(`{"id":"generative-ui.present","title":"生成页面","description":"d","paramsSchema":{"type":"object"},"exposeAsTool":{"risk":"low"}}`),
[]byte(`{"id":"other.echo","title":"回声","description":"e","paramsSchema":{"type":"object"},"exposeAsTool":{"risk":"low"}}`),
}); err != nil {
t.Fatal(err)
}
bus.SetToolGate(func(owner, id string) bool {
return !strings.HasPrefix(id, "generative-ui.")
})
on := bus.CommandToolsFor("alice")
for _, ct := range on {
if strings.HasPrefix(ct.ID, "generative-ui.") {
t.Fatalf("gated command leaked for enabled owner: %+v", ct)
}
}
if len(on) != 1 {
t.Fatalf("expected only other.echo, got %+v", on)
}
if n := len(bus.CommandToolsFor("")); n != 1 {
t.Fatalf("empty owner must be fail-closed on gated commands, got %d", n)
}
}
+96
View File
@@ -0,0 +1,96 @@
// Package cmdbuf 意图路由(G2-2:Intent 声明面与自然语言入口)。
// 插件经 extension.Registry.AddIntent 声明「短语 → 命令」映射;宿主把声明汇入
// 本表。匹配口径:文本归一(去空白/小写)后做包含匹配,最长短语优先——
// 无模型参与的确定性第一档;路由终点仍是命令总线(同审计信封与 HITL 收口)。
package cmdbus
import (
"fmt"
"sort"
"strings"
"sync"
)
// Intent 一条意图声明:短语命中后分发到 CommandID(参数模板随声明携带)。
type Intent struct {
ID string `json:"id"` // 形如 <domain>.<intent>(如 rdm.write-proposal)
Phrases []string `json:"phrases"` // 触发短语(归一后包含匹配)
Command string `json:"command"` // 目标命令 id(须已注册)
Params map[string]any `json:"params,omitempty"` // 参数模板(可被调用方覆盖)
Description string `json:"description,omitempty"`
Plugin string `json:"plugin"` // 声明方(审计归因)
}
// IntentStore 意图声明表(进程内;插件注册期写入,运行期只读)。
type IntentStore struct {
mu sync.RWMutex
byID map[string]Intent
phrases []string // 归一短语(排序后长词优先匹配)
}
// NewIntentStore 建表。
func NewIntentStore() *IntentStore {
return &IntentStore{byID: map[string]Intent{}}
}
// normalize 意图匹配归一:小写、压缩空白。
func normalize(s string) string {
return strings.Join(strings.Fields(strings.ToLower(s)), " ")
}
// Declare 登记意图(同 id 覆盖;短语与命令必填——fail-closed)。
func (s *IntentStore) Declare(in Intent) error {
if in.ID == "" || in.Command == "" || len(in.Phrases) == 0 {
return fmt.Errorf("cmdbus: intent %q 声明不完整(id/phrases/command 必填)", in.ID)
}
s.mu.Lock()
defer s.mu.Unlock()
s.byID[in.ID] = in
s.rebuild()
return nil
}
func (s *IntentStore) rebuild() {
s.phrases = s.phrases[:0]
for _, in := range s.byID {
for _, p := range in.Phrases {
s.phrases = append(s.phrases, normalize(p))
}
}
// 长短语优先:更具体的意图先命中。
sort.Slice(s.phrases, func(i, j int) bool { return len(s.phrases[i]) > len(s.phrases[j]) })
}
// Match 自然语言匹配:返回命中的意图(ok=false=未命中)。
func (s *IntentStore) Match(text string) (Intent, bool) {
n := normalize(text)
if n == "" {
return Intent{}, false
}
s.mu.RLock()
defer s.mu.RUnlock()
for _, ph := range s.phrases {
if ph != "" && strings.Contains(n, ph) {
for _, in := range s.byID {
for _, p := range in.Phrases {
if normalize(p) == ph {
return in, true
}
}
}
}
}
return Intent{}, false
}
// List 全部意图声明(清单面)。
func (s *IntentStore) List() []Intent {
s.mu.RLock()
defer s.mu.RUnlock()
out := make([]Intent, 0, len(s.byID))
for _, in := range s.byID {
out = append(out, in)
}
sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID })
return out
}
+97
View File
@@ -0,0 +1,97 @@
package cmdbus
import (
"context"
"encoding/json"
"testing"
agentcommand "agent-command"
)
// EXT-10② 红测:插件命令规格自带 slash 别名 → 四入口同源(slash 命中同一命令
// id,经总线分发同审计信封)。
func TestPluginSlashAliasBindsAndDispatches(t *testing.T) {
bus := New(nopAudit{}, nil, nil)
called := ""
if err := bus.RegisterPluginCommand(spec(t, `{
"id": "rdm.ping", "title": "自检", "slash": "/rdm.ping"
}`), func(ctx context.Context, req Request) (any, error) {
called = req.ID + "|" + req.Entrance
return map[string]any{"ok": true}, nil
}); err != nil {
t.Fatal(err)
}
id, ok := bus.ResolveSlash("/rdm.ping")
if !ok || id != "rdm.ping" {
t.Fatalf("ResolveSlash = %q ok=%v(插件别名未绑定)", id, ok)
}
// 大小写与首尾空白归一。
if id, ok := bus.ResolveSlash(" /RDM.PING "); !ok || id != "rdm.ping" {
t.Fatalf("归一解析失败: %q %v", id, ok)
}
resp := bus.Dispatch(context.Background(), Request{ID: id, Entrance: "slash", Owner: "alice"})
if resp.Receipt.State != "committed" {
t.Fatalf("slash 分发 = %+v", resp.Receipt)
}
if called != "rdm.ping|slash" {
t.Fatalf("处理器见到 %q(须同命令 id)", called)
}
// 同一命令按钮入口分发——同源同信封。
if resp2 := bus.Dispatch(context.Background(), Request{ID: "rdm.ping", Entrance: "button", Owner: "alice"}); resp2.Receipt.State != "committed" {
t.Fatalf("button 分发 = %+v", resp2.Receipt)
}
}
// G2-2 红测:意图声明→归一包含匹配(最长短语优先)→路由到命令(chat 入口)。
func TestIntentDeclareMatchRoute(t *testing.T) {
bus := New(nopAudit{}, nil, nil)
_ = bus.RegisterPluginCommand(spec(t, `{
"id": "rdm.draft.compose", "title": "起草"
}`), func(ctx context.Context, req Request) (any, error) {
return map[string]any{"sectionId": req.Params["sectionId"]}, nil
})
if err := bus.Intents().Declare(Intent{
ID: "rdm.write-proposal", Phrases: []string{"帮我写项目建议书", "写项目建议书"},
Command: "rdm.draft.compose", Params: map[string]any{"sectionId": "sec-1"},
Plugin: "rdm",
}); err != nil {
t.Fatal(err)
}
// 未命中。
if _, ok := bus.Intents().Match("今天天气如何"); ok {
t.Fatal("无关文本不得命中")
}
// 命中(含大小写/空白归一)。
in, ok := bus.Intents().Match("请 帮我写项目建议书,尽快")
if !ok || in.ID != "rdm.write-proposal" {
t.Fatalf("匹配 = %+v ok=%v", in, ok)
}
// 路由:匹配到的命令经总线 chat 入口分发成功。
resp := bus.Dispatch(context.Background(), Request{ID: in.Command, Params: in.Params, Entrance: "chat", Owner: "alice"})
if resp.Receipt.State != "committed" {
t.Fatalf("路由分发 = %+v", resp.Receipt)
}
// 声明校验:缺短语拒绝。
if err := bus.Intents().Declare(Intent{ID: "x", Command: "y"}); err == nil {
t.Fatal("不完整声明须拒绝")
}
// 最长短语优先:更具体的意图先命中。
if err := bus.Intents().Declare(Intent{
ID: "rdm.write-competitive", Phrases: []string{"写竞品分析"},
Command: "rdm.draft.compose", Plugin: "rdm",
}); err != nil {
t.Fatal(err)
}
if in, _ := bus.Intents().Match("帮我写项目建议书"); in.ID != "rdm.write-proposal" {
t.Fatalf("长短语意图应先命中: %+v", in)
}
}
func spec(t *testing.T, raw string) agentcommand.CommandSpec {
t.Helper()
var s agentcommand.CommandSpec
if err := json.Unmarshal([]byte(raw), &s); err != nil {
t.Fatal(err)
}
return s
}
+250 -23
View File
@@ -11,6 +11,7 @@ import (
"io"
"net"
"os"
"strconv"
"strings"
)
@@ -26,6 +27,26 @@ type Config struct {
Instance string `json:"instance"`
InsecureListen bool `json:"insecureListen"`
AllowedHostsRaw string `json:"allowedHosts"`
// DisableExecTools 启动级硬关闭执行原语(用户设置 runtime.exec 无法覆盖)。
DisableExecTools bool `json:"disableExecTools"`
// ExecBackend 执行后端(EXT-16):direct(空 = 现状直启)| sandbox(bwrap
// 隔离档)。DisableExecTools 优先级高于后端选择。
ExecBackend string `json:"execBackend"`
// ExecDepsDir sandbox 档只读依赖层目录(空 = 不挂载)。
ExecDepsDir string `json:"execDepsDir"`
// ExecWorkspace sandbox 档工作区模式:空=直写;overlay=冻结基底+upper。
ExecWorkspace string `json:"execWorkspace"`
// Tier 部署档(协作存储方案 KTD1):空=auto(DSN→PG,否则 SQLite);
// local=强制 SQLite(不允许 DSN);team=强制 PG(缺 DSN/未开认证拒绝启动,
// PG 故障拒绝就绪,绝不回落 SQLite)。
Tier string `json:"tier"`
// PG 连接预算(KTD12):零值走默认(16/8/300s/不限/不限);负值非法。
// PGQueryTimeoutSec>0 时以 statement_timeout 附着到 PG DSN。
PGMaxOpen int `json:"pgMaxOpen"`
PGMaxIdle int `json:"pgMaxIdle"`
PGConnMaxIdleSec int `json:"pgConnMaxIdleSec"`
PGConnMaxLifetimeSec int `json:"pgConnMaxLifetimeSec"`
PGQueryTimeoutSec int `json:"pgQueryTimeoutSec"`
}
var defaults = Config{
@@ -37,16 +58,26 @@ var defaults = Config{
// envKey 字段 → 环境变量名(AGENT_SYSTEM_ 前缀)。
var envKey = map[string]string{
"addr": "AGENT_SYSTEM_ADDR",
"data": "AGENT_SYSTEM_DATA",
"plugins": "AGENT_SYSTEM_PLUGINS",
"dsn": "AGENT_SYSTEM_DSN",
"artifacts": "AGENT_SYSTEM_ARTIFACTS",
"auth": "AGENT_SYSTEM_AUTH",
"mock": "AGENT_SYSTEM_MOCK",
"instance": "AGENT_SYSTEM_INSTANCE",
"insecureListen": "AGENT_SYSTEM_INSECURE_LISTEN",
"allowedHosts": "AGENT_SYSTEM_ALLOWED_HOSTS",
"addr": "AGENT_SYSTEM_ADDR",
"data": "AGENT_SYSTEM_DATA",
"plugins": "AGENT_SYSTEM_PLUGINS",
"dsn": "AGENT_SYSTEM_DSN",
"artifacts": "AGENT_SYSTEM_ARTIFACTS",
"auth": "AGENT_SYSTEM_AUTH",
"mock": "AGENT_SYSTEM_MOCK",
"instance": "AGENT_SYSTEM_INSTANCE",
"execBackend": "AGENT_SYSTEM_EXEC_BACKEND",
"execDepsDir": "AGENT_SYSTEM_EXEC_DEPS_DIR",
"execWorkspace": "AGENT_SYSTEM_EXEC_WORKSPACE",
"insecureListen": "AGENT_SYSTEM_INSECURE_LISTEN",
"allowedHosts": "AGENT_SYSTEM_ALLOWED_HOSTS",
"disableExecTools": "AGENT_SYSTEM_DISABLE_EXEC_TOOLS",
"tier": "AGENT_SYSTEM_TIER",
"pgMaxOpen": "AGENT_SYSTEM_PG_MAX_OPEN",
"pgMaxIdle": "AGENT_SYSTEM_PG_MAX_IDLE",
"pgConnMaxIdleSec": "AGENT_SYSTEM_PG_CONN_MAX_IDLE_SEC",
"pgConnMaxLifetimeSec": "AGENT_SYSTEM_PG_CONN_MAX_LIFETIME_SEC",
"pgQueryTimeoutSec": "AGENT_SYSTEM_PG_QUERY_TIMEOUT_SEC",
}
// Source 配置来源层级(数值越大优先级越高)。
@@ -96,11 +127,7 @@ func Load(file string, envLookup func(string) (string, bool), cli Config, cliSet
}
// 文件层按"键存在"覆盖(区分未出现与显式 false/空串;空值有效性由字段语义判定)。
for f := range present {
if isBoolField(f) {
setBool(&cfg, f, boolField(fc, f))
} else {
setString(&cfg, f, stringField(fc, f))
}
setField(&cfg, f, fc)
src[f] = SrcFile
}
}
@@ -113,6 +140,12 @@ func Load(file string, envLookup func(string) (string, bool), cli Config, cliSet
return Resolved{}, err
}
setBool(&cfg, f, b)
} else if isIntField(f) {
n, err := parseInt(key, v)
if err != nil {
return Resolved{}, err
}
setInt(&cfg, f, n)
} else {
setString(&cfg, f, v)
}
@@ -122,15 +155,22 @@ func Load(file string, envLookup func(string) (string, bool), cli Config, cliSet
// CLI 层:仅显式传入。
for f := range cliSet {
src[f] = SrcCLI
if isBoolField(f) {
setBool(&cfg, f, boolField(cli, f))
} else {
setString(&cfg, f, stringField(cli, f))
}
setField(&cfg, f, cli)
}
return Resolved{Config: cfg, sources: src}, nil
}
// setField 从同类型快照覆盖单个字段(文件/CLI 层复用;字段类型由字段名决定)。
func setField(dst *Config, f string, src Config) {
if isBoolField(f) {
setBool(dst, f, boolField(src, f))
} else if isIntField(f) {
setInt(dst, f, intField(src, f))
} else {
setString(dst, f, stringField(src, f))
}
}
func loadFile(path string) (Config, map[string]bool, error) {
raw, err := os.ReadFile(path)
if err != nil {
@@ -165,6 +205,26 @@ func loadFile(path string) (Config, map[string]bool, error) {
dst = &c.InsecureListen
case "allowedHosts":
dst = &c.AllowedHostsRaw
case "disableExecTools":
dst = &c.DisableExecTools
case "execBackend":
dst = &c.ExecBackend
case "execDepsDir":
dst = &c.ExecDepsDir
case "execWorkspace":
dst = &c.ExecWorkspace
case "tier":
dst = &c.Tier
case "pgMaxOpen":
dst = &c.PGMaxOpen
case "pgMaxIdle":
dst = &c.PGMaxIdle
case "pgConnMaxIdleSec":
dst = &c.PGConnMaxIdleSec
case "pgConnMaxLifetimeSec":
dst = &c.PGConnMaxLifetimeSec
case "pgQueryTimeoutSec":
dst = &c.PGQueryTimeoutSec
default:
return Config{}, nil, fmt.Errorf("config file %s: unknown key %q", path, k)
}
@@ -194,18 +254,67 @@ func stringField(c Config, f string) string {
return c.Instance
case "allowedHosts":
return c.AllowedHostsRaw
case "execBackend":
return c.ExecBackend
case "execDepsDir":
return c.ExecDepsDir
case "execWorkspace":
return c.ExecWorkspace
case "tier":
return c.Tier
}
return ""
}
func isBoolField(f string) bool {
return f == "mock" || f == "insecureListen"
return f == "mock" || f == "insecureListen" || f == "disableExecTools"
}
func isIntField(f string) bool {
switch f {
case "pgMaxOpen", "pgMaxIdle", "pgConnMaxIdleSec", "pgConnMaxLifetimeSec", "pgQueryTimeoutSec":
return true
}
return false
}
func intField(c Config, f string) int {
switch f {
case "pgMaxOpen":
return c.PGMaxOpen
case "pgMaxIdle":
return c.PGMaxIdle
case "pgConnMaxIdleSec":
return c.PGConnMaxIdleSec
case "pgConnMaxLifetimeSec":
return c.PGConnMaxLifetimeSec
case "pgQueryTimeoutSec":
return c.PGQueryTimeoutSec
}
return 0
}
func setInt(c *Config, f string, v int) {
switch f {
case "pgMaxOpen":
c.PGMaxOpen = v
case "pgMaxIdle":
c.PGMaxIdle = v
case "pgConnMaxIdleSec":
c.PGConnMaxIdleSec = v
case "pgConnMaxLifetimeSec":
c.PGConnMaxLifetimeSec = v
case "pgQueryTimeoutSec":
c.PGQueryTimeoutSec = v
}
}
func boolField(c Config, f string) bool {
switch f {
case "insecureListen":
return c.InsecureListen
case "disableExecTools":
return c.DisableExecTools
default:
return c.Mock
}
@@ -215,6 +324,8 @@ func setBool(c *Config, f string, v bool) {
switch f {
case "insecureListen":
c.InsecureListen = v
case "disableExecTools":
c.DisableExecTools = v
default:
c.Mock = v
}
@@ -238,6 +349,14 @@ func setString(c *Config, f, v string) {
c.Instance = v
case "allowedHosts":
c.AllowedHostsRaw = v
case "execBackend":
c.ExecBackend = v
case "execDepsDir":
c.ExecDepsDir = v
case "execWorkspace":
c.ExecWorkspace = v
case "tier":
c.Tier = v
}
}
@@ -251,6 +370,14 @@ func parseBool(name, v string) (bool, error) {
return false, fmt.Errorf("env %s: invalid bool %q", name, v)
}
func parseInt(name, v string) (int, error) {
n, err := strconv.Atoi(strings.TrimSpace(v))
if err != nil {
return 0, fmt.Errorf("env %s: invalid int %q", name, v)
}
return n, nil
}
// Explain 只读诊断:逐字段打印解析值与来源层级,DSN 脱敏。
func (r Resolved) Explain(w io.Writer, file string) {
if file != "" {
@@ -259,16 +386,32 @@ func (r Resolved) Explain(w io.Writer, file string) {
fmt.Fprintln(w, "config file: (none)")
}
fmt.Fprintln(w, "field value source")
for _, f := range []string{"addr", "data", "plugins", "dsn", "artifacts", "auth", "mock", "instance", "insecureListen", "allowedHosts"} {
v := stringField(r.Config, f)
for _, f := range []string{"addr", "data", "plugins", "dsn", "artifacts", "auth", "mock", "instance", "insecureListen", "allowedHosts", "disableExecTools", "execBackend", "tier", "pgMaxOpen", "pgMaxIdle", "pgConnMaxIdleSec", "pgConnMaxLifetimeSec", "pgQueryTimeoutSec"} {
var v string
if isBoolField(f) {
v = fmt.Sprintf("%t", boolField(r.Config, f))
} else if isIntField(f) {
n := intField(r.Config, f)
if d := intDefault(f); n == 0 && d != 0 {
v = fmt.Sprintf("%d (默认)", d)
} else {
v = fmt.Sprintf("%d", n)
}
} else {
v = stringField(r.Config, f)
}
if f == "dsn" {
v = redact(v)
}
fmt.Fprintf(w, "%-16s %-56s %s\n", f, v, r.sources[f])
}
if msgs := r.TierProblems(); len(msgs) > 0 {
for _, msg := range msgs {
fmt.Fprintf(w, "tier guard: 拒绝启动(%s)\n", msg)
}
} else {
fmt.Fprintln(w, "tier guard: 放行")
}
if msg := r.ListenRefusal(); msg != "" {
fmt.Fprintf(w, "listen guard: 拒绝启动(%s)\n", msg)
} else if r.InsecureListen && !IsLoopbackListen(r.Addr) && !r.AuthEnabled() {
@@ -283,6 +426,90 @@ func (r Resolved) AuthEnabled() bool {
return r.AuthMode == "on" || (r.AuthMode != "off" && r.StorageDSN != "")
}
// intDefault 返回 int 字段的默认值(0 表示零值合法、无默认替换)。
func intDefault(f string) int {
switch f {
case "pgMaxOpen":
return DefaultPGMaxOpen
case "pgMaxIdle":
return DefaultPGMaxIdle
case "pgConnMaxIdleSec":
return DefaultPGConnMaxIdleSec
}
return 0
}
// 连接预算默认值(KTD12;与 store 层默认保持一致,两处同时修改时以测试对齐)。
const (
DefaultPGMaxOpen = 16
DefaultPGMaxIdle = 8
DefaultPGConnMaxIdleSec = 300
)
// TierProblems 部署档语义校验:返回阻止启动的问题列表(空 = 放行)。
// team 档 fail-closed:缺 DSN、未开认证直接拒绝;local 档不允许 DSN(KTD1)。
// 执行隔离在 team 档是启动警告而非拒绝(本增量先可见、U3/U5 再硬化)。
func (r Resolved) TierProblems() []string {
var msgs []string
switch r.Tier {
case "", "local", "team":
default:
msgs = append(msgs, fmt.Sprintf("未知部署档 %q(local | team)", r.Tier))
}
if r.Tier == "team" {
if r.StorageDSN == "" {
msgs = append(msgs, "team 档需要 PostgreSQL DSN(dsn / AGENT_SYSTEM_DSN);不会回落 SQLite")
}
if !r.AuthEnabled() {
msgs = append(msgs, "team 档必须开启认证(auth=on)")
}
}
if r.Tier == "local" && r.StorageDSN != "" {
msgs = append(msgs, "local 档不使用 PostgreSQL DSN(改用 team 档,或去掉 dsn)")
}
if v := r.PGMaxOpen; v < 0 {
msgs = append(msgs, fmt.Sprintf("pgMaxOpen 不能为负(当前 %d)", v))
}
if v := r.PGMaxIdle; v < 0 {
msgs = append(msgs, fmt.Sprintf("pgMaxIdle 不能为负(当前 %d)", v))
}
if v := r.PGConnMaxIdleSec; v < 0 {
msgs = append(msgs, fmt.Sprintf("pgConnMaxIdleSec 不能为负(当前 %d)", v))
}
if v := r.PGConnMaxLifetimeSec; v < 0 {
msgs = append(msgs, fmt.Sprintf("pgConnMaxLifetimeSec 不能为负(当前 %d)", v))
}
if v := r.PGQueryTimeoutSec; v < 0 {
msgs = append(msgs, fmt.Sprintf("pgQueryTimeoutSec 不能为负(当前 %d)", v))
}
if idle, open := r.EffectivePGMaxIdle(), r.EffectivePGMaxOpen(); idle > open {
msgs = append(msgs, fmt.Sprintf("pgMaxIdle(%d) 大于 pgMaxOpen(%d)", idle, open))
}
return msgs
}
// EffectivePGMaxOpen/Idle/ConnMaxIdleSec 返回连接预算有效值(零值落默认,KTD12)。
func (r Resolved) EffectivePGMaxOpen() int {
if r.PGMaxOpen > 0 {
return r.PGMaxOpen
}
return DefaultPGMaxOpen
}
func (r Resolved) EffectivePGMaxIdle() int {
if r.PGMaxIdle > 0 {
return r.PGMaxIdle
}
return DefaultPGMaxIdle
}
func (r Resolved) EffectivePGConnMaxIdleSec() int {
if r.PGConnMaxIdleSec > 0 {
return r.PGConnMaxIdleSec
}
return DefaultPGConnMaxIdleSec
}
// AllowedHosts 把逗号分隔的主机名拆开,忽略空段。
func (r Resolved) AllowedHosts() []string {
var out []string
+165
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
@@ -123,6 +124,38 @@ func TestEnvBool(t *testing.T) {
}
}
// disableExecTools:默认 false;文件、环境、CLI 三层均可打开,CLI 优先。
func TestDisableExecToolsLayers(t *testing.T) {
r, err := Load("", noEnv, Config{}, nil)
if err != nil || r.DisableExecTools {
t.Fatalf("default must be false: %v %+v", err, r.Config)
}
f := fileWith(t, `{"disableExecTools":true}`)
r, err = Load(f, noEnv, Config{}, nil)
if err != nil || !r.DisableExecTools || r.SourceOf("disableExecTools") != SrcFile {
t.Fatalf("file layer: %v %+v src=%v", err, r.Config, r.SourceOf("disableExecTools"))
}
env := func(k string) (string, bool) {
if k == "AGENT_SYSTEM_DISABLE_EXEC_TOOLS" {
return "1", true
}
return "", false
}
r, err = Load("", env, Config{}, nil)
if err != nil || !r.DisableExecTools || r.SourceOf("disableExecTools") != SrcEnv {
t.Fatalf("env layer: %v %+v", err, r.Config)
}
r, err = Load("", env, Config{DisableExecTools: false}, map[string]bool{"disableExecTools": true})
if err != nil || r.DisableExecTools || r.SourceOf("disableExecTools") != SrcCLI {
t.Fatalf("cli must win: %v %+v", err, r.Config)
}
var out bytes.Buffer
r.Explain(&out, "")
if !bytes.Contains(out.Bytes(), []byte("disableExecTools")) {
t.Fatalf("explain must list disableExecTools:\n%s", out.String())
}
}
// explain:来源标注 + DSN 脱敏;只读(不落任何盘外文件)。
func TestExplainRedactsDSN(t *testing.T) {
env := func(k string) (string, bool) {
@@ -188,3 +221,135 @@ func TestListenGuard(t *testing.T) {
t.Fatalf("auth on should pass: %+v %q", authed.Config, authed.ListenRefusal())
}
}
// ---- 协作存储方案 U2:部署档(KTD1)与连接预算(KTD12)----
// team 档 fail-closed:缺 DSN、未开认证都拒绝;DSN+auth=on 放行。
func TestTierTeamFailClosed(t *testing.T) {
r, err := Load("", noEnv, Config{Tier: "team"}, map[string]bool{"tier": true})
if err != nil {
t.Fatal(err)
}
msgs := r.TierProblems()
if len(msgs) != 2 {
t.Fatalf("team without dsn/auth should refuse with 2 problems, got %v", msgs)
}
joined := msgs[0] + msgs[1]
if !strings.Contains(joined, "PostgreSQL DSN") || !strings.Contains(joined, "认证") {
t.Fatalf("refusal messages wrong: %v", msgs)
}
// 补上 DSN(auto 认证随 DSN 开启)→ 放行。
r, err = Load("", noEnv, Config{Tier: "team", StorageDSN: "postgres://u:p@h/db"}, map[string]bool{"tier": true, "dsn": true})
if err != nil {
t.Fatal(err)
}
if msgs = r.TierProblems(); len(msgs) != 0 {
t.Fatalf("team with dsn should pass, got %v", msgs)
}
// 显式关认证仍然拒绝。
r, err = Load("", noEnv, Config{Tier: "team", StorageDSN: "postgres://u:p@h/db", AuthMode: "off"}, map[string]bool{"tier": true, "dsn": true, "auth": true})
if err != nil {
t.Fatal(err)
}
if msgs = r.TierProblems(); len(msgs) != 1 || !strings.Contains(msgs[0], "认证") {
t.Fatalf("team with auth=off should refuse, got %v", msgs)
}
}
// local 档不允许 DSN;未知档名拒绝。
func TestTierLocalAndUnknown(t *testing.T) {
r, err := Load("", noEnv, Config{Tier: "local", StorageDSN: "postgres://u:p@h/db"}, map[string]bool{"tier": true, "dsn": true})
if err != nil {
t.Fatal(err)
}
if msgs := r.TierProblems(); len(msgs) != 1 || !strings.Contains(msgs[0], "local 档不使用") {
t.Fatalf("local with dsn should refuse, got %v", msgs)
}
r, err = Load("", noEnv, Config{Tier: "cluster"}, map[string]bool{"tier": true})
if err != nil {
t.Fatal(err)
}
if msgs := r.TierProblems(); len(msgs) != 1 || !strings.Contains(msgs[0], "未知部署档") {
t.Fatalf("unknown tier should refuse, got %v", msgs)
}
// auto(空)档维持现状语义:无 DSN 不拒绝。
r, err = Load("", noEnv, Config{}, nil)
if err != nil {
t.Fatal(err)
}
if msgs := r.TierProblems(); len(msgs) != 0 {
t.Fatalf("auto default should pass, got %v", msgs)
}
}
// 连接预算四源解析与环境变量整数校验;MaxIdle>MaxOpen 拒绝。
func TestPGPoolConfigLayers(t *testing.T) {
env := func(k string) (string, bool) {
switch k {
case "AGENT_SYSTEM_PG_MAX_OPEN":
return "7", true
case "AGENT_SYSTEM_PG_MAX_IDLE":
return "9", true
}
return "", false
}
f := fileWith(t, `{"pgMaxOpen":5}`)
r, err := Load(f, env, Config{PGMaxOpen: 3, PGMaxIdle: 2}, map[string]bool{"pgMaxIdle": true})
if err != nil {
t.Fatal(err)
}
// 文件 5 被 env 7 覆盖(CLI 值 3 未显式传入不参与);pgMaxIdle CLI 显式 2 胜。
if r.PGMaxOpen != 7 || r.SourceOf("pgMaxOpen") != SrcEnv {
t.Fatalf("pgMaxOpen = %d (%v), want env 7", r.PGMaxOpen, r.SourceOf("pgMaxOpen"))
}
if r.PGMaxIdle != 2 || r.SourceOf("pgMaxIdle") != SrcCLI {
t.Fatalf("pgMaxIdle = %d (%v), want CLI 2", r.PGMaxIdle, r.SourceOf("pgMaxIdle"))
}
if msgs := r.TierProblems(); len(msgs) != 0 {
t.Fatalf("2/2 pool valid, got %v", msgs)
}
// 有效值:零值落默认。
r, err = Load("", noEnv, Config{}, nil)
if err != nil {
t.Fatal(err)
}
if r.EffectivePGMaxOpen() != DefaultPGMaxOpen || r.EffectivePGMaxIdle() != DefaultPGMaxIdle || r.EffectivePGConnMaxIdleSec() != DefaultPGConnMaxIdleSec {
t.Fatalf("effective defaults wrong: %+v", r.Config)
}
// idle > open 拒绝。
r, err = Load("", noEnv, Config{PGMaxOpen: 2, PGMaxIdle: 3}, map[string]bool{"pgMaxOpen": true, "pgMaxIdle": true})
if err != nil {
t.Fatal(err)
}
if msgs := r.TierProblems(); len(msgs) != 1 || !strings.Contains(msgs[0], "pgMaxIdle(3) 大于 pgMaxOpen(2)") {
t.Fatalf("idle>open should refuse, got %v", msgs)
}
// 非法整数环境变量报错定位到变量。
bad := func(k string) (string, bool) {
if k == "AGENT_SYSTEM_PG_MAX_OPEN" {
return "many", true
}
return "", false
}
if _, err := Load("", bad, Config{}, nil); err == nil || !strings.Contains(err.Error(), "AGENT_SYSTEM_PG_MAX_OPEN") {
t.Fatalf("invalid int env should fail with var name, got %v", err)
}
}
// explain 输出包含部署档与预算行,且 tier guard 结论可见(脱敏诊断)。
func TestExplainTierAndPool(t *testing.T) {
r, err := Load("", noEnv, Config{Tier: "team"}, map[string]bool{"tier": true})
if err != nil {
t.Fatal(err)
}
var b bytes.Buffer
r.Explain(&b, "")
out := b.String()
for _, want := range []string{"tier", "pgMaxOpen", "pgMaxIdle", "pgConnMaxIdleSec", "pgConnMaxLifetimeSec", "pgQueryTimeoutSec", "tier guard: 拒绝启动"} {
if !strings.Contains(out, want) {
t.Fatalf("explain missing %q:\n%s", want, out)
}
}
}
+103 -14
View File
@@ -36,13 +36,23 @@ type CardResolver func(id core.ID) *core.ActivityCard
// 对称)。ok=false = 无工件服务/无此卡/内容不可读,部件留占位行。
type ArtifactResolver func(id core.ID) (art core.Artifact, content []byte, ok bool)
// AssembleOptions 是组装的可选行为。零值全部关闭(向后兼容)。
type AssembleOptions struct {
// ThinkReplay 为真时,历史 assistant 消息的思考部件以 <think> 块原样回传
//(交错思考模型如 MiniMax 的官方要求:丢弃思考内容会降低后续轮性能)。
ThinkReplay bool
// Surfaces 是当前会话最近保存的生成页面(U3/R6):以语义投影并入上下文,
// 使下一轮模型能解释并修改已有结构(修改走 generative-ui.present 重提)。
Surfaces []core.SurfaceSnapshot
}
// AssembleInput 组装模型输入消息。sys 为提示词库组装出的 system 段(可为多段;
// 调用方经 prompts.Service.ResolveSystem 产出),promptObs 为对应可观测条目。
// prof.SystemPrompt 非空时作为画像自定义段追加(向后兼容旧画像)。
// resolveCard 用于 card 引用部件回填产物卡正文(submit_card;D35:卡片为产物权威
// 载体,下一轮模型不失忆),nil 时 card 部件只留占位。resolveArtifact 同理回填
// artifact_ref 工件正文(D46 统一;仅 UTF-8 文本,超 limit 截断留痕)。
func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history []core.Message, currentUser core.Message, attachments []core.ParsedDocument, knowledge []core.RetrievalHit, sys []agentllm.Message, promptObs []string, resolveCard CardResolver, resolveArtifact ArtifactResolver) ([]agentllm.Message, *Report) {
func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history []core.Message, currentUser core.Message, attachments []core.ParsedDocument, knowledge []core.RetrievalHit, sys []agentllm.Message, promptObs []string, resolveCard CardResolver, resolveArtifact ArtifactResolver, opts AssembleOptions) ([]agentllm.Message, *Report) {
rep := &Report{}
msgs := make([]agentllm.Message, 0, len(sys)+1)
msgs = append(msgs, sys...)
@@ -51,6 +61,17 @@ func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history [
msgs = append(msgs, agentllm.Message{Role: agentllm.RoleSystem, Content: prof.SystemPrompt})
rep.Items = append(rep.Items, "system(画像自定义段)")
}
if len(opts.Surfaces) > 0 {
var b strings.Builder
b.WriteString("[本会话已保存的生成页面(结构摘要;正文以预览区为准)]\n")
for _, snap := range opts.Surfaces {
b.WriteString(projectSurface(snap))
b.WriteString("\n")
}
b.WriteString("用户要求调整页面时,按结构摘要理解现状并用 generative-ui.present 提交修订(带 surfaceId 与 baseRevision)。")
msgs = append(msgs, agentllm.Message{Role: agentllm.RoleSystem, Content: b.String()})
rep.Items = append(rep.Items, fmt.Sprintf("surface(%d)", len(opts.Surfaces)))
}
maxHist := prof.Context.MaxHistoryMessages
if maxHist <= 0 {
@@ -84,7 +105,7 @@ func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history [
continue
}
if m.Role == core.RoleAssistant {
msgs = append(msgs, renderAssistantTurns(m, resolveCard, limitCard, resolveArtifact, rep, &cardsBackfilled, &artsBackfilled)...)
msgs = append(msgs, renderAssistantTurns(m, resolveCard, limitCard, resolveArtifact, rep, &cardsBackfilled, &artsBackfilled, opts)...)
continue
}
content := renderMessage(m, resolveCard, limitCard, resolveArtifact, rep, &cardsBackfilled, &artsBackfilled)
@@ -102,6 +123,16 @@ func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history [
}
// 当前轮:文本 + 附件解析结果 + 知识检索注入。
msgs = append(msgs, BuildCurrentUser(eff, currentUser, attachments, knowledge, rep))
fillEstimate(msgs, rep)
return msgs, rep
}
// BuildCurrentUser 组装当前轮的用户消息(文本部件 + 附件解析全文 + 知识检索)。
// 会话转录为权威上下文时,每轮只构建这一条消息,历史由转录承载——
// 此处与整段组装共用同一实现,保证两条路径的当前轮语义一致。
func BuildCurrentUser(eff core.EffectiveSettings, currentUser core.Message, attachments []core.ParsedDocument, knowledge []core.RetrievalHit, rep *Report) agentllm.Message {
var sb strings.Builder
for _, p := range currentUser.Parts {
if p.Type == core.PartText {
@@ -118,23 +149,26 @@ func AssembleInput(prof core.AgentProfile, eff core.EffectiveSettings, history [
text := doc.PlainText()
if runes := []rune(text); len(runes) > limit {
text = string(runes[:limit]) + "…(附件超长截断)"
rep.Truncations = append(rep.Truncations, fmt.Sprintf("附件 %s 超 %d 字符截断(原文 %d 字符)", doc.Meta.Filename, limit, len(runes)))
if rep != nil {
rep.Truncations = append(rep.Truncations, fmt.Sprintf("附件 %s 超 %d 字符截断(原文 %d 字符)", doc.Meta.Filename, limit, len(runes)))
}
}
fmt.Fprintf(&sb, "### %s\n%s\n", doc.Meta.Filename, text)
}
rep.Items = append(rep.Items, fmt.Sprintf("attachments(%d)", len(attachments)))
if rep != nil {
rep.Items = append(rep.Items, fmt.Sprintf("attachments(%d)", len(attachments)))
}
}
if len(knowledge) > 0 {
sb.WriteString("\n\n[相关知识]\n")
for i, h := range knowledge {
fmt.Fprintf(&sb, "[%d] %s(来源 %s)\n%s\n", i+1, h.Doc.Title, h.Doc.Source, h.Doc.Chunk)
}
rep.Items = append(rep.Items, fmt.Sprintf("knowledge(%d)", len(knowledge)))
if rep != nil {
rep.Items = append(rep.Items, fmt.Sprintf("knowledge(%d)", len(knowledge)))
}
}
msgs = append(msgs, agentllm.Message{Role: agentllm.RoleUser, Content: sb.String()})
fillEstimate(msgs, rep)
return msgs, rep
return agentllm.Message{Role: agentllm.RoleUser, Content: sb.String()}
}
// maxToolArgRunes 是回放进模型的单次工具参数上限。超限时截断字符串值,保持合法 JSON。
@@ -171,17 +205,24 @@ func renderMessage(m core.Message, resolveCard CardResolver, limitCard int, reso
// 前置文本与 tool_calls 组成一条 assistant,每个调用紧跟一条 tool 结果。
// 没有结果的调用补一条失败占位,避免模型把中断当成成功。孤立的结果直接跳过。
// 被守卫降级、且后面不是工具调用的过程文本不回放。
func renderAssistantTurns(m core.Message, resolveCard CardResolver, limitCard int, resolveArtifact ArtifactResolver, rep *Report, cardsBackfilled, artsBackfilled *int) []agentllm.Message {
func renderAssistantTurns(m core.Message, resolveCard CardResolver, limitCard int, resolveArtifact ArtifactResolver, rep *Report, cardsBackfilled, artsBackfilled *int, opts AssembleOptions) []agentllm.Message {
var out []agentllm.Message
var text strings.Builder
var think strings.Builder
var calls []agentllm.ToolCall
results := map[string]core.ToolResultPart{}
seq := 0
flush := func() {
content := strings.TrimSpace(text.String())
thinkText := strings.TrimSpace(think.String())
if opts.ThinkReplay && thinkText != "" {
// 交错思考模型要求历史 assistant 原样带 <think>;只进请求,不落盘。
content = "<think>\n" + thinkText + "\n</think>\n\n" + content
}
if content == "" && len(calls) == 0 {
text.Reset()
think.Reset()
results = map[string]core.ToolResultPart{}
return
}
@@ -202,12 +243,18 @@ func renderAssistantTurns(m core.Message, resolveCard CardResolver, limitCard in
})
}
text.Reset()
think.Reset()
calls = nil
results = map[string]core.ToolResultPart{}
}
for i, p := range m.Parts {
switch p.Type {
case core.PartThinking:
if strings.TrimSpace(p.Text) != "" {
think.WriteString(p.Text)
think.WriteString("\n")
}
case core.PartText:
if demotedProcessText(m.Parts, i) {
continue
@@ -381,19 +428,22 @@ func renderCardPart(cp *core.CardPart, resolveCard CardResolver, limit int, rep
// renderArtifactRef 渲染 artifact_ref 部件:有 resolver 且内容为 UTF-8 文本时
// 回填正文(与卡同 limit,超限截断留痕);二进制/不可读/无 resolver 留占位行
// (可见即信号,不静默丢弃)。
// 占位文案不带「[产出工件 …]」方括号形态:那正是 ClaimGate 用来识别伪造调用
// 标记的模式——历史里自己产出这种格式,下一轮模型照抄就会被终轮检查误判为
// 「正文里出现了工具调用标记」。
func renderArtifactRef(a *core.ArtifactRef, resolve ArtifactResolver, limit int, rep *Report, backfilled *int) string {
if a == nil {
return ""
}
if resolve == nil {
return fmt.Sprintf("[产出工件 %s]\n", a.Name)
return fmt.Sprintf("此前已生成文件:%s。\n", a.Name)
}
art, content, ok := resolve(a.ArtifactID)
if !ok {
return fmt.Sprintf("[产出工件 %s]\n", a.Name)
return fmt.Sprintf("此前已生成文件:%s。\n", a.Name)
}
if !utf8.Valid(content) {
return fmt.Sprintf("[产出工件 %s(二进制内容,正文见工件查看器)]\n", art.Name)
return fmt.Sprintf("此前已生成文件:%s(二进制内容,正文见工件查看器)。\n", art.Name)
}
*backfilled++
text := string(content)
@@ -401,7 +451,7 @@ func renderArtifactRef(a *core.ArtifactRef, resolve ArtifactResolver, limit int,
text = string(runes[:limit]) + "…(工件超长截断)"
rep.Truncations = append(rep.Truncations, fmt.Sprintf("工件 %s 超 %d 字符截断", art.Name, limit))
}
return fmt.Sprintf("[产出工件 %s]\n%s\n", art.Name, text)
return fmt.Sprintf("此前已生成文件 %s,内容如下:\n%s\n", art.Name, text)
}
// extractiveSummary 把窗口外历史压成短摘要(不调用模型)。summarizeBeyond 打开时注入。
@@ -463,3 +513,42 @@ func InjectMemory(sys []agentllm.Message, entries []core.MemoryEntry) []agentllm
out = append(out, agentllm.Message{Role: agentllm.RoleSystem, Content: b.String()})
return out
}
// projectSurface 是 ctxasm 侧的结构摘要投影(与 surfacesvc.ProjectSurface 同义;
// 独立实现避免 ctxasm→surfacesvc 依赖边)。
func projectSurface(snap core.SurfaceSnapshot) string {
var b strings.Builder
fmt.Fprintf(&b, "- 页面 %s(修订 %d)%q", string(snap.Identity.SurfaceID), snap.Revision, snap.Spec.Title)
if snap.SemanticText != "" {
fmt.Fprintf(&b, ":%s", snap.SemanticText)
}
b.WriteString("\n 结构:")
var walk func(n core.UINode)
first := true
walk = func(n core.UINode) {
label := string(n.Type)
if n.Title != "" {
label += "(\"" + n.Title + "\")"
}
if n.ID != "" {
label += "#" + n.ID
}
if len(n.Source) > 0 {
label += "[" + strings.Join(n.Source, ",") + "]"
}
if first {
b.WriteString(label)
first = false
} else {
b.WriteString(" > " + label)
}
for _, c := range n.Children {
walk(c)
}
}
for _, n := range snap.Spec.Nodes {
walk(n)
}
b.WriteString("\n 修订此页:present 带 surfaceId=" + string(snap.Identity.SurfaceID) + " 与 baseRevision=" + fmt.Sprint(snap.Revision) + "。")
return b.String()
}
+12 -12
View File
@@ -18,7 +18,7 @@ func TestAssembleInputAttachmentLimit(t *testing.T) {
Meta: core.DocMeta{Filename: "纪要.docx"},
Blocks: []core.DocBlock{{Type: core.BlockPara, Text: long}},
}
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, nil, fakeUser("请总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil)
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, nil, fakeUser("请总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil, AssembleOptions{})
last := msgs[len(msgs)-1]
if !strings.Contains(last.Content, "纪要.docx") {
@@ -42,7 +42,7 @@ func TestAssembleInputAttachmentLimitConfigurable(t *testing.T) {
Blocks: []core.DocBlock{{Type: core.BlockPara, Text: strings.Repeat("甲", 120)}},
}
eff := core.EffectiveSettings{"context": map[string]any{"maxAttachmentChars": float64(100)}}
msgs, rep := AssembleInput(prof, eff, nil, fakeUser("总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil)
msgs, rep := AssembleInput(prof, eff, nil, fakeUser("总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil, AssembleOptions{})
last := msgs[len(msgs)-1]
if got := strings.Count(last.Content, "甲"); got != 100 {
t.Errorf("injected chars = %d, want 100 (configured limit)", got)
@@ -58,7 +58,7 @@ func TestAssembleInputAttachmentNoTruncationUnderLimit(t *testing.T) {
Meta: core.DocMeta{Filename: "b.txt"},
Blocks: []core.DocBlock{{Type: core.BlockPara, Text: strings.Repeat("乙", 100)}},
}
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, nil, fakeUser("总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil)
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, nil, fakeUser("总结"), []core.ParsedDocument{doc}, nil, nil, nil, nil, nil, AssembleOptions{})
if len(rep.Truncations) != 0 {
t.Errorf("under-limit doc should not truncate: %v", rep.Truncations)
}
@@ -89,7 +89,7 @@ func TestAssembleInputCardBackfill(t *testing.T) {
}
return nil
}
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("把第二章改短"), nil, nil, nil, nil, resolve, nil)
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("把第二章改短"), nil, nil, nil, nil, resolve, nil, AssembleOptions{})
var assistantIn string
for _, m := range msgs {
@@ -128,7 +128,7 @@ func TestAssembleInputCardTruncationAndFallback(t *testing.T) {
}},
}
eff := core.EffectiveSettings{"context": map[string]any{"maxCardChars": float64(100)}}
msgs, rep := AssembleInput(prof, eff, hist, fakeUser("继续"), nil, nil, nil, nil, func(id core.ID) *core.ActivityCard { return &card }, nil)
msgs, rep := AssembleInput(prof, eff, hist, fakeUser("继续"), nil, nil, nil, nil, func(id core.ID) *core.ActivityCard { return &card }, nil, AssembleOptions{})
if strings.Count(msgs[1].Content, "丁") != 100 {
t.Errorf("card chars = %d, want 100 (configured limit)", strings.Count(msgs[1].Content, "丁"))
}
@@ -137,7 +137,7 @@ func TestAssembleInputCardTruncationAndFallback(t *testing.T) {
}
// 无 resolver(收件箱关闭/卡片缺失):占位可见。
msgs2, _ := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil)
msgs2, _ := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil, AssembleOptions{})
if !strings.Contains(msgs2[1].Content, "内容见收件箱") {
t.Errorf("fallback placeholder missing: %q", msgs2[1].Content)
}
@@ -159,8 +159,8 @@ func TestAssembleInputArtifactBackfill(t *testing.T) {
}
return core.Artifact{}, nil, false
}
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, resolve)
if !strings.Contains(msgs[1].Content, "[产出工件 检索汇总.md]") || !strings.Contains(msgs[1].Content, "关键结论一。") {
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, resolve, AssembleOptions{})
if !strings.Contains(msgs[1].Content, "此前已生成文件 检索汇总.md,内容如下:") || !strings.Contains(msgs[1].Content, "关键结论一。") {
t.Errorf("artifact body not backfilled: %q", msgs[1].Content)
}
if !contains(rep.Items, "artifact(1)") {
@@ -172,7 +172,7 @@ func TestAssembleInputArtifactBackfill(t *testing.T) {
eff := core.EffectiveSettings{"context": map[string]any{"maxCardChars": float64(100)}}
msgs2, rep2 := AssembleInput(prof, eff, hist, fakeUser("继续"), nil, nil, nil, nil, nil, func(core.ID) (core.Artifact, []byte, bool) {
return art, []byte(long), true
})
}, AssembleOptions{})
if strings.Count(msgs2[1].Content, "乙") != 100 {
t.Errorf("artifact chars = %d, want 100", strings.Count(msgs2[1].Content, "乙"))
}
@@ -183,14 +183,14 @@ func TestAssembleInputArtifactBackfill(t *testing.T) {
// 二进制内容:占位不回灌乱码。
msgs3, _ := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, func(core.ID) (core.Artifact, []byte, bool) {
return art, []byte{0xff, 0xfe, 0x00, 0x01}, true
})
}, AssembleOptions{})
if !strings.Contains(msgs3[1].Content, "二进制内容") || strings.Contains(msgs3[1].Content, "关键结论") {
t.Errorf("binary artifact should keep placeholder: %q", msgs3[1].Content)
}
// 无 resolver:占位可见。
msgs4, _ := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil)
if !strings.Contains(msgs4[1].Content, "[产出工件 检索汇总.md]") {
msgs4, _ := AssembleInput(prof, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil, AssembleOptions{})
if !strings.Contains(msgs4[1].Content, "此前已生成文件:检索汇总.md。") {
t.Errorf("artifact placeholder missing without resolver: %q", msgs4[1].Content)
}
}
+33 -8
View File
@@ -109,6 +109,9 @@ func FinishAssembly(msgs []agentllm.Message, rep *Report, tail Tail) ([]agentllm
return msgs, rep
}
// TailNotes 导出尾注渲染(转录权威模式下由调用方经 Reminders 逐轮注入)。
func TailNotes(tail Tail) []string { return tailNotes(tail) }
func tailNotes(tail Tail) []string {
var notes []string
if text := strings.TrimSpace(tail.Instructions); text != "" {
@@ -207,12 +210,12 @@ func trimOldest(msgs []agentllm.Message, window, reserve int, rep *Report) []age
}
dropped := 0
for estimateAll(msgs) > limit {
idx := oldestDrop(msgs)
if idx < 0 {
start, end := oldestDropRange(msgs)
if start < 0 {
break
}
msgs = append(msgs[:idx], msgs[idx+1:]...)
dropped++
dropped += end - start
msgs = append(msgs[:start], msgs[end:]...)
}
if dropped > 0 {
rep.Truncations = append(rep.Truncations, fmtDropped(dropped))
@@ -226,9 +229,14 @@ func fmtDropped(n int) string {
func itoa(n int) string { return strconv.Itoa(n) }
func oldestDrop(msgs []agentllm.Message) int {
// oldestDropRange 返回下一次丢弃的区间 [start, end)(找不到返回 -1,-1)。
// 丢弃按工具组对齐:带 tool_calls 的 assistant 与其连续 tool 结果同退,
// 开头的孤立 tool 结果组整组同退——裁切后既不剩无结果的调用,
// 也不剩无调用的结果(与库层 alignToolTail 同语义、方向相反)。
// 组越过最后一条(当前轮)时不再丢弃:窗口允许略超,配对不可破坏。
func oldestDropRange(msgs []agentllm.Message) (int, int) {
if len(msgs) < 3 {
return -1
return -1, -1
}
// 保留开头的系统前缀和最后一条(当前轮)。
start := 0
@@ -236,9 +244,26 @@ func oldestDrop(msgs []agentllm.Message) int {
start++
}
if start >= len(msgs)-1 {
return -1
return -1, -1
}
return start
end := start + 1
m := msgs[start]
switch {
case m.Role == agentllm.RoleAssistant && len(m.ToolCalls) > 0:
// assistant 及其连续 tool 结果整组同退。
for end < len(msgs) && msgs[end].Role == agentllm.RoleTool {
end++
}
case m.Role == agentllm.RoleTool:
// 开头即孤立 tool 结果(其 assistant 必已被丢或不存在):整组同退。
for end < len(msgs) && msgs[end].Role == agentllm.RoleTool {
end++
}
}
if end > len(msgs)-1 {
return -1, -1
}
return start, end
}
func estimateAll(msgs []agentllm.Message) int {
+116 -1
View File
@@ -14,7 +14,7 @@ func TestVisibleHistoryDropsExcludedAndUnselectedVariants(t *testing.T) {
{ID: "a1", Role: core.RoleAssistant, VariantGroup: "g", VariantIndex: 0, Excluded: true, Parts: []core.Part{{Type: core.PartText, Text: "旧"}}},
{ID: "a2", Role: core.RoleAssistant, VariantGroup: "g", VariantIndex: 1, Parts: []core.Part{{Type: core.PartText, Text: "新"}}},
}
msgs, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, core.Message{ID: "cur", Role: core.RoleUser}, nil, nil, nil, nil, nil, nil)
msgs, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, core.Message{ID: "cur", Role: core.RoleUser}, nil, nil, nil, nil, nil, nil, AssembleOptions{})
blob := ""
for _, m := range msgs {
blob += m.Content
@@ -62,3 +62,118 @@ func TestTrimDropsOldestHistory(t *testing.T) {
t.Fatal("expected truncation")
}
}
// ---- N2 WP1:裁切工具组对齐(验收② 产品侧机器证据)----
// assertNoOrphanToolPairs 双向断言:每个 tool_call 有结果,每个结果有调用。
func assertNoOrphanToolPairs(t *testing.T, msgs []agentllm.Message) {
t.Helper()
calls := map[string]bool{}
results := map[string]bool{}
for _, m := range msgs {
for _, c := range m.ToolCalls {
calls[c.ID] = true
}
if m.Role == agentllm.RoleTool && m.ToolCallID != "" {
results[m.ToolCallID] = true
}
}
for id := range calls {
if !results[id] {
t.Fatalf("孤立 tool_call:%s(无结果)", id)
}
}
for id := range results {
if !calls[id] {
t.Fatalf("悬空 tool 结果:%s(无调用)", id)
}
}
}
func toolMsgs() []agentllm.Message {
call := func(id, name string) agentllm.ToolCall {
return agentllm.ToolCall{ID: id, Name: name, Arguments: "{}"}
}
return []agentllm.Message{
{Role: agentllm.RoleSystem, Content: "s"},
{Role: agentllm.RoleUser, Content: strings.Repeat("旧一", 300)},
{Role: agentllm.RoleAssistant, ToolCalls: []agentllm.ToolCall{call("tc1", "read")}},
{Role: agentllm.RoleTool, ToolCallID: "tc1", Name: "read", Content: strings.Repeat("结果一", 200)},
{Role: agentllm.RoleUser, Content: strings.Repeat("旧二", 300)},
{Role: agentllm.RoleAssistant, ToolCalls: []agentllm.ToolCall{call("tc2", "grep"), call("tc3", "read")}},
{Role: agentllm.RoleTool, ToolCallID: "tc2", Name: "grep", Content: strings.Repeat("结果二", 200)},
{Role: agentllm.RoleTool, ToolCallID: "tc3", Name: "read", Content: strings.Repeat("结果三", 200)},
{Role: agentllm.RoleUser, Content: "现在"},
}
}
// 连续多轮裁切后配对仍成立:assistant 与其结果同退,不逐条拆散。
func TestTrimOldestKeepsToolPairs(t *testing.T) {
out, rep := FinishAssembly(toolMsgs(), &Report{}, Tail{Window: 1600})
assertNoOrphanToolPairs(t, out)
if len(rep.Truncations) == 0 {
t.Fatal("expected truncation")
}
if out[len(out)-1].Content != "现在" {
t.Fatal("当前轮必须保留")
}
// 窗口再压:旧二组也须整组同退,仍不出现孤立配对。
out2, _ := FinishAssembly(toolMsgs(), &Report{}, Tail{Window: 900})
assertNoOrphanToolPairs(t, out2)
}
// 开头即孤立 tool 结果组:整组同退(其 assistant 已不存在)。
func TestTrimOldestDropsLeadingOrphanToolRun(t *testing.T) {
msgs := []agentllm.Message{
{Role: agentllm.RoleSystem, Content: "s"},
{Role: agentllm.RoleTool, ToolCallID: "ghost", Name: "read", Content: strings.Repeat("孤", 500)},
{Role: agentllm.RoleUser, Content: strings.Repeat("旧", 500)},
{Role: agentllm.RoleUser, Content: "现在"},
}
out, _ := FinishAssembly(msgs, &Report{}, Tail{Window: 700})
assertNoOrphanToolPairs(t, out)
for _, m := range out {
if m.ToolCallID == "ghost" {
t.Fatal("孤立结果组应整组丢弃")
}
}
}
// 工具组越过当前轮时不再丢弃:窗口可略超,配对不可破坏。
func TestTrimOldestStopsWhenGroupCrossesCurrentTurn(t *testing.T) {
msgs := []agentllm.Message{
{Role: agentllm.RoleSystem, Content: "s"},
{Role: agentllm.RoleAssistant, ToolCalls: []agentllm.ToolCall{{ID: "tc9", Name: "read", Arguments: "{}"}}},
{Role: agentllm.RoleTool, ToolCallID: "tc9", Name: "read", Content: strings.Repeat("结果", 900)},
}
start, end := oldestDropRange(msgs)
if start != -1 || end != -1 {
t.Fatalf("组越过当前轮应返回 -1,-1,得到 %d,%d", start, end)
}
}
// ---- N2 WP1:PrefixHash 稳定契约(同一输入 → 同一前缀哈希)----
func TestPrefixHashStableForIdenticalInput(t *testing.T) {
build := func() []agentllm.Message {
return []agentllm.Message{
{Role: agentllm.RoleSystem, Content: "系统前缀A"},
{Role: agentllm.RoleSystem, Content: "系统前缀B"},
{Role: agentllm.RoleUser, Content: "历史一"},
{Role: agentllm.RoleAssistant, Content: "回复一"},
{Role: agentllm.RoleUser, Content: "现在"},
}
}
_, rep1 := FinishAssembly(build(), &Report{}, Tail{Instructions: "个人指令"})
_, rep2 := FinishAssembly(build(), &Report{}, Tail{Instructions: "个人指令"})
if rep1.PrefixHash == "" || rep1.PrefixHash != rep2.PrefixHash {
t.Fatalf("同一输入的前缀哈希必须一致:%q vs %q", rep1.PrefixHash, rep2.PrefixHash)
}
// 前缀变化必须改变哈希(钉住哈希语义,防退化为常量)。
other := build()
other[0].Content = "系统前缀X"
_, rep3 := FinishAssembly(other, &Report{}, Tail{Instructions: "个人指令"})
if rep3.PrefixHash == rep1.PrefixHash {
t.Fatal("前缀变化后哈希必须不同")
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ func assistantWith(id string, parts ...core.Part) core.Message {
}
func assembleHistory(hist []core.Message, resolve ArtifactResolver) []agentllm.Message {
msgs, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, resolve)
msgs, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, resolve, AssembleOptions{})
if len(msgs) == 0 {
return nil
}
+1 -1
View File
@@ -14,7 +14,7 @@ func TestSummarizeBeyondInjectsExtract(t *testing.T) {
}
current := core.Message{ID: "cur", Role: core.RoleUser, Parts: []core.Part{{Type: core.PartText, Text: "现在"}}}
prof := core.AgentProfile{Context: core.ContextPolicy{MaxHistoryMessages: 2, SummarizeBeyond: true}}
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, history, current, nil, nil, nil, nil, nil, nil)
msgs, rep := AssembleInput(prof, core.EffectiveSettings{}, history, current, nil, nil, nil, nil, nil, nil, AssembleOptions{})
joined := ""
for _, m := range msgs {
joined += m.Content
+52
View File
@@ -0,0 +1,52 @@
package ctxasm
// U3/R6:已保存页面的语义投影并入上下文——下一轮模型能看到结构、来源与修订
// 入口(present + baseRevision),可据此解释并修改已有 table/compare/tabs。
import (
"strings"
"testing"
agentllm "agent-native/runtime/llm"
core "agent-system-core"
)
func surfaceSnap() core.SurfaceSnapshot {
return core.SurfaceSnapshot{
Identity: core.SurfaceIdentity{SurfaceID: "surf_x", Owner: "alice", PluginID: "generative-ui", ConversationID: "c1"},
Revision: 2,
Spec: core.SurfaceSpec{Title: "选型对比", Nodes: []core.UINode{
{Type: core.UISection, ID: "n1", Title: "概览", Children: []core.UINode{
{Type: core.UIMarkdown, ID: "n1a", Source: []string{"doc:a"}, Markdown: "结论"},
}},
{Type: core.UITable, ID: "n2", Source: []string{"doc:a", "doc:b"}},
}},
SemanticText: "三工具对比页",
}
}
func TestSurfaceProjectionInContext(t *testing.T) {
msgs, rep := AssembleInput(core.AgentProfile{}, nil, nil,
core.Message{Role: core.RoleUser, Parts: []core.Part{{Type: core.PartText, Text: "改一下"}}},
nil, nil, nil, nil, nil, nil, AssembleOptions{Surfaces: []core.SurfaceSnapshot{surfaceSnap()}})
joined := make([]string, len(msgs))
for i, m := range msgs {
joined[i] = m.Content
}
all := strings.Join(joined, "\n---\n")
for _, need := range []string{"surf_x", "修订 2", "选型对比", "section(\"概览\")#n1", "markdown#n1a[doc:a]", "table#n2", "baseRevision=2"} {
if !strings.Contains(all, need) {
t.Fatalf("projection missing %q in assembled context:\n%s", need, all)
}
}
found := false
for _, it := range rep.Items {
if strings.HasPrefix(it, "surface(") {
found = true
}
}
if !found {
t.Fatalf("report should count surface injection: %v", rep.Items)
}
_ = agentllm.RoleSystem
}
+43
View File
@@ -0,0 +1,43 @@
package ctxasm
import (
"strings"
"testing"
agentllm "agent-native/runtime/llm"
core "agent-system-core"
)
// 交错思考模型(MiniMax 等)要求历史 assistant 原样回传 <think>;
// 关闭时不回传(默认兼容旧行为)。思考块只进请求,不落库。
func TestThinkReplayWrapsAssistantHistory(t *testing.T) {
hist := []core.Message{
{ID: "m1", Role: core.RoleUser, Parts: []core.Part{{Type: core.PartText, Text: "总结"}}},
{ID: "m2", Role: core.RoleAssistant, Parts: []core.Part{
{Type: core.PartThinking, Text: "先读材料。", Process: true},
{Type: core.PartText, Text: "结论是先入库。", Process: true},
{Type: core.PartToolCall, ToolCall: &core.ToolCallPart{ID: "c1", Name: "run_python", Arguments: "{}"}},
{Type: core.PartToolResult, ToolResult: &core.ToolResultPart{CallID: "c1", Status: "ok", Summary: "done"}},
{Type: core.PartText, Text: "已完成。"},
}},
}
on, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil, AssembleOptions{ThinkReplay: true})
var sawThink int
for _, m := range on {
if m.Role == agentllm.RoleAssistant && strings.HasPrefix(m.Content, "<think>") {
sawThink++
if !strings.Contains(m.Content, "</think>") || !strings.Contains(m.Content, "先读材料。") {
t.Fatalf("think block malformed: %q", m.Content)
}
}
}
if sawThink == 0 {
t.Fatalf("ThinkReplay=on should inject <think> into assistant history: %+v", on)
}
off, _ := AssembleInput(core.AgentProfile{}, core.EffectiveSettings{}, hist, fakeUser("继续"), nil, nil, nil, nil, nil, nil, AssembleOptions{})
for _, m := range off {
if m.Role == agentllm.RoleAssistant && strings.Contains(m.Content, "<think>") {
t.Fatalf("ThinkReplay=off must not inject think blocks: %q", m.Content)
}
}
}
+95
View File
@@ -0,0 +1,95 @@
package genui
import (
"sync"
"time"
)
// AutoBudgetPerHour 是每任务(conversation)滑动一小时内允许的自动建议次数。
const AutoBudgetPerHour = 3
// AutoCooldown 是同会话两次自动触发的最小间隔。
const AutoCooldown = 60 * time.Second
// SettingAutoPresent 是 owner 级 opt-in 设置键(默认 false;与 activation 分离)。
const SettingAutoPresent = "plugins.generative-ui.autoPresent"
// AutoReminder 是注入 system-reminder 的建议文案(只引导 present,禁止改任务/点动作)。
const AutoReminder = "任务语义事件已发生(阶段或材料/工件完成)。若 generative-ui 已启用,请调用一次 generative-ui.present 生成或修订建议页,按当前任务阶段组织信息。不要点击页面 actions,不要改写 plan/todo/任务完成状态;任务事实只能来自真实任务记录。"
// ActivationLookup 查询 owner 的 Surface activation(generative-ui)。
type ActivationLookup func(owner string) (enabled bool)
// SettingsLookup 查询 owner 的 autoPresent opt-in。
type SettingsLookup func(owner string) (enabled bool)
// Auditor 写审计事件(耗尽等);nil = 跳过。
type Auditor func(owner, kind, detail string)
// Auto 是自动触发闸门:opt-in ∧ activation ∧ 预算 ∧ 冷却。
// 触发源由 runmgr 显式调用;present/surface/action 路径不得调用。
type Auto struct {
activation ActivationLookup
settings SettingsLookup
audit Auditor
now func() time.Time
mu sync.Mutex
budget map[string][]time.Time // convID → 触发时刻
last map[string]time.Time // convID → 上次成功放行
}
// NewAuto 构造闸门(lookup 不得为 nil)。
func NewAuto(activation ActivationLookup, settings SettingsLookup, audit Auditor) *Auto {
return &Auto{
activation: activation,
settings: settings,
audit: audit,
now: func() time.Time { return time.Now().UTC() },
budget: map[string][]time.Time{},
last: map[string]time.Time{},
}
}
// Try 尝试放行一次自动建议。ok=false 时 skip 含原因(供测试);ok=true 时调用方应注入 AutoReminder。
func (a *Auto) Try(owner, convID, reason string) (ok bool, skip string) {
if a == nil || a.activation == nil || a.settings == nil {
return false, "unconfigured"
}
if owner == "" || convID == "" {
return false, "missing_ids"
}
if !a.settings(owner) {
return false, "opt_in_off"
}
if !a.activation(owner) {
return false, "activation_off"
}
now := a.now()
a.mu.Lock()
defer a.mu.Unlock()
if t, hit := a.last[convID]; hit && now.Sub(t) < AutoCooldown {
return false, "cooldown"
}
cutoff := now.Add(-time.Hour)
kept := a.budget[convID][:0]
for _, t := range a.budget[convID] {
if t.After(cutoff) {
kept = append(kept, t)
}
}
a.budget[convID] = kept
if len(kept) >= AutoBudgetPerHour {
if a.audit != nil {
a.audit(owner, "genui.auto_present_exhausted", "conversation="+convID+" reason="+reason)
}
return false, "budget"
}
a.budget[convID] = append(kept, now)
a.last[convID] = now
return true, ""
}
+53
View File
@@ -0,0 +1,53 @@
package genui
import (
"testing"
"time"
)
func TestAutoDefaultOff(t *testing.T) {
a := NewAuto(func(string) bool { return true }, func(string) bool { return false }, nil)
if ok, skip := a.Try("alice", "c1", "plan"); ok || skip != "opt_in_off" {
t.Fatalf("want opt_in_off, got ok=%v skip=%q", ok, skip)
}
}
func TestAutoRequiresActivation(t *testing.T) {
a := NewAuto(func(string) bool { return false }, func(string) bool { return true }, nil)
if ok, skip := a.Try("alice", "c1", "plan"); ok || skip != "activation_off" {
t.Fatalf("want activation_off, got ok=%v skip=%q", ok, skip)
}
}
func TestAutoBudgetAndCooldown(t *testing.T) {
var exhausted int
now := time.Unix(1_700_000_000, 0).UTC()
a := NewAuto(func(string) bool { return true }, func(string) bool { return true },
func(owner, kind, detail string) {
if kind == "genui.auto_present_exhausted" {
exhausted++
}
})
a.now = func() time.Time { return now }
for i := 0; i < AutoBudgetPerHour; i++ {
ok, skip := a.Try("alice", "c1", "plan")
if !ok {
t.Fatalf("shot %d should pass, skip=%q", i, skip)
}
now = now.Add(AutoCooldown + time.Second)
}
if ok, skip := a.Try("alice", "c1", "plan"); ok || skip != "budget" || exhausted != 1 {
t.Fatalf("want budget exhausted, ok=%v skip=%q exhausted=%d", ok, skip, exhausted)
}
// 冷却:同一时刻第二次拒绝
a2 := NewAuto(func(string) bool { return true }, func(string) bool { return true }, nil)
a2.now = func() time.Time { return now }
if ok, _ := a2.Try("alice", "c2", "art"); !ok {
t.Fatal("first should pass")
}
if ok, skip := a2.Try("alice", "c2", "art"); ok || skip != "cooldown" {
t.Fatalf("want cooldown, got ok=%v skip=%q", ok, skip)
}
}
+539
View File
@@ -0,0 +1,539 @@
// Package genui 是 Generative UI 的内置进程内插件(提案 U3):
// 当前 Agent 经 exposeAsTool 的 generative-ui.present 命令提交声明式页面,
// 宿主(EXT-17 SurfaceHost)校验并原子发布;目录=提示=校验三同源。
// 不接管对话循环、模型凭据或任务执行权(KTD2);不另造 Agent runtime。
package genui
import (
"context"
"encoding/json"
"fmt"
"strconv"
"strings"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
// PluginID 是插件身份(activation 台账与命令前缀共用)。
const PluginID = "generative-ui"
const (
// CommandPresent 是页面提交命令(revise 即带 surfaceId+baseRevision 的再次 present)。
CommandPresent = PluginID + ".present"
// CommandSubmit 是 U5 已批准的 C 类页面动作(表单数据落插件 KV 台账;
// 不 exposeAsTool——按钮专用,模型不可直接调用)。
CommandSubmit = PluginID + ".submit"
// ActionHelp / ActionIntent 是 B 类首批加白(独立评审:builtin.help、system.intent)。
ActionHelp = "builtin.help"
ActionIntent = "system.intent"
)
// AllowedActionIDs 是 present 校验与 surfacesvc.AllowedActions 的开放动作集合(须同步)。
var AllowedActionIDs = map[string]struct{}{
CommandSubmit: {},
ActionHelp: {},
ActionIntent: {},
}
// Catalog 是组件目录与纪律(U1 试验 CATALOG_PROMPT 的 Go 权威版;工具描述携带,
// 模型与校验器看到同一份规则)。修改需同步 core.ValidateSurfaceSpec 与目录版本。
const Catalog = `何时用本工具:用户要求"整理成一页 / 做个对比 / 出一页报告 / 看板 / 时间线"这类**页面化交付**时,优先调用本工具提交声明式页面,而不是写文件工件——页面会在用户预览区直接打开并可继续修订。纯文本问答、简短列表、代码交付保持原方式,不强行出页。
任务工作页组织(用户在任务/项目工作区查看时):按任务阶段决定页面结构——需求阶段突出待澄清项与决策点(form/kv 优先);执行阶段突出步骤、进度与产物(steps/timeline/table 优先);验收阶段突出验证结果与缺项(kv+markdown 结论优先)。任务完成状态、步骤勾选等**事实字段只能来自对话上下文中的真实任务记录**(计划/待办/验证结果),你不能凭页面渲染改变任务状态——只做事实的组织与呈现。
可用节点类型(16 种;字段必须严格匹配,禁止目录外类型/字段):
section{title?,children?};list{items:string[]};kv{kv:[[k,v]]};
form{fields:[{key,label,type:text|textarea|number|boolean|select,options?(select必填),required?,placeholder?,help?,default?}]};
markdown{markdown};doc-ref{docId}(须为会话内存在的工件ID);
actions{actions:[{id,label,style?:primary|default|danger}]}(不带command);
chart-lite{chart:{kind:bar|line,labels:[],series:[]等长}};avatar{avatar:{name,sub?,tone?:ok|warn|danger|info,stack?}};
progress{progress:{label?,value:0~1}};compare{compare:{left:{title?,tone?,nodes?},right:{同left}}};
tree{tree:{nodes:[{id,label,tone?,badge?,count?,children?}]}};
table{table:{columns:[{key,label}],rows:[{id?,cells:[[key,值]与columns等长同序}]}};
steps{steps:{current:0起,steps:[{label,status?:pending|active|done|failed,tone?}]}};
tabs{tabs:{tabs:[{id,label,nodes?}],active?}};timeline{timeline:{items:[{title,time?,tone?,markdown?,ref?}]}}。
规则:每节点可带 id(整页唯一稳定)与 source[](支撑事实的来源标识);事实性内容标注来源,
分析/建议只放 markdown 且不加 source;≤200 节点、≤8 层、≤256KiB;未知类型/字段与未知
doc-ref 会被拒绝并返回字段级错误(可修正后重提一次)。结构服务目标:比较→compare/table;
风险→按严重度组织;顺序→steps/timeline;收集输入→form。
actions 节点纪律:action 的 id 仅允许开放白名单——generative-ui.submit(C 类表单提交)、
builtin.help(B 类只读帮助)、system.intent(B 类意图匹配;button 入口只返回匹配结果、
不嵌套分发业务命令)。引用其他 action id 会被拒绝;按钮点击由用户发起,宿主经命令总线分发并审计。
最小完整示例(spec 参数的形状;title 与 nodes 在顶层,source 挂在节点上):
{"title":"示例页","nodes":[
{"type":"section","id":"n1","title":"结论","children":[
{"type":"markdown","id":"n1a","source":["doc:m1"],"markdown":"结论先行。"}]},
{"type":"table","id":"n2","table":{"columns":[{"key":"k","label":"维度"},{"key":"v","label":"值"}],
"rows":[{"id":"r1","cells":[["k","报价"],["v","¥8/人/天"]]}]}},
{"type":"steps","id":"n3","steps":{"current":0,"steps":[{"label":"评审","status":"pending"}]}}]}`
func init() {
extension.RegisterCompiled(extension.InProc{ID: PluginID, Register: register})
}
func register(reg extension.Registry) error {
spec := json.RawMessage(fmt.Sprintf(`{
"id": %q,
"title": "生成页面",
"description": %q,
"paramsSchema": {
"type": "object",
"properties": {
"spec": {"type": "object", "description": "页面规格:顶层恰为 {title: string, nodes: [节点数组]}",
"properties": {"title": {"type": "string"}, "nodes": {"type": "array", "items": {"type": "object"}}},
"required": ["title", "nodes"]},
"surfaceId": {"type": "string", "description": "修订已有页面时传入"},
"baseRevision": {"type": "integer", "description": "修订基于的当前修订号"},
"summary": {"type": "string", "description": "一句话语义摘要(回放与索引用)"}
},
"required": ["spec"]
},
"exposeAsTool": {"risk": "low"}
}`, CommandPresent, "提交一页声明式页面(由宿主校验、保存并在预览区渲染)。"+Catalog))
if err := reg.AddCommandHandler(spec, present); err != nil {
return err
}
submitSpec := json.RawMessage(`{
"id": "generative-ui.submit",
"title": "提交表单",
"description": "页面动作:提交用户填写的表单数据(落插件台账;按钮触发,不经模型)",
"paramsSchema": {
"type": "object",
"properties": {
"surfaceId": {"type": "string"},
"revision": {"type": "integer"},
"requestId": {"type": "string"},
"data": {"type": "object"}
},
"required": ["surfaceId", "requestId", "data"]
}
}`)
return reg.AddCommandHandler(submitSpec, submitForm)
}
// submitForm 是 C 类动作处理器:表单数据落插件自有 KV 台账(键含 requestId
// 天然幂等;已批准范围限定 KV+任务台账,不建业务事实表)。经命令总线四入口
// 同源:审计信封与(若未来升档)高危确认卡由总线收口。
func submitForm(ctx context.Context, h extension.Host, req extension.Request) (any, error) {
surfaceID, _ := req.Params["surfaceId"].(string)
requestID, _ := req.Params["requestId"].(string)
if surfaceID == "" || requestID == "" {
return nil, fmt.Errorf("generative-ui.submit: surfaceId and requestId required")
}
data, _ := json.Marshal(req.Params["data"])
key := "form:" + surfaceID + ":" + requestID
if err := h.KVPut(key, string(data)); err != nil {
return nil, err
}
if err := h.Audit("surface.form_submitted", surfaceID, "requestId="+requestID+" bytes="+fmt.Sprint(len(data))); err != nil {
return nil, err
}
return map[string]any{"ok": true, "stored": true, "key": key}, nil
}
// ValidateActionIDs 供 present 校验:actions 节点引用的 id 必须在已开放集合内。
func ValidateActionIDs(spec []byte) []string {
var s struct {
Nodes []core.UINode `json:"nodes"`
}
if err := json.Unmarshal(spec, &s); err != nil {
return nil
}
var errs []string
var walk func(n core.UINode, path string)
walk = func(n core.UINode, path string) {
if n.Type == core.UIActions {
for _, a := range n.Actions {
if _, ok := AllowedActionIDs[a.ID]; !ok {
errs = append(errs, fmt.Sprintf("%s: 未知动作 id %q(不在开放白名单)", path, a.ID))
}
}
}
for i, c := range n.Children {
walk(c, fmt.Sprintf("%s/child[%d]", path, i))
}
if n.Compare != nil {
for side, sd := range map[string]core.UICompareSide{"left": n.Compare.Left, "right": n.Compare.Right} {
for i, c := range sd.Nodes {
walk(c, fmt.Sprintf("%s/%s[%d]", path, side, i))
}
}
}
if n.Tabs != nil {
for i, tb := range n.Tabs.Tabs {
for j, c := range tb.Nodes {
walk(c, fmt.Sprintf("%s/tab[%d][%d]", path, i, j))
}
}
}
}
for i, n := range s.Nodes {
walk(n, fmt.Sprintf("$[%d]", i))
}
return errs
}
// present 是命令处理器:经 owner 作用域 Host 的 SurfaceHost 能力提交页面。
// 能力缺失(宿主未装配)与未启用明确报错;doc-ref 经 owner 作用域工件解析做
// 真实授权(伪造引用拒绝);校验失败返回字段级清单供模型修复重提(至多一次)。
func present(ctx context.Context, h extension.Host, req extension.Request) (any, error) {
sh, ok := h.(extension.SurfaceHost)
if !ok {
return nil, fmt.Errorf("generative-ui: surface capability not available in this host")
}
enabled, epoch, err := sh.SurfaceActivation(ctx)
if err != nil {
return nil, err
}
if !enabled {
return nil, fmt.Errorf("generative-ui: 插件未启用(owner 未开启 Surface 能力),本次提交被拒绝")
}
raw, _ := json.Marshal(normalizeSpecParam(req.Params["spec"]))
surfaceID, _ := req.Params["surfaceId"].(string)
base := 0
if v, ok := req.Params["baseRevision"].(float64); ok {
base = int(v)
}
summary, _ := req.Params["summary"].(string)
// doc-ref 真实授权:spec 里的 docId 必须是当前 owner 可解析的工件(伪造即拒)。
if errs := authorizeDocRefs(raw, h); len(errs) > 0 {
return toolError("invalid spec", errs), nil
}
// 动作白名单(U5 已批准范围):未知动作 id 拒绝并回字段级错误。
if errs := ValidateActionIDs(raw); len(errs) > 0 {
return toolError("invalid spec", errs), nil
}
snap, err := sh.SurfaceSave(ctx, extension.SurfaceSaveRequest{
SurfaceID: core.ID(surfaceID),
ConversationID: core.ID(req.ConversationID),
BaseRevision: base,
ObservedEpoch: epoch,
SemanticText: summary,
Spec: raw,
})
if err != nil {
if strings.HasPrefix(err.Error(), "surface: invalid spec:") {
details := strings.TrimSpace(strings.TrimPrefix(err.Error(), "surface: invalid spec:"))
var list []string
for _, d := range strings.Split(details, ";") {
if t := strings.TrimSpace(d); t != "" {
list = append(list, t)
}
}
return toolError("invalid spec", list), nil
}
return nil, err
}
return map[string]any{
"ok": true,
"surfaceId": string(snap.Identity.SurfaceID),
"revision": snap.Revision,
"epoch": snap.ActivationEpoch,
"summary": snap.SemanticText,
"note": "页面已保存;用户在预览区查看。修订时带 surfaceId 与 baseRevision 重提。",
}, nil
}
// toolError 是返回给模型的字段级错误(工具结果错误,非分发错误)。
type toolErrorBody struct {
OK bool `json:"ok"`
Error string `json:"error"`
Details []string `json:"details,omitempty"`
Note string `json:"note"`
}
func toolError(msg string, details []string) toolErrorBody {
return toolErrorBody{OK: false, Error: msg, Details: details,
Note: "按错误修正 spec 后可重提一次;再次失败请改用文本回答。"}
}
// normalizeSpecParam 是插件的 schema 修复策略(r1 KTD1;U3 回放实测的模型偏差
// 逐条落地):单键包装({"page":…})解包;顶层 children→nodes;kv/cells 的
// 对象形式({"k":…,"v":…}/{"key":…,"value":…})改写为二元组数组。只做结构
// 改写,不改内容语义;修复后仍过 core 校验器。
func normalizeSpecParam(v any) any {
if arr, ok := v.([]any); ok {
return map[string]any{"title": "", "nodes": arr}
}
m, ok := v.(map[string]any)
if !ok {
return v
}
if len(m) == 1 {
if inner, ok := m["page"].(map[string]any); ok {
m = inner
}
}
if _, ok := m["nodes"]; !ok {
if ch, ok := m["children"].([]any); ok {
m["nodes"] = ch
delete(m, "children")
}
}
// 页级 source(模型的常见引用写法):分发给未自带 source 的节点——保留引用
// 信息,不发明内容;节点自带 source 者不受影响。
if top, ok := m["source"].([]any); ok {
delete(m, "source")
if nodes, ok := m["nodes"].([]any); ok {
for _, n := range nodes {
if nm, ok := n.(map[string]any); ok {
if _, has := nm["source"]; !has {
cp := make([]any, len(top))
copy(cp, top)
nm["source"] = cp
}
}
}
}
}
repairChildrenShape(m)
// {"item": …} 单键包装在 children 形状修复之后剥:children 的 {"item":{node}}
// 由 repairChildrenShape 的对象映射规则兜住,先剥反而会把单子节点打散进
// 值数组;kv/cells/items/columns 等其余数组字段在这里统一解到本体。
if unwrapped, ok := repairItemWrappers(m).(map[string]any); ok {
m = unwrapped
}
repairPairs(m)
repairTableCells(m)
return m
}
// repairItemWrappers 递归剥掉 XML-RPC 风格的 {"item": X} 单键包装(2026-10-12
// dev 实测 MiniMax-M3,事件 seq 1395/2171):数组字段(kv/cells/items/columns…)
// 整字段被包一层 {"item":[…]},数组元素再被包 {"item":["k","v"]}/{"item":"v"}。
// 规范词汇无名为 item 的字段(core/ui.go 全量核对),单键 item 不承载语义,
// 安全解包;只改结构不改内容。
func repairItemWrappers(v any) any {
switch t := v.(type) {
case map[string]any:
if len(t) == 1 {
if inner, ok := t["item"]; ok {
return repairItemWrappers(inner)
}
}
out := make(map[string]any, len(t))
for k, val := range t {
out[k] = repairItemWrappers(val)
}
return out
case []any:
out := make([]any, len(t))
for i, item := range t {
out[i] = repairItemWrappers(item)
}
return out
default:
return v
}
}
// repairTableCells 表格行 cells 的裸标量数组(同场实测):cells 期望与
// columns.key 对应的 [key,value] 二元组,模型输出成与列等长的裸值数组。
// 长度一致且列键齐全时按列序配对(结构改写,不发明内容);不一致则不动,
// 交校验器给字段级错误供模型修复重提。
func repairTableCells(node any) {
m, ok := node.(map[string]any)
if !ok {
return
}
if tbl, ok := m["table"].(map[string]any); ok {
cols, colsOK := tbl["columns"].([]any)
rows, rowsOK := tbl["rows"].([]any)
if colsOK && rowsOK {
keys := make([]string, len(cols))
valid := true
for i, c := range cols {
cm, ok := c.(map[string]any)
if !ok {
valid = false
break
}
k, ok := cm["key"].(string)
if !ok || k == "" {
valid = false
break
}
keys[i] = k
}
if valid {
for _, r := range rows {
rm, ok := r.(map[string]any)
if !ok {
continue
}
cells, ok := rm["cells"].([]any)
if !ok || len(cells) != len(keys) {
continue
}
paired := make([]any, 0, len(cells))
convertible := true
for i, c := range cells {
switch cv := c.(type) {
case string:
paired = append(paired, []any{keys[i], cv})
case float64:
paired = append(paired, []any{keys[i], strconv.FormatFloat(cv, 'f', -1, 64)})
default:
convertible = false
break
}
}
if convertible {
rm["cells"] = paired
}
}
}
}
}
for _, v := range m {
if vm, ok := v.(map[string]any); ok {
repairTableCells(vm)
}
if va, ok := v.([]any); ok {
for _, item := range va {
if im, ok := item.(map[string]any); ok {
repairTableCells(im)
}
}
}
}
}
// repairChildrenShape 递归修复 children 的常见形状偏差(真实模型实测):
// children 为对象(按 key 编号的子节点映射)→ 取其值转数组;单对象 → 包一层数组。
func repairChildrenShape(node any) {
m, ok := node.(map[string]any)
if !ok {
return
}
if ch, ok := m["children"]; ok && ch != nil {
switch v := ch.(type) {
case map[string]any:
arr := make([]any, 0, len(v))
for _, val := range v {
arr = append(arr, val)
}
m["children"] = arr
case []any:
for _, c := range v {
if cm, ok := c.(map[string]any); ok {
repairChildrenShape(cm)
}
}
default:
// 单对象子节点
if cm, ok := ch.(map[string]any); ok {
m["children"] = []any{cm}
}
}
}
for _, v := range m {
if vm, ok := v.(map[string]any); ok {
repairChildrenShape(vm)
}
if va, ok := v.([]any); ok {
for _, item := range va {
if im, ok := item.(map[string]any); ok {
repairChildrenShape(im)
}
}
}
}
}
// repairPairs 递归把 "kv"/"cells" 下的对象项改写为二元组。
func repairPairs(node any) {
m, ok := node.(map[string]any)
if !ok {
return
}
for key, val := range m {
if (key == "kv" || key == "cells") && val != nil {
if arr, ok := val.([]any); ok {
fixed := make([]any, 0, len(arr))
changed := false
for _, item := range arr {
if p, ok := pairFromObject(item); ok {
fixed = append(fixed, p)
changed = true
} else {
fixed = append(fixed, item)
}
}
if changed {
m[key] = fixed
}
}
}
repairPairs(val)
}
}
// pairFromObject 从 {"k","v"}/{"key","value"}/{"name","value"} 对象提取二元组。
func pairFromObject(item any) ([2]any, bool) {
obj, ok := item.(map[string]any)
if !ok {
return [2]any{}, false
}
var k, v any
for kk, vv := range obj {
switch kk {
case "k", "key", "name":
k = vv
case "v", "value":
v = vv
default:
return [2]any{}, false
}
}
if k == nil || v == nil {
return [2]any{}, false
}
ks, ok1 := k.(string)
vs, ok2 := v.(string)
if !ok1 || !ok2 {
return [2]any{}, false
}
return [2]any{ks, vs}, true
}
// authorizeDocRefs 收集 spec 中的 doc-ref docId 并经 Host 工件面做 owner 作用域解析。
func authorizeDocRefs(raw []byte, h extension.Host) []string {
var spec struct {
Nodes []core.UINode `json:"nodes"`
}
if err := json.Unmarshal(raw, &spec); err != nil {
return []string{"spec: " + err.Error()}
}
var errs []string
var walk func(n core.UINode, path string)
walk = func(n core.UINode, path string) {
if n.Type == core.UIDocRef && n.DocID != "" {
if _, err := h.ArtifactGet(string(n.DocID)); err != nil {
errs = append(errs, fmt.Sprintf("%s: doc-ref 引用了不存在或无权限的工件 %q", path, n.DocID))
}
}
for i, c := range n.Children {
walk(c, fmt.Sprintf("%s/child[%d]", path, i))
}
if n.Compare != nil {
for side, sd := range map[string]core.UICompareSide{"left": n.Compare.Left, "right": n.Compare.Right} {
for i, c := range sd.Nodes {
walk(c, fmt.Sprintf("%s/%s[%d]", path, side, i))
}
}
}
if n.Tabs != nil {
for i, tb := range n.Tabs.Tabs {
for j, c := range tb.Nodes {
walk(c, fmt.Sprintf("%s/tab[%d][%d]", path, i, j))
}
}
}
}
for i, n := range spec.Nodes {
walk(n, fmt.Sprintf("$[%d]", i))
}
return errs
}
+389
View File
@@ -0,0 +1,389 @@
package genui
// U3 红测(提案 U3 验证行):能力缺失明确拒绝;未启用拒绝;伪造 doc-ref 拒绝;
// 目录违规返回字段级清单(供模型修复重提);成功路径经 EXT-17 提交并携带观察 epoch。
import (
"context"
"encoding/json"
"fmt"
"path/filepath"
"strings"
"testing"
"agent-system-backend/internal/store"
"agent-system-backend/internal/surfacesvc"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
// realSvc 让 fakeHost.SurfaceSave 走真实 surfacesvc(校验/守卫行为与生产一致)。
var realSvc *surfacesvc.Service
func newRealSvc(t *testing.T) {
t.Helper()
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { st.Close() })
realSvc = surfacesvc.New(st, nil)
if _, err := realSvc.SetActivation(context.Background(), "alice", PluginID, true); err != nil {
t.Fatal(err)
}
}
type fakeHost struct {
extension.Host // 未覆盖方法仅测试未触达
enabled bool
epoch int64
saved []extension.SurfaceSaveRequest
artifacts map[string]bool
}
func (h *fakeHost) SurfaceActivation(ctx context.Context) (bool, int64, error) {
return h.enabled, h.epoch, nil
}
func (h *fakeHost) SurfaceSave(ctx context.Context, req extension.SurfaceSaveRequest) (core.SurfaceSnapshot, error) {
h.saved = append(h.saved, req)
return realSvc.Submit(ctx, "alice", PluginID, surfacesvc.SubmitRequest{
SurfaceID: req.SurfaceID, ConversationID: req.ConversationID,
BaseRevision: req.BaseRevision, ObservedEpoch: req.ObservedEpoch,
SemanticText: req.SemanticText, Spec: req.Spec,
})
}
func (h *fakeHost) SurfaceGet(ctx context.Context, surfaceID core.ID) (core.SurfaceSnapshot, error) {
return realSvc.Get(ctx, "alice", surfaceID)
}
func (h *fakeHost) SurfaceList(ctx context.Context, conversationID core.ID) ([]core.SurfaceMeta, error) {
return realSvc.List(ctx, "alice", conversationID)
}
func (h *fakeHost) ArtifactGet(id string) (core.Artifact, error) {
if h.artifacts[id] {
return core.Artifact{ID: core.ID(id)}, nil
}
return core.Artifact{}, store.ErrNotFound
}
func validSpec() map[string]any {
return map[string]any{
"title": "选型对比",
"nodes": []any{
map[string]any{"type": "section", "id": "n1", "title": "概览", "children": []any{
map[string]any{"type": "markdown", "id": "n1a", "source": []any{"doc:a"}, "markdown": "结论先行。"},
}},
map[string]any{"type": "table", "id": "n2", "table": map[string]any{
"columns": []any{map[string]any{"key": "k", "label": "维度"}},
"rows": []any{map[string]any{"cells": []any{[]any{"k", "价格"}}}},
}},
},
}
}
func reqWith(spec map[string]any, extra ...map[string]any) extension.Request {
params := map[string]any{"spec": spec}
for _, e := range extra {
for k, v := range e {
params[k] = v
}
}
return extension.Request{ID: "r1", Params: params, Owner: "alice", ConversationID: "c1"}
}
func TestPresentGuards(t *testing.T) {
newRealSvc(t)
ctx := context.Background()
// 能力缺失(Host 未实现 SurfaceHost)→ 明确报错
if _, err := present(ctx, plainHost{}, reqWith(validSpec())); err == nil {
t.Fatal("capability-missing host should fail explicitly")
}
// 未启用 → 拒绝
h := &fakeHost{enabled: false, artifacts: map[string]bool{}}
if _, err := present(ctx, h, reqWith(validSpec())); err == nil {
t.Fatal("disabled plugin should reject submit")
}
// 伪造 doc-ref:工件不存在 → 字段级工具错误
h = &fakeHost{enabled: true, epoch: 1, artifacts: map[string]bool{}}
spec := map[string]any{"title": "t", "nodes": []any{map[string]any{"type": "doc-ref", "docId": "art_ghost"}}}
res, err := present(ctx, h, reqWith(spec))
if err != nil {
t.Fatalf("doc-ref violation should be tool-error, got %v", err)
}
body, _ := json.Marshal(res)
if !strings.Contains(string(body), "art_ghost") {
t.Fatalf("field errors should name the bad docId: %s", body)
}
// 目录违规(未知字段 color)→ 字段级清单
bad := map[string]any{"title": "x", "nodes": []any{map[string]any{"type": "list", "items": []any{"a"}, "color": "red"}}}
res, err = present(ctx, h, reqWith(bad))
if err != nil {
t.Fatalf("invalid spec should be tool-error, got %v", err)
}
te, ok := res.(toolErrorBody)
if !ok || te.OK || len(te.Details) == 0 {
t.Fatalf("expected field-level details, got %+v", res)
}
}
func TestPresentSubmitCarriesEpochAndRevises(t *testing.T) {
newRealSvc(t)
ctx := context.Background()
h := &fakeHost{enabled: true, epoch: 1, artifacts: map[string]bool{}}
res, err := present(ctx, h, reqWith(validSpec()))
if err != nil {
t.Fatal(err)
}
m, _ := res.(map[string]any)
if m["ok"] != true || fmt.Sprint(m["revision"]) != "1" {
t.Fatalf("submit result: %+v", m)
}
if len(h.saved) != 1 || h.saved[0].ObservedEpoch != 1 {
t.Fatalf("submit must carry observed epoch: %+v", h.saved)
}
// 修订:带 surfaceId + baseRevision → revision 2
res2, err := present(ctx, h, reqWith(validSpec(), map[string]any{
"surfaceId": m["surfaceId"], "baseRevision": float64(1)}))
if err != nil {
t.Fatal(err)
}
m2, _ := res2.(map[string]any)
if fmt.Sprint(m2["revision"]) != "2" {
t.Fatalf("revision should advance to 2: %+v", m2)
}
// 禁用后:执行端拒绝(闸外第二道守卫)
h.enabled = false
if _, err := present(ctx, h, reqWith(validSpec())); err == nil {
t.Fatal("post-disable submit should reject")
}
}
func TestRegisterSpecShape(t *testing.T) {
var captured []json.RawMessage
reg := &fakeRegistry{captureList: &captured}
if err := register(reg); err != nil {
t.Fatal(err)
}
byID := map[string]json.RawMessage{}
for _, raw := range captured {
var id struct {
ID string `json:"id"`
}
if err := json.Unmarshal(raw, &id); err != nil {
t.Fatal(err)
}
byID[id.ID] = raw
}
var spec struct {
ID string `json:"id"`
Description string `json:"description"`
ExposeAsTool map[string]any `json:"exposeAsTool"`
}
if raw, ok := byID[CommandPresent]; !ok {
t.Fatalf("present command not registered: %v", captured)
} else if err := json.Unmarshal(raw, &spec); err != nil {
t.Fatal(err)
}
if spec.ID != CommandPresent || spec.ExposeAsTool == nil {
t.Fatalf("present must expose as tool: %s", byID[CommandPresent])
}
for _, need := range []string{"section{", "compare{", "steps{", "≤200 节点", "generative-ui.submit", "builtin.help", "system.intent"} {
if !strings.Contains(spec.Description, need) {
t.Fatalf("catalog incomplete: missing %q", need)
}
}
var submit struct {
ExposeAsTool map[string]any `json:"exposeAsTool"`
}
if raw, ok := byID[CommandSubmit]; !ok {
t.Fatalf("submit command not registered: %v", captured)
} else {
_ = json.Unmarshal(raw, &submit)
if submit.ExposeAsTool != nil {
t.Fatal("submit must NOT expose as tool (button-only; model cannot call)")
}
}
}
type plainHost struct{ extension.Host }
type fakeRegistry struct{ captureList *[]json.RawMessage }
func (r *fakeRegistry) AddCommandHandler(spec json.RawMessage, h extension.Handler) error {
*r.captureList = append(*r.captureList, spec)
return nil
}
func (r *fakeRegistry) AddSkill(name, markdown string) error { return nil }
func (r *fakeRegistry) AddCommand(spec json.RawMessage) error { return nil }
func (r *fakeRegistry) Note(kind, name string, p json.RawMessage) error { return nil }
func (r *fakeRegistry) RegisterResourceAuthorizer(fn extension.ResourceAuthorizer) error {
return nil
}
func (r *fakeRegistry) RegisterResourceAuthorizerV2(fn extension.ResourceAuthorizerV2) error {
return nil
}
func (r *fakeRegistry) AddIntent(spec json.RawMessage) error { return nil }
func (r *fakeRegistry) AddAdminZone(spec json.RawMessage) error { return nil }
func TestPresentRejectsUnknownActionIDs(t *testing.T) {
newRealSvc(t)
ctx := context.Background()
h := &fakeHost{enabled: true, epoch: 1, artifacts: map[string]bool{}}
spec := map[string]any{"title": "t", "nodes": []any{map[string]any{
"type": "actions", "id": "a1",
"actions": []any{
map[string]any{"id": "shell.exec", "label": "危险"},
map[string]any{"id": "generative-ui.submit", "label": "提交"},
}}}}
res, err := present(ctx, h, reqWith(spec))
if err != nil {
t.Fatalf("should be tool error, got %v", err)
}
te, ok := res.(toolErrorBody)
if !ok || te.OK || len(te.Details) == 0 {
t.Fatalf("expected field errors, got %+v", res)
}
if !strings.Contains(strings.Join(te.Details, ";"), "shell.exec") {
t.Fatalf("errors should name the unknown action: %+v", te)
}
}
func TestValidateActionIDsAllowsBClassHelpAndIntent(t *testing.T) {
raw, _ := json.Marshal(map[string]any{"title": "t", "nodes": []any{map[string]any{
"type": "actions", "id": "a1",
"actions": []any{
map[string]any{"id": ActionHelp, "label": "帮助"},
map[string]any{"id": ActionIntent, "label": "意图"},
map[string]any{"id": CommandSubmit, "label": "提交"},
}}}})
if errs := ValidateActionIDs(raw); len(errs) != 0 {
t.Fatalf("B-class help/intent + submit should pass, got %v", errs)
}
}
func TestSubmitFormStoresLedger(t *testing.T) {
// C 类:表单数据落插件 KV 台账(键含 requestId 天然幂等)。
var kv map[string]string = map[string]string{}
h := &ledgerHost{kv: kv}
res, err := submitForm(context.Background(), h, extension.Request{Params: map[string]any{
"surfaceId": "surf_x", "requestId": "req-1", "revision": float64(1),
"data": map[string]any{"years": "5"},
}})
if err != nil {
t.Fatal(err)
}
m, _ := res.(map[string]any)
if m["ok"] != true || m["stored"] != true {
t.Fatalf("submit result: %+v", m)
}
if v, ok := kv["form:surf_x:req-1"]; !ok || !strings.Contains(v, `"years":"5"`) {
t.Fatalf("ledger missing data: %v", kv)
}
// 幂等重放同键覆盖同值
if _, err := submitForm(context.Background(), h, extension.Request{Params: map[string]any{
"surfaceId": "surf_x", "requestId": "req-1", "data": map[string]any{"years": "5"},
}}); err != nil {
t.Fatal(err)
}
}
type ledgerHost struct {
extension.Host
kv map[string]string
}
func (h *ledgerHost) KVPut(key, value string) error { h.kv[key] = value; return nil }
func (h *ledgerHost) Audit(action, target, evidence string) error { return nil }
func TestNormalizeChildrenShape(t *testing.T) {
// 真实偏差(2026-10-11 dev 实测):模型把 children 写成按 key 编号的对象映射。
in := map[string]any{
"type": "section", "id": "n1", "title": "结论",
"children": map[string]any{
"item": map[string]any{"type": "markdown", "id": "m1", "markdown": "结论先行"},
},
}
out := normalizeSpecParam(map[string]any{"title": "t", "nodes": []any{in}})
nodes := out.(map[string]any)["nodes"].([]any)
ch := nodes[0].(map[string]any)["children"].([]any)
if len(ch) != 1 || ch[0].(map[string]any)["id"] != "m1" {
t.Fatalf("children should be repaired to array, got %v", ch)
}
}
// 真实偏差(2026-10-12 dev 实测,事件 seq 1395/2171):MiniMax-M3 对数组字段输出
// XML-RPC 风格 {"item": …} 包装——kv/cells/items 整字段被包一层 {"item":[…]},
// 数组元素再被包 {"item":["k","v"]}/{"item":"v"};表格行 cells 另有与列等长的
// 裸标量数组形态。修复层须解包并按列 key 配对,present 才能落库。
func TestPresentRepairsItemWrappers(t *testing.T) {
newRealSvc(t)
h := &fakeHost{enabled: true, epoch: 1}
spec := map[string]any{
"title": "本月支出结构分析(示例数据演示)",
"nodes": []any{
map[string]any{"type": "markdown", "id": "disclaimer", "markdown": "示例数据,非真实账单。"},
map[string]any{"type": "kv", "id": "kv1", "kv": map[string]any{"item": []any{
map[string]any{"item": []any{"月份", "2026-10"}},
map[string]any{"item": []any{"总支出", "¥10,000"}},
}}},
map[string]any{"type": "list", "id": "l1", "items": map[string]any{"item": []any{
map[string]any{"item": "餐饮 35%"},
map[string]any{"item": "住房 25%"},
}}},
map[string]any{"type": "table", "id": "t1", "table": map[string]any{
"columns": []any{
map[string]any{"key": "cat", "label": "分类"},
map[string]any{"key": "amt", "label": "金额"},
map[string]any{"key": "pct", "label": "占比"},
},
"rows": []any{
map[string]any{"cells": map[string]any{"item": []any{
map[string]any{"item": "餐饮"}, map[string]any{"item": "¥3,500"}, map[string]any{"item": "35%"},
}}},
},
}},
},
}
res, err := present(context.Background(), h, reqWith(spec))
if err != nil {
t.Fatalf("present: %v", err)
}
if body, ok := res.(toolErrorBody); ok {
t.Fatalf("present should repair item wrappers, got %s %v", body.Error, body.Details)
}
if len(h.saved) != 1 {
t.Fatalf("expected one saved surface, got %d", len(h.saved))
}
var saved core.SurfaceSpec
if err := json.Unmarshal(h.saved[0].Spec, &saved); err != nil {
t.Fatalf("saved spec should unmarshal: %v", err)
}
if saved.Nodes[1].KV[0] != [2]string{"月份", "2026-10"} || saved.Nodes[1].KV[1][1] != "¥10,000" {
t.Fatalf("kv pairs should be unwrapped, got %v", saved.Nodes[1].KV)
}
if len(saved.Nodes[2].Items) != 2 || saved.Nodes[2].Items[0] != "餐饮 35%" {
t.Fatalf("list items should be unwrapped, got %v", saved.Nodes[2].Items)
}
cells := saved.Nodes[3].Table.Rows[0].Cells
if len(cells) != 3 || cells[0] != [2]string{"cat", "餐饮"} || cells[2][1] != "35%" {
t.Fatalf("table cells should pair with column keys, got %v", cells)
}
}
// 长度不匹配的裸 cells 不做发明性配对:保留原样交校验器给字段级错误。
func TestRepairTableCellsLeavesMismatchedShapes(t *testing.T) {
in := map[string]any{"type": "table", "id": "t1", "table": map[string]any{
"columns": []any{map[string]any{"key": "a"}, map[string]any{"key": "b"}},
"rows": []any{map[string]any{"cells": []any{"x", "y", "z"}}},
}}
out := normalizeSpecParam(map[string]any{"title": "t", "nodes": []any{in}})
rows := out.(map[string]any)["nodes"].([]any)[0].(map[string]any)["table"].(map[string]any)["rows"].([]any)
got := rows[0].(map[string]any)["cells"].([]any)
if len(got) != 3 || got[0] != "x" {
t.Fatalf("mismatched scalars should stay untouched, got %v", got)
}
}
+251
View File
@@ -0,0 +1,251 @@
package genui
// U3 完成条件回放(真实模型,env 门控):任务→规格→保存→(语义投影回灌)→
// 继续修改→新修订。跑生产代码路径:present 处理器 + surfacesvc + store + ctxasm
// 投影(AssembleOptions.Surfaces)。模型 MiniMax-M3(2026-10-11 裁定)。
//
// 启用:GENUI_MINIMAX_API_KEY=<key> go test ./internal/genui/ -run TestRealModelSurfaceLoop -v -count=1
// 未设 env 时跳过(CI 无凭据;本地收据经 make evidence 记录)。
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"time"
"agent-system-backend/internal/ctxasm"
"agent-system-backend/internal/store"
"agent-system-backend/internal/surfacesvc"
"agent-system-backend/pkg/extension"
core "agent-system-core"
)
const materials = `材料:
【doc:m1】甲方案:报价 ¥8/人/天;私有化可选;近半年发布 6 次,平均修复 2 天。
【doc:m2】乙方案:报价 ¥12/人/天;仅 SaaS;万行表格场景无官方基准;升级断层(近两季仅小版本)。
【doc:m3】尽调备忘:合规缺口(高)——数据出境条款未覆盖;培训成本(低)——术语不一致。`
type chatResp struct {
Choices []struct {
Message struct {
ToolCalls []struct {
Function struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
} `json:"function"`
} `json:"tool_calls"`
} `json:"message"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
} `json:"usage"`
}
func callModelWithRepair(t *testing.T, ctx context.Context, system, user string, try func(map[string]any) (any, error)) (map[string]any, any, chatResp) {
t.Helper()
var usage chatResp
params, r := callModel(t, ctx, system, user)
usage.Usage.PromptTokens += r.Usage.PromptTokens
usage.Usage.CompletionTokens += r.Usage.CompletionTokens
res, err := try(params)
if err == nil {
if te, ok := res.(toolErrorBody); !ok || te.OK {
return params, res, usage
} else {
res = te
}
} else if res == nil {
t.Fatalf("dispatch error: %v", err)
}
// 修复轮:字段级错误回传,模型重提一次(KTD2)。
fixUser := user + "\n\n上一次提交被拒绝:" + fmt.Sprint(res) + "\n请修正后重新调用工具。"
params2, r2b := callModel(t, ctx, system, fixUser)
usage.Usage.PromptTokens += r2b.Usage.PromptTokens
usage.Usage.CompletionTokens += r2b.Usage.CompletionTokens
res2, err2 := try(params2)
if err2 != nil {
t.Fatalf("repair dispatch error: %v", err2)
}
if te, ok := res2.(toolErrorBody); ok && !te.OK {
t.Fatalf("repair still invalid: %+v", te)
}
return params2, res2, usage
}
func callModel(t *testing.T, ctx context.Context, system, user string) (map[string]any, chatResp) {
t.Helper()
key := os.Getenv("GENUI_MINIMAX_API_KEY")
tool := map[string]any{"type": "function", "function": map[string]any{
"name": CommandPresent, "description": Catalog, "parameters": map[string]any{
"type": "object",
"properties": map[string]any{
"spec": map[string]any{"type": "object"},
"surfaceId": map[string]any{"type": "string"},
"baseRevision": map[string]any{"type": "integer"},
"summary": map[string]any{"type": "string"},
},
"required": []string{"spec"},
}}}
body, _ := json.Marshal(map[string]any{
"model": "MiniMax-M3", "temperature": 0.3, "max_tokens": 14000,
"tools": []any{tool}, "tool_choice": map[string]any{"type": "function", "function": map[string]any{"name": CommandPresent}},
"messages": []any{
map[string]any{"role": "system", "content": system},
map[string]any{"role": "user", "content": user},
}})
req, _ := http.NewRequestWithContext(ctx, "POST", "https://api.minimaxi.com/v1/chat/completions", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
var out chatResp
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("model call: %v", err)
}
defer resp.Body.Close()
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode: %v", err)
}
if len(out.Choices) == 0 || len(out.Choices[0].Message.ToolCalls) == 0 {
t.Fatalf("no tool call in response")
}
var params map[string]any
if err := json.Unmarshal([]byte(out.Choices[0].Message.ToolCalls[0].Function.Arguments), &params); err != nil {
t.Fatalf("tool args: %v", err)
}
return params, out
}
func TestRealModelSurfaceLoop(t *testing.T) {
if os.Getenv("GENUI_MINIMAX_API_KEY") == "" {
t.Skip("GENUI_MINIMAX_API_KEY not set; real-model replay runs locally with recorded receipts")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
st, err := store.OpenSQLite(filepath.Join(t.TempDir(), "replay.db"))
if err != nil {
t.Fatal(err)
}
defer st.Close()
svc := surfacesvc.New(st, nil)
if _, err := svc.SetActivation(ctx, "alice", PluginID, true); err != nil {
t.Fatal(err)
}
h := &realHost{svc: svc, artifacts: map[string]bool{}}
// 回合 1:任务→规格→保存
sys1 := "你是本产品中的助手。用户需要一页比较视图时,用 generative-ui.present 提交页面。"
try1 := func(p map[string]any) (any, error) {
return present(ctx, h, extension.Request{ID: "t1", Params: p, Owner: "alice", ConversationID: "c1"})
}
_, res1, r1 := callModelWithRepair(t, ctx, sys1, materials+"\n\n目标:给决策会准备风险分析页(按严重度组织,标注来源)。", try1)
m1, _ := res1.(map[string]any)
if m1["ok"] != true {
t.Fatalf("turn1 not accepted: %+v", res1)
}
snap1, err := svc.Get(ctx, "alice", core.ID(fmt.Sprint(m1["surfaceId"])))
if err != nil {
t.Fatal(err)
}
t.Logf("turn1: surface=%s revision=%d nodes=%d tokens=%d/%d",
m1["surfaceId"], snap1.Revision, countNodes(snap1), r1.Usage.PromptTokens, r1.Usage.CompletionTokens)
// 回合 2:语义投影回灌 + 修订指令 → 新修订
msgs2, _ := ctxasm.AssembleInput(core.AgentProfile{}, nil, nil,
core.Message{Role: core.RoleUser, Parts: []core.Part{{Type: core.PartText, Text: "改为实施顺序"}}},
nil, nil, nil, nil, nil, nil, ctxasm.AssembleOptions{Surfaces: []core.SurfaceSnapshot{snap1}})
var sysParts []string
for _, m := range msgs2 {
if string(m.Role) == "system" {
sysParts = append(sysParts, m.Content)
}
}
sys2 := strings.Join(sysParts, "\n")
try2 := func(p map[string]any) (any, error) {
p["surfaceId"] = m1["surfaceId"]
p["baseRevision"] = float64(snap1.Revision)
return present(ctx, h, extension.Request{ID: "t2", Params: p, Owner: "alice", ConversationID: "c1"})
}
_, res2, r2 := callModelWithRepair(t, ctx, sys1+"\n"+sys2, materials+"\n\n目标:改为实施顺序(阶段与依赖),修订已有页面。", try2)
m2, _ := res2.(map[string]any)
if fmt.Sprint(m2["revision"]) != fmt.Sprint(snap1.Revision+1) {
t.Fatalf("turn2 should revise to %d, got %+v", snap1.Revision+1, res2)
}
snap2, _ := svc.Get(ctx, "alice", core.ID(fmt.Sprint(m1["surfaceId"])))
types2 := dominantTypes(snap2)
same := len(types2) == len(snap1.Spec.Nodes)
if same {
for i := range types2 {
if types2[i] != string(snap1.Spec.Nodes[i].Type) {
same = false
break
}
}
}
if len(snap2.Spec.Nodes) == 0 || (snap2.Spec.Title == snap1.Spec.Title && same) {
t.Fatalf("turn2 structure did not change: %v", types2)
}
t.Logf("turn2: revision=%d title=%q types=%v tokens=%d/%d",
snap2.Revision, snap2.Spec.Title, types2, r2.Usage.PromptTokens, r2.Usage.CompletionTokens)
totalIn := r1.Usage.PromptTokens + r2.Usage.PromptTokens
totalOut := r1.Usage.CompletionTokens + r2.Usage.CompletionTokens
cost := float64(totalIn)/1e6*0.3*7.2 + float64(totalOut)/1e6*1.2*7.2
t.Logf("cost: in=%d out=%d ≈¥%.3f (M3 $0.3/$1.2 per M, 7.2)", totalIn, totalOut, cost)
}
type realHost struct {
extension.Host
svc *surfacesvc.Service
artifacts map[string]bool
}
func (h *realHost) SurfaceActivation(ctx context.Context) (bool, int64, error) {
row, err := h.svc.Activation(ctx, "alice", PluginID)
return row.Enabled, row.Epoch, err
}
func (h *realHost) SurfaceSave(ctx context.Context, req extension.SurfaceSaveRequest) (core.SurfaceSnapshot, error) {
return h.svc.Submit(ctx, "alice", PluginID, surfacesvc.SubmitRequest{
SurfaceID: req.SurfaceID, ConversationID: req.ConversationID, BaseRevision: req.BaseRevision,
ObservedEpoch: req.ObservedEpoch, SemanticText: req.SemanticText, Spec: req.Spec})
}
func (h *realHost) SurfaceGet(ctx context.Context, id core.ID) (core.SurfaceSnapshot, error) {
return h.svc.Get(ctx, "alice", id)
}
func (h *realHost) SurfaceList(ctx context.Context, cid core.ID) ([]core.SurfaceMeta, error) {
return h.svc.List(ctx, "alice", cid)
}
func (h *realHost) ArtifactGet(id string) (core.Artifact, error) {
if h.artifacts[id] {
return core.Artifact{ID: core.ID(id)}, nil
}
return core.Artifact{}, store.ErrNotFound
}
func countNodes(s core.SurfaceSnapshot) int {
var n int
var walk func(u core.UINode)
walk = func(u core.UINode) {
n++
for _, c := range u.Children {
walk(c)
}
}
for _, x := range s.Spec.Nodes {
walk(x)
}
return n
}
func dominantTypes(s core.SurfaceSnapshot) []string {
var out []string
for _, n := range s.Spec.Nodes {
out = append(out, string(n.Type))
}
return out
}
+5
View File
@@ -0,0 +1,5 @@
package hostapi
import "encoding/json"
func jsonUnmarshal(b []byte, v any) error { return json.Unmarshal(b, v) }
+10 -2
View File
@@ -18,7 +18,10 @@ type Server struct {
Store *store.Store
Dispatch func(ctx context.Context, pluginID, owner, commandID string, params map[string]any) error
Search func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error)
Complete func(ctx context.Context, owner, prompt string) (string, error)
// SharedSearch EXT-8 余项:非 nil 时 search 端点改用之(组合根传入
// 「本人结果 + 适配器放行的共享追加」组合;nil=现状仅本人)。
SharedSearch func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error)
Complete func(ctx context.Context, owner, prompt string) (string, error)
}
func (s *Server) Handler() http.Handler {
@@ -143,7 +146,12 @@ func (s *Server) search(w http.ResponseWriter, r *http.Request) {
http.Error(w, "retriever unavailable", http.StatusServiceUnavailable)
return
}
hits, err := s.Search(r.Context(), s.owner(r), r.URL.Query().Get("q"))
// EXT-8 余项:共享版可用时整体切换(本人+共享追加)。
fn := s.Search
if s.SharedSearch != nil {
fn = s.SharedSearch
}
hits, err := fn(r.Context(), s.owner(r), r.URL.Query().Get("q"))
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
+49
View File
@@ -0,0 +1,49 @@
package hostapi
import (
"context"
"net/http"
"net/http/httptest"
"testing"
core "agent-system-core"
)
// EXT-8 余项:hostapi 检索面——SharedSearch 可用时整体切换(本人+适配器放行的
// 共享追加);不可用时维持现状(仅本人回调)。
func TestSharedSearchFace(t *testing.T) {
own := []core.RetrievalHit{{Doc: core.KnowledgeDoc{Owner: "bob", Chunk: "本人命中"}}}
sharedHit := core.RetrievalHit{Doc: core.KnowledgeDoc{Owner: "alice", ProjectID: "prj-s", Chunk: "共享命中"}}
srv := &Server{Token: "t", Search: func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error) {
return own, nil
}}
run := func() int {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/host/v1/knowledge/search?q=x", nil)
req.Header.Set("Authorization", "Bearer "+srv.Token)
srv.Handler().ServeHTTP(rec, req)
return rec.Code
}
if code := run(); code != http.StatusOK {
t.Fatalf("plain search: %d", code)
}
srv.SharedSearch = func(ctx context.Context, owner, text string) ([]core.RetrievalHit, error) {
return append(own, sharedHit), nil
}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/host/v1/knowledge/search?q=x", nil)
req.Header.Set("Authorization", "Bearer "+srv.Token)
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("shared search: %d %s", rec.Code, rec.Body.String())
}
var hits []core.RetrievalHit
if err := jsonUnmarshal(rec.Body.Bytes(), &hits); err != nil {
t.Fatal(err)
}
if len(hits) != 2 || hits[1].Doc.Owner != "alice" {
t.Fatalf("hits = %+v, want 本人+共享", hits)
}
}
+171 -60
View File
@@ -4,7 +4,6 @@
package ingest
import (
"archive/zip"
"bytes"
"encoding/xml"
"fmt"
@@ -34,11 +33,16 @@ func Parse(filename, mime string, content []byte) core.ParsedDocument {
doc.Meta.Confidence = 1.0
case "docx":
doc.Meta.Parser = "builtin:docx"
blocks, err := parseDocx(content)
if err != nil {
blocks, revised, err := parseDocxRevised(content)
switch {
case err != nil:
doc.Warnings = append(doc.Warnings, "docx 解析失败:"+err.Error())
doc.Meta.Confidence = 0.3
} else {
case revised:
doc.Blocks = blocks
doc.Warnings = append(doc.Warnings, docxRevisionWarning)
doc.Meta.Confidence = 0.85
default:
doc.Blocks = blocks
doc.Meta.Confidence = 0.92
}
@@ -52,6 +56,35 @@ func Parse(filename, mime string, content []byte) core.ParsedDocument {
doc.Blocks = blocks
doc.Meta.Confidence = 0.85
}
case "xlsx", "xlsm":
doc.Meta.Parser = "builtin:xlsx"
blocks, warns := parseXlsx(content)
doc.Blocks = blocks
doc.Warnings = append(doc.Warnings, warns...)
switch {
case len(blocks) == 0:
doc.Meta.Confidence = 0.3
case len(warns) > 0:
doc.Meta.Confidence = 0.6 // 超限截断:内容可用但注明不完整
default:
doc.Meta.Confidence = 0.9
}
case "pptx", "pptm", "ppsx":
doc.Meta.Parser = "builtin:pptx"
blocks, warns := parsePptx(content)
doc.Blocks = blocks
doc.Warnings = append(doc.Warnings, warns...)
switch {
case len(blocks) == 0:
doc.Meta.Confidence = 0.3
case len(warns) > 0:
doc.Meta.Confidence = 0.6
default:
doc.Meta.Confidence = 0.9
}
case "doc", "xls", "ppt":
doc.Meta.Parser = "builtin:legacy-office"
doc.Warnings = append(doc.Warnings, "旧版二进制格式 ."+ext+" 不解析:请转存为 ."+legacyOfficeTarget(ext)+" 后重传")
default:
if isImage(ext) {
doc.Warnings = append(doc.Warnings, "图片不走文本解析(OCR 为 MinerU 深度档,远期)")
@@ -133,22 +166,73 @@ func parsePlain(s string) []core.DocBlock {
// ---- docx(OOXML:zip 内 word/document.xml)----
// xpar 是解码后的段落。文字按文档顺序从任意深度的 w:t 收集,w:ins、
// w:hyperlink、w:fldSimple、w:sdt 等包裹层内的文字因此不会丢;被删除或
// 移走的原文(w:del、w:moveFrom)、域代码(w:instrText)与兼容回退分支
// (mc:Fallback,与 mc:Choice 内容重复)不进正文。
type xpar struct {
Props struct {
Style struct {
Val string `xml:"val,attr"`
} `xml:"pStyle"`
} `xml:"pPr"`
Runs []xrun `xml:"r"`
Style string
Text string
Revised bool // 段内含修订标记(插入/删除/移动)
}
type xrun struct {
Text []struct {
Content string `xml:",chardata"`
} `xml:"t"`
Break struct {
Val string `xml:"val,attr"`
} `xml:"br"`
// docxSkipText 子树里的文字不属于当前正文。
var docxSkipText = map[string]bool{
"del": true, "moveFrom": true, "instrText": true, "delInstrText": true, "Fallback": true,
}
// docxRevision 出现即说明文档带修订。
var docxRevision = map[string]bool{"ins": true, "del": true, "moveFrom": true, "moveTo": true}
// UnmarshalXML 按 token 流遍历段落子树。按固定层级(p/r/t)映射只认 w:p
// 的直接子级 w:r,修订插入、超链接与域结果里的文字会被静默跳过。
func (p *xpar) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error {
var sb strings.Builder
skip := 0
for {
tok, err := d.Token()
if err != nil {
return err
}
switch t := tok.(type) {
case xml.StartElement:
name := t.Name.Local
if docxRevision[name] {
p.Revised = true
}
switch {
case docxSkipText[name]:
skip++
case skip > 0:
case name == "pStyle" && p.Style == "":
p.Style = attrAnyNS(t, "val")
case name == "t":
var s string
if err := d.DecodeElement(&s, &t); err != nil {
return err
}
sb.WriteString(s)
}
case xml.EndElement:
if docxSkipText[t.Name.Local] && skip > 0 {
skip--
}
if t.Name == start.Name {
p.Text = sb.String()
return nil
}
}
}
}
// attrAnyNS 按本地名取属性值(w:val 与无前缀 val 都算)。
func attrAnyNS(st xml.StartElement, name string) string {
for _, a := range st.Attr {
if a.Name.Local == name {
return a.Value
}
}
return ""
}
type xtbl struct {
@@ -164,97 +248,108 @@ type xtc struct {
}
func parseDocx(content []byte) ([]core.DocBlock, error) {
zr, err := zip.NewReader(bytes.NewReader(content), int64(len(content)))
blocks, _, err := parseDocxRevised(content)
return blocks, err
}
// parseDocxRevised 同 parseDocx,另报告文档是否带修订标记。
func parseDocxRevised(content []byte) ([]core.DocBlock, bool, error) {
zr, err := openZip(content)
if err != nil {
return nil, fmt.Errorf("open zip: %w", err)
return nil, false, fmt.Errorf("open zip: %w", err)
}
var docXML []byte
for _, f := range zr.File {
if f.Name == "word/document.xml" {
rc, err := f.Open()
if err != nil {
return nil, err
}
docXML, _ = io.ReadAll(rc)
rc.Close()
break
}
}
if docXML == nil {
return nil, fmt.Errorf("word/document.xml not found")
docXML, err := zipPart(zr, "word/document.xml")
if err != nil {
return nil, false, err
}
return parseDocxBody(docXML)
}
// docxRevisionWarning 文档带修订时的显式告警(SR-A5:不静默)。
const docxRevisionWarning = "docx 含修订标记:正文按「接受全部修订」提取(插入保留、删除不计入);审阅修订过程请查看原件"
// parseDocxBody 按文档顺序解码 body 的段落与表格:token 流上遇到 p/tbl 就地
// DecodeElement(整元素消费,单元格内嵌 p 不会重复入列),表格与前后段落保持原位。
func parseDocxBody(docXML []byte) ([]core.DocBlock, error) {
// 段落可能包在 w:ins、w:sdt 等容器里,所以任意深度的 p 与 tbl 都识别;
// 返回值 revised 报告是否遇到修订标记。
func parseDocxBody(docXML []byte) (blocks []core.DocBlock, revised bool, err error) {
dec := xml.NewDecoder(bytes.NewReader(docXML))
var blocks []core.DocBlock
skip := 0
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return nil, fmt.Errorf("decode document.xml: %w", err)
return nil, revised, fmt.Errorf("decode document.xml: %w", err)
}
if end, ok := tok.(xml.EndElement); ok {
if docxSkipText[end.Name.Local] && skip > 0 {
skip--
}
continue
}
start, ok := tok.(xml.StartElement)
if !ok {
continue
}
switch start.Name.Local {
name := start.Name.Local
if docxRevision[name] {
revised = true
}
if docxSkipText[name] {
skip++
continue
}
if skip > 0 {
continue
}
switch name {
case "p":
var p xpar
if err := dec.DecodeElement(&p, &start); err != nil {
return nil, fmt.Errorf("decode paragraph: %w", err)
return nil, revised, fmt.Errorf("decode paragraph: %w", err)
}
text := paragraphText(p)
if strings.TrimSpace(text) == "" {
revised = revised || p.Revised
if strings.TrimSpace(p.Text) == "" {
continue
}
if lvl, ok := headingLevel(p.Props.Style.Val); ok {
blocks = append(blocks, core.DocBlock{Type: core.BlockHeading, Level: lvl, Text: text})
if lvl, ok := headingLevel(p.Style); ok {
blocks = append(blocks, core.DocBlock{Type: core.BlockHeading, Level: lvl, Text: p.Text})
} else {
blocks = append(blocks, core.DocBlock{Type: core.BlockPara, Text: text})
blocks = append(blocks, core.DocBlock{Type: core.BlockPara, Text: p.Text})
}
case "tbl":
var tbl xtbl
if err := dec.DecodeElement(&tbl, &start); err != nil {
return nil, fmt.Errorf("decode table: %w", err)
return nil, revised, fmt.Errorf("decode table: %w", err)
}
if rows := tableRows(tbl); len(rows) > 0 {
rows, tblRevised := tableRows(tbl)
revised = revised || tblRevised
if len(rows) > 0 {
blocks = append(blocks, core.DocBlock{Type: core.BlockTable, Rows: rows})
}
}
}
return blocks, nil
return blocks, revised, nil
}
func tableRows(tbl xtbl) [][]string {
func tableRows(tbl xtbl) ([][]string, bool) {
var rows [][]string
revised := false
for _, tr := range tbl.Rows {
row := []string{}
for _, tc := range tr.Cells {
cell := []string{}
for _, p := range tc.Paragraphs {
cell = append(cell, paragraphText(p))
revised = revised || p.Revised
cell = append(cell, p.Text)
}
row = append(row, strings.Join(cell, " "))
}
rows = append(rows, row)
}
return rows
}
func paragraphText(p xpar) string {
var sb strings.Builder
for _, r := range p.Runs {
for _, t := range r.Text {
sb.WriteString(t.Content)
}
}
return sb.String()
return rows, revised
}
func headingLevel(style string) (int, bool) {
@@ -322,13 +417,29 @@ func isMostlyPrintable(s string) bool {
return nonPrint*10 < len(s)
}
// legacyOfficeTarget 旧版二进制 Office 格式的现代等价格式(提示转存用)。
func legacyOfficeTarget(ext string) string {
switch ext {
case "doc":
return "docx"
case "xls":
return "xlsx"
case "ppt":
return "pptx"
}
return ext
}
// ParsersDescription 返回解析器注册表描述(来源标记;设置页/文档用)。
func ParsersDescription() []core.ParserDesc {
return []core.ParserDesc{
{ID: "builtin:md", Formats: []string{"md", "markdown"}, Source: core.SourceBuiltin},
{ID: "builtin:txt", Formats: []string{"txt", "csv", "log"}, Source: core.SourceBuiltin},
{ID: "builtin:txt", Formats: []string{"txt", "text", "csv", "log"}, Source: core.SourceBuiltin},
{ID: "builtin:docx", Formats: []string{"docx"}, Source: core.SourceBuiltin},
{ID: "builtin:pdf", Formats: []string{"pdf"}, Source: core.SourceBuiltin, Note: "文本层;扫描件为远期 MinerU 深度档"},
{ID: "builtin:xlsx", Formats: []string{"xlsx", "xlsm"}, Source: core.SourceBuiltin},
{ID: "builtin:pptx", Formats: []string{"pptx", "pptm", "ppsx"}, Source: core.SourceBuiltin},
{ID: "builtin:legacy-office", Formats: []string{"doc", "xls", "ppt"}, Source: core.SourceBuiltin, Note: "不解析;提示转存现代格式后重传"},
{ID: "mineru", Formats: []string{"pdf", "png", "jpg"}, Source: core.SourcePlugin, Note: "远期可选:深度解析与交叉验证(默认不部署)"},
}
}
+62
View File
@@ -112,6 +112,68 @@ func TestParseDocx(t *testing.T) {
}
}
// docxWith 把 body 片段包成最小 docx(zip 内仅 word/document.xml)。
func docxWith(t *testing.T, body string) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
f, err := zw.Create("word/document.xml")
if err != nil {
t.Fatal(err)
}
f.Write([]byte(`<?xml version="1.0"?><w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"><w:body>` + body + `</w:body></w:document>`))
zw.Close()
return buf.Bytes()
}
// 修订与超链接:插入/超链接/域结果内的文字必须保留;删除文字不进正文;
// 含修订时显式告警(SR-A5:禁止静默降级)。
func TestParseDocxTrackedChangesAndHyperlinks(t *testing.T) {
body := `<w:p><w:r><w:t>周期</w:t></w:r><w:ins w:id="1" w:author="a"><w:r><w:t>18</w:t></w:r></w:ins>` +
`<w:del w:id="2" w:author="a"><w:r><w:delText>12</w:delText></w:r></w:del><w:r><w:t>个月</w:t></w:r></w:p>` +
`<w:p><w:r><w:t>见</w:t></w:r><w:hyperlink r:id="rId5"><w:r><w:t>附件A</w:t></w:r></w:hyperlink></w:p>` +
`<w:p><w:fldSimple w:instr="PAGE"><w:r><w:t>3</w:t></w:r></w:fldSimple><w:r><w:t>页</w:t></w:r></w:p>` +
`<w:tbl><w:tr><w:tc><w:p><w:ins w:id="3" w:author="a"><w:r><w:t>新增项</w:t></w:r></w:ins></w:p></w:tc></w:tr></w:tbl>`
doc := Parse("修订稿.docx", "", docxWith(t, body))
var paras []string
var cell string
for _, b := range doc.Blocks {
switch b.Type {
case core.BlockPara:
paras = append(paras, b.Text)
case core.BlockTable:
if len(b.Rows) > 0 && len(b.Rows[0]) > 0 {
cell = b.Rows[0][0]
}
}
}
want := []string{"周期18个月", "见附件A", "3页"}
if strings.Join(paras, "|") != strings.Join(want, "|") {
t.Errorf("paragraphs: got %q want %q", paras, want)
}
if cell != "新增项" {
t.Errorf("table cell inside w:ins: got %q", cell)
}
if strings.Contains(doc.PlainText(), "12个月") {
t.Errorf("deleted text leaked into body: %q", doc.PlainText())
}
warned := false
for _, w := range doc.Warnings {
warned = warned || strings.Contains(w, "修订")
}
if !warned {
t.Errorf("tracked changes must be reported in warnings, got %v", doc.Warnings)
}
}
// 无修订的普通文档不产生修订告警(避免噪音)。
func TestParseDocxNoRevisionNoWarning(t *testing.T) {
doc := Parse("干净.docx", "", docxWith(t, `<w:p><w:r><w:t>正文</w:t></w:r></w:p>`))
if len(doc.Warnings) != 0 {
t.Errorf("clean docx warnings: %v", doc.Warnings)
}
}
func TestParsePDFInvalid(t *testing.T) {
doc := Parse("scan.pdf", "application/pdf", []byte("not a pdf"))
if len(doc.Warnings) == 0 || doc.Meta.Confidence > 0.5 {
+140
View File
@@ -0,0 +1,140 @@
package ingest
// OOXML 公共件:zip 部件读取(解压上限防 zip 炸弹)、关系部件解析、
// r:id 属性与相对路径归一。docx/xlsx/pptx 解析器共用;纯标准库。
import (
"archive/zip"
"bytes"
"encoding/xml"
"fmt"
"io"
"path"
"strings"
)
// ooxmlPartLimit 单个 zip 部件的解压上限(测试可覆盖)。
var ooxmlPartLimit int64 = 64 << 20
func openZip(content []byte) (*zip.Reader, error) {
return zip.NewReader(bytes.NewReader(content), int64(len(content)))
}
// zipPart 按名称读取部件;解压超过 ooxmlPartLimit 视为异常拒绝。
func zipPart(zr *zip.Reader, name string) ([]byte, error) {
for _, f := range zr.File {
if f.Name != name {
continue
}
rc, err := f.Open()
if err != nil {
return nil, err
}
b, err := io.ReadAll(io.LimitReader(rc, ooxmlPartLimit+1))
rc.Close()
if err != nil {
return nil, err
}
if int64(len(b)) > ooxmlPartLimit {
return nil, fmt.Errorf("part %s exceeds %d bytes decompression limit", name, ooxmlPartLimit)
}
return b, nil
}
return nil, fmt.Errorf("part %s not found", name)
}
// relsPath 计算部件对应的关系部件名(xl/workbook.xml → xl/_rels/workbook.xml.rels)。
func relsPath(part string) string {
dir, base := path.Split(part)
return dir + "_rels/" + base + ".rels"
}
type ooxmlRel struct {
ID, Target, Type string
}
// parseRels 解析部件的关系表;无关系部件返回空表(不是错误)。
func parseRels(zr *zip.Reader, part string) (map[string]ooxmlRel, error) {
data, err := zipPart(zr, relsPath(part))
if err != nil {
return map[string]ooxmlRel{}, nil
}
dec := xml.NewDecoder(bytes.NewReader(data))
out := map[string]ooxmlRel{}
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
st, ok := tok.(xml.StartElement)
if !ok || st.Name.Local != "Relationship" {
continue
}
r := ooxmlRel{}
for _, a := range st.Attr {
switch a.Name.Local {
case "Id":
r.ID = a.Value
case "Target":
r.Target = a.Value
case "Type":
r.Type = a.Value
}
}
if r.ID != "" {
out[r.ID] = r
}
}
return out, nil
}
// resolveRelTarget 把关系 target 解析为 zip 部件名:绝对 target(/xl/…)去
// 前导斜杠;相对 target 相对 partDir;归一化 ../ 与 ./ 段。
func resolveRelTarget(partDir, target string) string {
t := target
if !strings.HasPrefix(target, "/") {
t = partDir + "/" + target
}
var seg []string
for _, s := range strings.Split(t, "/") {
switch s {
case "", ".":
case "..":
if len(seg) > 0 {
seg = seg[:len(seg)-1]
}
default:
seg = append(seg, s)
}
}
return strings.Join(seg, "/")
}
// ridAttr 取 r:id:局部名 id 且带命名空间前缀(Transitional 与 Strict OOXML
// 的关系引用都满足该形态;无前缀的 id 是别的编号,不误取)。
func ridAttr(st xml.StartElement) string {
for _, a := range st.Attr {
if a.Name.Local == "id" && a.Name.Space != "" {
return a.Value
}
}
return ""
}
// attrLocal 取无命名空间前缀的属性值。
func attrLocal(st xml.StartElement, name string) string {
for _, a := range st.Attr {
if a.Name.Local == name && a.Name.Space == "" {
return a.Value
}
}
return ""
}
// relIsType 判断关系类型是否以标准关系类型后缀结尾(/worksheet、/slide 等)。
func relIsType(rel ooxmlRel, suffix string) bool {
return strings.HasSuffix(rel.Type, suffix)
}
+140
View File
@@ -0,0 +1,140 @@
package ingest
// 真实生成器轮测:openpyxl / python-pptx 现场生成 xlsx/pptx 再解析,防止
// 手工夹具只验证"自己写的 XML"。生成器缺失时 t.Skip(诚实出口);CI 已装
// 这两个包(.gitea/workflows/ci.yaml 安装 python-docx python-pptx openpyxl)。
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
core "agent-system-core"
)
func pythonModuleAvailable(t *testing.T, module string) bool {
t.Helper()
if _, err := exec.LookPath("python3"); err != nil {
return false
}
return exec.Command("python3", "-c", "import "+module).Run() == nil
}
// runPythonScript 把脚本写到临时目录执行;失败即 t.Fatal(脚本属测试自身)。
func runPythonScript(t *testing.T, script string) {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "gen.py")
if err := os.WriteFile(path, []byte(script), 0o644); err != nil {
t.Fatal(err)
}
out, err := exec.Command("python3", path).CombinedOutput()
if err != nil {
t.Fatalf("python generator failed: %v\n%s", err, out)
}
}
func TestParseXlsxFromOpenpyxl(t *testing.T) {
if !pythonModuleAvailable(t, "openpyxl") {
t.Skip("python3 openpyxl not available")
}
dir := t.TempDir()
file := filepath.Join(dir, "报价单.xlsx")
runPythonScript(t, `
import openpyxl
from datetime import datetime
wb = openpyxl.Workbook()
ws = wb.active
ws.title = "报价"
ws.append(["物料", "单价", "日期"])
ws.append(["支架", 12.5, datetime(2024, 1, 1)])
ws["C2"].number_format = "yyyy-mm-dd"
ws["D2"] = "=A2&\"-x\""
ws["D3"] = True
wb.create_sheet("说明").append(["备注行"])
wb.save(r"`+file+`")
`)
data, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
doc := Parse(file, "", data)
if doc.Meta.Parser != "builtin:xlsx" || len(doc.Warnings) != 0 {
t.Fatalf("meta: %+v warnings: %v", doc.Meta, doc.Warnings)
}
var heads []string
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads = append(heads, b.Text)
}
}
if len(heads) != 2 || heads[0] != "报价" || heads[1] != "说明" {
t.Fatalf("headings: %v", heads)
}
plain := doc.PlainText()
for _, want := range []string{"物料", "支架", "12.5", "2024-01-01"} {
if !strings.Contains(plain, want) {
t.Errorf("missing %q in: %s", want, plain)
}
}
// openpyxl 公式不带缓存值:回显公式本身(可检索、不冒充结果)。
if !strings.Contains(plain, "=A2") {
t.Errorf("formula without cache should echo: %s", plain)
}
if !strings.Contains(plain, "TRUE") {
t.Errorf("boolean cell should render TRUE: %s", plain)
}
}
func TestParsePptxFromPythonPptx(t *testing.T) {
if !pythonModuleAvailable(t, "pptx") {
t.Skip("python3 pptx not available")
}
dir := t.TempDir()
file := filepath.Join(dir, "方案评审.pptx")
runPythonScript(t, `
from pptx import Presentation
from pptx.util import Inches
prs = Presentation()
s1 = prs.slides.add_slide(prs.slide_layouts[0])
s1.shapes.title.text = "评审结论"
s1.placeholders[1].text = "通过"
s2 = prs.slides.add_slide(prs.slide_layouts[5])
s2.shapes.title.text = "报价明细"
tbl = s2.shapes.add_table(2, 2, Inches(1), Inches(2), Inches(6), Inches(1)).table
tbl.cell(0, 0).text = "物料"
tbl.cell(0, 1).text = "单价"
tbl.cell(1, 0).text = "支架"
tbl.cell(1, 1).text = "12.5"
notes = s1.notes_slide.notes_text_frame
notes.text = "需要补充法规依据"
prs.save(r"`+file+`")
`)
data, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
doc := Parse(file, "", data)
if doc.Meta.Parser != "builtin:pptx" || len(doc.Warnings) != 0 {
t.Fatalf("meta: %+v warnings: %v", doc.Meta, doc.Warnings)
}
var heads []string
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads = append(heads, b.Text)
}
}
if len(heads) != 2 || !strings.HasPrefix(heads[0], "幻灯片 1:评审结论") || !strings.HasPrefix(heads[1], "幻灯片 2:报价明细") {
t.Fatalf("headings: %v", heads)
}
plain := doc.PlainText()
for _, want := range []string{"通过", "物料", "支架", "12.5", "备注:需要补充法规依据"} {
if !strings.Contains(plain, want) {
t.Errorf("missing %q in: %s", want, plain)
}
}
}
+432
View File
@@ -0,0 +1,432 @@
package ingest
// OOXML(xlsx/pptx)解析器测试:手工构造最小 zip 部件,覆盖顺序解析、富文本、
// 公式、日期、稀疏列、隐藏表、组合形状、AlternateContent、表格合并、备注、
// 上限与失败路径。真实生成器的轮测见 ooxml_python_test.go。
import (
"archive/zip"
"bytes"
"sort"
"strings"
"testing"
core "agent-system-core"
)
// zipBytes 按名称构造 zip(键排序保证确定性)。
func zipBytes(t *testing.T, files map[string]string) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
names := make([]string, 0, len(files))
for n := range files {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
w, err := zw.Create(n)
if err != nil {
t.Fatal(err)
}
if _, err := w.Write([]byte(files[n])); err != nil {
t.Fatal(err)
}
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
const relTypeNS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
func xlsxFixture() map[string]string {
return map[string]string{
"xl/workbook.xml": `<workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:r="` + relTypeNS + `">
<sheets>
<sheet name="报价" sheetId="1" r:id="rId1"/>
<sheet name="草稿" sheetId="2" state="hidden" r:id="rId2"/>
</sheets></workbook>`,
"xl/_rels/workbook.xml.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="` + relTypeNS + `/worksheet" Target="worksheets/sheet1.xml"/>
<Relationship Id="rId2" Type="` + relTypeNS + `/worksheet" Target="/xl/worksheets/sheet2.xml"/>
</Relationships>`,
// shared[0]=物料;shared[1]=富文本"单价"(rPh 注音必须排除)。
"xl/sharedStrings.xml": `<sst xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<si><t>物料</t></si>
<si><rPh sb="0" eb="2"><t>たんか</t></rPh><r><rPr lang="ja"/><t>单价</t></r></si>
</sst>`,
"xl/styles.xml": `<styleSheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<numFmts><numFmt numFmtId="164" formatCode="yyyy&quot;年&quot;m&quot;月&quot;d&quot;日&quot;"/></numFmts>
<cellXfs count="3">
<xf numFmtId="0"/>
<xf numFmtId="14" applyNumberFormat="1"/>
<xf numFmtId="164" applyNumberFormat="1"/>
</cellXfs></styleSheet>`,
// A1=物料 B1=单价(富文本) A2=公式缓存 10 B2=无缓存公式 C2=日期(内置14)=2024-01-01
// D2=TRUE E2=#DIV/0! F2=内联 G2=自定义日期格式(2024-01-02) 稀疏行 C4=物料。
"xl/worksheets/sheet1.xml": `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<sheetData>
<row r="1"><c r="A1" t="s"><v>0</v></c><c r="B1" t="s"><v>1</v></c></row>
<row r="2"><c r="A2"><f>B9*2</f><v>10</v></c><c r="B2"><f>B9*3</f></c><c r="C2" s="1"><v>45292</v></c><c r="D2" t="b"><v>1</v></c><c r="E2" t="e"><v>#DIV/0!</v></c><c r="F2" t="inlineStr"><is><t>内联</t></is></c><c r="G2" s="2"><v>45293</v></c></row>
<row r="4"><c r="C4" t="s"><v>0</v></c></row>
</sheetData></worksheet>`,
"xl/worksheets/sheet2.xml": `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<sheetData><row r="1"><c r="A1"><v>7</v></c></row></sheetData></worksheet>`,
}
}
func findTable(doc core.ParsedDocument, after int) *core.DocBlock {
for i := after; i < len(doc.Blocks); i++ {
if doc.Blocks[i].Type == core.BlockTable {
return &doc.Blocks[i]
}
}
return nil
}
func TestParseXlsxBuiltFromParts(t *testing.T) {
doc := Parse("报价单.xlsx", "", zipBytes(t, xlsxFixture()))
if doc.Meta.Parser != "builtin:xlsx" || len(doc.Warnings) != 0 {
t.Fatalf("meta: %+v warnings: %v", doc.Meta, doc.Warnings)
}
if doc.Meta.Confidence != 0.9 {
t.Fatalf("confidence: %v", doc.Meta.Confidence)
}
// 工作表顺序取 workbook.xml(报价在前),隐藏表标注;绝对 target /xl/… 也能定位。
if doc.Blocks[0].Type != core.BlockHeading || doc.Blocks[0].Text != "报价" {
t.Fatalf("first block: %+v", doc.Blocks[0])
}
heads := []string{}
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads = append(heads, b.Text)
}
}
if strings.Join(heads, "|") != "报价|草稿(隐藏)" {
t.Fatalf("headings: %v", heads)
}
tbl := findTable(doc, 0)
if tbl == nil || len(tbl.Rows) != 3 {
t.Fatalf("table: %+v", tbl)
}
row1, row2, row4 := tbl.Rows[0], tbl.Rows[1], tbl.Rows[2]
if row1[0] != "物料" || row1[1] != "单价" {
t.Errorf("header row (phonetic must be excluded): %v", row1)
}
want := []string{"10", "=B9*3", "2024-01-01", "TRUE", "#DIV/0!", "内联", "2024-01-02"}
if len(row2) != len(want) {
t.Fatalf("row2: %v", row2)
}
for i, w := range want {
if row2[i] != w {
t.Errorf("row2[%d]: got %q want %q", i, row2[i], w)
}
}
// 稀疏单元格按引用就位:C4 → 第三列。
if len(row4) != 3 || row4[0] != "" || row4[1] != "" || row4[2] != "物料" {
t.Errorf("sparse row: %v", row4)
}
if !strings.Contains(doc.PlainText(), "2024-01-01") {
t.Error("PlainText missing date text")
}
}
func TestParseXlsxEmptySheetSkipped(t *testing.T) {
files := xlsxFixture()
files["xl/worksheets/sheet2.xml"] = `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData><row r="1"/></sheetData></worksheet>`
doc := Parse("x.xlsx", "", zipBytes(t, files))
heads := 0
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads++
}
}
if heads != 1 || len(doc.Warnings) != 0 {
t.Fatalf("empty sheet must be skipped: blocks=%+v warnings=%v", doc.Blocks, doc.Warnings)
}
}
func TestParseXlsxInvalid(t *testing.T) {
doc := Parse("坏.xlsx", "", []byte("not a zip"))
if len(doc.Warnings) == 0 || len(doc.Blocks) != 0 || doc.Meta.Confidence != 0.3 {
t.Fatalf("invalid xlsx should fail visibly: %+v %v", doc.Meta, doc.Warnings)
}
files := xlsxFixture()
delete(files, "xl/workbook.xml")
doc = Parse("无工作簿.xlsx", "", zipBytes(t, files))
if len(doc.Warnings) == 0 || len(doc.Blocks) != 0 {
t.Fatalf("missing workbook.xml should fail visibly: %v", doc.Warnings)
}
}
func TestParseXlsxCellCap(t *testing.T) {
old := xlsxMaxCells
xlsxMaxCells = 3
defer func() { xlsxMaxCells = old }()
doc := Parse("超限.xlsx", "", zipBytes(t, xlsxFixture()))
if len(doc.Blocks) == 0 {
t.Fatal("truncated parse should keep partial blocks")
}
if len(doc.Warnings) == 0 || doc.Meta.Confidence != 0.6 {
t.Fatalf("truncation must warn and lower confidence: %v %v", doc.Meta.Confidence, doc.Warnings)
}
if !strings.Contains(strings.Join(doc.Warnings, ";"), "上限") {
t.Errorf("warnings: %v", doc.Warnings)
}
}
func TestParseXlsxPartSizeCap(t *testing.T) {
old := ooxmlPartLimit
ooxmlPartLimit = 16
defer func() { ooxmlPartLimit = old }()
doc := Parse("炸弹.xlsx", "", zipBytes(t, xlsxFixture()))
if len(doc.Warnings) == 0 || len(doc.Blocks) != 0 || doc.Meta.Confidence != 0.3 {
t.Fatalf("oversized part must fail visibly: %+v %v", doc.Meta, doc.Warnings)
}
}
func TestParseXlsxDate1904(t *testing.T) {
files := xlsxFixture()
files["xl/workbook.xml"] = strings.Replace(files["xl/workbook.xml"],
"<sheets>", `<workbookPr date1904="1"/><sheets>`, 1)
files["xl/worksheets/sheet1.xml"] = `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
<sheetData><row r="1"><c r="A1" s="1"><v>1</v></c><c r="B1" s="1"><v>0.520833333333333</v></c></row></sheetData></worksheet>`
doc := Parse("mac.xlsx", "", zipBytes(t, files))
tbl := findTable(doc, 0)
// 1904 历法:serial 0 = 1904-01-01,serial 1 = 1904-01-02。
if tbl == nil || tbl.Rows[0][0] != "1904-01-02" {
t.Fatalf("1904 epoch: %+v", tbl)
}
if !strings.Contains(tbl.Rows[0][1], "1904-01-01 12:30") {
t.Errorf("time fraction: %q", tbl.Rows[0][1])
}
}
const pNS = "http://schemas.openxmlformats.org/presentationml/2006/main"
const aNS = "http://schemas.openxmlformats.org/drawingml/2006/main"
func sp(phType, text string) string {
ph := ""
if phType != "" {
ph = `<p:nvPr><p:ph type="` + phType + `"/></p:nvPr>`
} else {
ph = `<p:nvPr/>`
}
return `<p:sp><p:nvSpPr><p:cNvPr id="2" name="S"/><p:cNvSpPr/></p:nvSpPr>` + ph + `<p:spPr/>` +
`<p:txBody><a:bodyPr/><a:p><a:r><a:t>` + text + `</a:t></a:r></a:p></p:txBody></p:sp>`
}
func pptxFixture() map[string]string {
presRel := func(id, target string) string {
return `<Relationship Id="` + id + `" Type="` + relTypeNS + `/slide" Target="` + target + `"/>`
}
slide1 := `<p:sld xmlns:p="` + pNS + `" xmlns:a="` + aNS + `" xmlns:r="` + relTypeNS + `"><p:cSld><p:spTree>
<p:nvGrpSpPr/><p:grpSpPr/>
` + sp("title", "封面标题") + `
` + sp("body", "") + `
<p:sp><p:nvSpPr><p:cNvPr id="4" name="B"/><p:cNvSpPr/></p:nvSpPr><p:nvPr/><p:spPr/><p:txBody><a:bodyPr/>
<a:p><a:r><a:t>第一行</a:t></a:r><a:br/><a:r><a:t>第二行</a:t></a:r></a:p></p:txBody></p:sp>
<p:grpSp><p:nvGrpSpPr/><p:grpSpPr/>` + sp("", "组合内文本") + `</p:grpSp>
<mc:AlternateContent xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006">
<mc:Choice Requires="p14">` + sp("", "选内文本") + `</mc:Choice>
<mc:Fallback>` + sp("", "弃内文本") + `</mc:Fallback>
</mc:AlternateContent>
<p:sp><p:nvSpPr><p:cNvPr id="6" name="N"/><p:cNvSpPr/></p:nvSpPr><p:nvPr><p:ph type="sldNum"/></p:nvPr><p:spPr/><p:txBody><a:p><a:fld id="{F1}"><a:t>3</a:t></a:fld></a:p></p:txBody></p:sp>
<p:graphicFrame><p:nvGraphicFramePr/><p:xfrm/><a:graphic><a:graphicData uri="` + aNS + `/table"><a:tbl>
<a:tr h="1"><a:tc gridSpan="2"><a:txBody><a:p><a:r><a:t>合并头</a:t></a:r></a:p></a:txBody></a:tc><a:tc><a:txBody><a:p><a:r><a:t>列三</a:t></a:r></a:p></a:txBody></a:tc></a:tr>
<a:tr h="1"><a:tc hMerge="1"><a:txBody><a:p/></a:txBody></a:tc><a:tc hMerge="1"><a:txBody><a:p/></a:txBody></a:tc><a:tc><a:txBody><a:p><a:r><a:t>值三</a:t></a:r></a:p></a:txBody></a:tc></a:tr>
</a:tbl></a:graphicData></a:graphic></p:graphicFrame>
</p:spTree></p:cSld></p:sld>`
slide2 := `<p:sld xmlns:p="` + pNS + `" xmlns:a="` + aNS + `"><p:cSld><p:spTree>
<p:nvGrpSpPr/><p:grpSpPr/>` + sp("body", "第二页要点") + `
</p:spTree></p:cSld></p:sld>`
return map[string]string{
"ppt/presentation.xml": `<p:presentation xmlns:p="` + pNS + `" xmlns:r="` + relTypeNS + `"><p:sldIdLst>
<p:sldId id="256" r:id="rIdA"/><p:sldId id="257" r:id="rIdB"/>
</p:sldIdLst></p:presentation>`,
"ppt/_rels/presentation.xml.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">` +
presRel("rIdA", "slides/slide1.xml") + presRel("rIdB", "slides/slide2.xml") + `</Relationships>`,
"ppt/slides/slide1.xml": slide1,
"ppt/slides/slide2.xml": slide2,
"ppt/slides/_rels/slide1.xml.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rIdN" Type="` + relTypeNS + `/notesSlide" Target="../notesSlides/notesSlide1.xml"/></Relationships>`,
// slide2 的备注故意缺失(无 rels 文件)。
"ppt/notesSlides/notesSlide1.xml": `<p:notes xmlns:p="` + pNS + `" xmlns:a="` + aNS + `"><p:cSld><p:spTree>
<p:nvGrpSpPr/><p:grpSpPr/>` + sp("body", "补充说明") + `
</p:spTree></p:cSld></p:notes>`,
}
}
func TestParsePptxBuiltFromParts(t *testing.T) {
doc := Parse("演示.pptx", "", zipBytes(t, pptxFixture()))
if doc.Meta.Parser != "builtin:pptx" || len(doc.Warnings) != 0 || doc.Meta.Confidence != 0.9 {
t.Fatalf("meta: %+v warnings: %v", doc.Meta, doc.Warnings)
}
var heads []string
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads = append(heads, b.Text)
}
}
// sldIdLst 顺序:slide1(有标题)在前;slide2 无标题占位符 → 无标题兜底。
if len(heads) != 2 || heads[0] != "幻灯片 1:封面标题" || heads[1] != "幻灯片 2" {
t.Fatalf("headings: %v", heads)
}
plain := doc.PlainText()
for _, want := range []string{"第一行\n第二行", "组合内文本", "选内文本", "备注:补充说明"} {
if !strings.Contains(plain, want) {
t.Errorf("missing %q in plain text", want)
}
}
if strings.Contains(plain, "弃内文本") {
t.Error("mc:Fallback content must be skipped")
}
// 页码占位符与字段文本不出现。
if strings.Contains(plain, "\n3\n") {
t.Error("slide number placeholder text leaked")
}
// 表格:gridSpan 占位 + hMerge 续格留空,保持三列对齐。
tbl := findTable(doc, 0)
if tbl == nil || len(tbl.Rows) != 2 {
t.Fatalf("table: %+v", tbl)
}
if strings.Join(tbl.Rows[0], "|") != "合并头||列三" {
t.Errorf("row1: %v", tbl.Rows[0])
}
if strings.Join(tbl.Rows[1], "|") != "||值三" {
t.Errorf("row2: %v", tbl.Rows[1])
}
}
func TestParsePptxInvalidAndEmpty(t *testing.T) {
doc := Parse("坏.pptx", "", []byte("garbage"))
if len(doc.Warnings) == 0 || len(doc.Blocks) != 0 || doc.Meta.Confidence != 0.3 {
t.Fatalf("invalid pptx: %+v %v", doc.Meta, doc.Warnings)
}
files := pptxFixture()
files["ppt/presentation.xml"] = `<p:presentation xmlns:p="` + pNS + `"></p:presentation>`
doc = Parse("空演示.pptx", "", zipBytes(t, files))
if len(doc.Warnings) == 0 || len(doc.Blocks) != 0 {
t.Fatalf("no slides: %v", doc.Warnings)
}
}
func TestParseLegacyOfficeHint(t *testing.T) {
for ext, want := range map[string]string{
"doc": "docx", "xls": "xlsx", "ppt": "pptx",
} {
doc := Parse("旧档."+ext, "", []byte{0xD0, 0xCF})
if doc.Meta.Parser != "builtin:legacy-office" {
t.Fatalf("%s parser: %+v", ext, doc.Meta)
}
if len(doc.Blocks) != 0 || len(doc.Warnings) != 1 {
t.Fatalf("%s blocks/warnings: %+v %v", ext, doc.Blocks, doc.Warnings)
}
if !strings.Contains(doc.Warnings[0], want) {
t.Errorf("%s hint should mention %s: %v", ext, want, doc.Warnings[0])
}
}
}
// 注册表与分派行为一致:每个已注册格式的解析器 ID 与 Parse 分派结果一致,
// 且 builtin 解析器覆盖的扩展名都出现在注册表里。
func TestParsersDescriptionMatchesDispatch(t *testing.T) {
byFormat := map[string]string{}
for _, d := range ParsersDescription() {
if d.Source != core.SourceBuiltin {
continue
}
for _, f := range d.Formats {
byFormat[f] = d.ID
}
}
check := map[string]string{
"md": "builtin:md", "markdown": "builtin:md",
"txt": "builtin:txt", "text": "builtin:txt", "csv": "builtin:txt", "log": "builtin:txt",
"docx": "builtin:docx", "pdf": "builtin:pdf",
"xlsx": "builtin:xlsx", "xlsm": "builtin:xlsx",
"pptx": "builtin:pptx", "pptm": "builtin:pptx", "ppsx": "builtin:pptx",
"doc": "builtin:legacy-office", "xls": "builtin:legacy-office", "ppt": "builtin:legacy-office",
}
for ext, wantID := range check {
got, ok := byFormat[ext]
if !ok {
t.Errorf("format .%s missing from registry", ext)
continue
}
doc := Parse("a."+ext, "", []byte("x"))
if doc.Meta.Parser != wantID || got != wantID {
t.Errorf(".%s: registry=%s dispatch=%s want=%s", ext, got, doc.Meta.Parser, wantID)
}
}
}
// TestParseXlsxWideColumnSkipsCellNotSheets:稀疏样式单元格落在远端列(XFD 区,
// 列序 ≥1024)只跳过该格并告警——不误报单元格上限、不丢弃后续工作表。
func TestParseXlsxWideColumnSkipsCellNotSheets(t *testing.T) {
files := xlsxFixture()
// 第一表加一个列引用 XFD1(16384 列)的样式单元格;第二表(隐藏的"草稿")必须照常解析。
files["xl/worksheets/sheet1.xml"] = strings.Replace(files["xl/worksheets/sheet1.xml"],
"</row>\n<row r=\"4\"", "</row>\n<row r=\"3\"><c r=\"XFD3\" t=\"inlineStr\"><is><t>边角样式</t></is></c></row>\n<row r=\"4\"", 1)
doc := Parse("宽表.xlsx", "", zipBytes(t, files))
heads := []string{}
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading {
heads = append(heads, b.Text)
}
}
if strings.Join(heads, "|") != "报价|草稿(隐藏)" {
t.Fatalf("later sheets must survive a wide column: headings=%v warnings=%v", heads, doc.Warnings)
}
joined := strings.Join(doc.Warnings, ";")
if !strings.Contains(joined, "超过 1024 列") {
t.Fatalf("wide-column warning missing: %v", doc.Warnings)
}
if strings.Contains(joined, "单元格总数") {
t.Fatalf("cell-cap warning must not fire for a wide column: %v", doc.Warnings)
}
if doc.Meta.Confidence != 0.6 {
t.Fatalf("partial-content confidence expected 0.6: %v", doc.Meta.Confidence)
}
// 被跳过的远端列内容不得混进表格。
for _, b := range doc.Blocks {
if b.Type == core.BlockTable {
for _, row := range b.Rows {
for _, cell := range row {
if strings.Contains(cell, "边角样式") {
t.Fatalf("skipped wide-column cell leaked into table: %v", row)
}
}
}
}
}
}
// TestParseXlsxBadSheetXMLWarns:单张工作表 XML 损坏必须显式告警(其余表照常),
// 与提案「显式失败,不静默」一致。
func TestParseXlsxBadSheetXMLWarns(t *testing.T) {
files := xlsxFixture()
files["xl/worksheets/sheet1.xml"] = `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData><row`
doc := Parse("坏表.xlsx", "", zipBytes(t, files))
if len(doc.Warnings) == 0 {
t.Fatal("malformed sheet XML must warn")
}
joined := strings.Join(doc.Warnings, ";")
if !strings.Contains(joined, "工作表 报价") || !strings.Contains(joined, "XML 解码失败") {
t.Fatalf("warning should name the sheet and the failure: %v", doc.Warnings)
}
// 第二张表照常解析。
found := false
for _, b := range doc.Blocks {
if b.Type == core.BlockHeading && b.Text == "草稿(隐藏)" {
found = true
}
}
if !found {
t.Fatalf("healthy sheets must still parse: %+v", doc.Blocks)
}
if doc.Meta.Confidence != 0.6 {
t.Fatalf("partial parse confidence expected 0.6: %v", doc.Meta.Confidence)
}
}
+291
View File
@@ -0,0 +1,291 @@
package ingest
// pptx/pptm/ppsx(PresentationML)内置解析器。幻灯片顺序取 presentation.xml
// 的 sldIdLst;标题占位符(title/ctrTitle)产标题块,正文形状按段落输出
//(a:br 换行),组合形状经同一 token 走查天然递归;mc:AlternateContent 只取
// Choice 分支;graphicFrame 内的表格按 tr/tc 输出(gridSpan/hMerge 占位保列
// 对齐);备注页正文输出「备注:…」;页码/日期/页脚占位符跳过。图片与
// SmartArt 不解析。纯标准库。
import (
"archive/zip"
"bytes"
"encoding/xml"
"fmt"
"io"
"path"
"strconv"
"strings"
core "agent-system-core"
)
func parsePptx(content []byte) ([]core.DocBlock, []string) {
zr, err := openZip(content)
if err != nil {
return nil, []string{"pptx 解析失败:" + err.Error()}
}
slides, warns := pptxSlideOrder(zr)
if slides == nil && len(warns) > 0 {
return nil, warns
}
var blocks []core.DocBlock
for i, sp := range slides {
n := i + 1
data, err := zipPart(zr, sp)
if err != nil {
warns = append(warns, fmt.Sprintf("幻灯片 %d 读取失败:%v", n, err))
blocks = append(blocks, core.DocBlock{Type: core.BlockHeading, Level: 2, Text: fmt.Sprintf("幻灯片 %d", n)})
continue
}
title, paras, tables := pptxSlideBlocks(data)
heading := fmt.Sprintf("幻灯片 %d", n)
if title != "" {
heading += ":" + title
}
blocks = append(blocks, core.DocBlock{Type: core.BlockHeading, Level: 2, Text: heading})
blocks = append(blocks, paras...)
blocks = append(blocks, tables...)
if note := pptxSlideNotes(zr, sp); note != "" {
blocks = append(blocks, core.DocBlock{Type: core.BlockPara, Text: "备注:" + note})
}
}
if len(blocks) == 0 && len(warns) == 0 {
warns = append(warns, "pptx 无可解析的幻灯片内容")
}
return blocks, warns
}
// pptxSlideOrder 读 presentation.xml 的 sldIdLst → 幻灯片部件名(按顺序)。
func pptxSlideOrder(zr *zip.Reader) ([]string, []string) {
data, err := zipPart(zr, "ppt/presentation.xml")
if err != nil {
return nil, []string{"pptx 解析失败:" + err.Error()}
}
rels, err := parseRels(zr, "ppt/presentation.xml")
if err != nil {
return nil, []string{"pptx 解析失败:" + err.Error()}
}
dec := xml.NewDecoder(bytes.NewReader(data))
var out, warns []string
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return out, append(warns, "presentation.xml 解码失败:"+err.Error())
}
st, ok := tok.(xml.StartElement)
if !ok || st.Name.Local != "sldId" {
continue
}
rel, ok := rels[ridAttr(st)]
if !ok || !relIsType(rel, "/slide") {
warns = append(warns, "幻灯片关系缺失或类型不符")
continue
}
out = append(out, resolveRelTarget("ppt", rel.Target))
}
return out, warns
}
// pptxSlideBlocks 解析一张幻灯片:标题、正文段落、表格块。
// 表格块统一附在段落后(图形框与正文的精确穿插对检索无价值)。
func pptxSlideBlocks(data []byte) (title string, paras []core.DocBlock, tables []core.DocBlock) {
dec := xml.NewDecoder(bytes.NewReader(data))
var stack []string
has := func(name string) bool {
for _, s := range stack {
if s == name {
return true
}
}
return false
}
skip := 0 // mc:Fallback 子树深度
// 形状状态
spDepth := -1
phSkip := false
isTitle := false
var spParas []string
var para strings.Builder
paraOpen := false
// 文本收集目标
textInPara := false
textInCell := false
// 表格状态
var tblRows [][]string
var tblRow []string
var cellBuf strings.Builder
cellOpen := false
gridSpan := 1
cellHM := false
finishPara := func() {
if !paraOpen {
return
}
paraOpen = false
if s := strings.TrimRight(para.String(), "\n"); strings.TrimSpace(s) != "" {
spParas = append(spParas, strings.TrimSpace(s))
}
para.Reset()
}
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return "", paras, tables
}
switch t := tok.(type) {
case xml.StartElement:
if skip > 0 {
skip++
continue
}
local := t.Name.Local
switch local {
case "Fallback":
if has("AlternateContent") {
skip = 1
continue
}
case "sp":
spDepth = len(stack)
phSkip, isTitle, spParas = false, false, nil
case "ph":
if spDepth >= 0 && len(stack) > spDepth {
switch attrLocal(t, "type") {
case "title", "ctrTitle":
isTitle = true
case "dt", "ftr", "sldNum":
phSkip = true
}
}
case "p":
if spDepth >= 0 && has("txBody") && !has("tbl") {
finishPara()
paraOpen = true
para.Reset()
} else if cellOpen {
if cellBuf.Len() > 0 {
cellBuf.WriteByte('\n')
}
}
case "br":
if paraOpen {
para.WriteByte('\n')
}
case "t":
if has("fld") {
break // 字段(页码/日期)不取
}
textInPara = paraOpen
textInCell = cellOpen && !paraOpen
case "tc":
if has("tbl") {
cellBuf.Reset()
cellOpen = true
gridSpan, _ = strconv.Atoi(attrLocal(t, "gridSpan"))
if gridSpan < 1 {
gridSpan = 1
}
hm := attrLocal(t, "hMerge")
cellHM = hm == "1" || strings.EqualFold(hm, "true")
}
}
stack = append(stack, local)
case xml.CharData:
if textInPara {
para.Write(t)
} else if textInCell {
cellBuf.Write(t)
}
case xml.EndElement:
if skip > 0 {
skip--
continue
}
if len(stack) > 0 {
stack = stack[:len(stack)-1]
}
switch t.Name.Local {
case "t":
textInPara, textInCell = false, false
case "p":
finishPara()
case "sp":
if spDepth >= 0 && len(stack) == spDepth {
if !phSkip && len(spParas) > 0 {
if isTitle && title == "" {
title = strings.Join(spParas, " ")
} else {
for _, p := range spParas {
paras = append(paras, core.DocBlock{Type: core.BlockPara, Text: p})
}
}
}
spDepth = -1
phSkip, isTitle, spParas = false, false, nil
}
case "tc":
if cellOpen {
text := strings.TrimSpace(cellBuf.String())
if cellHM {
text = "" // 横向合并的续格:占位保列对齐
}
tblRow = append(tblRow, text)
for i := 1; i < gridSpan; i++ {
tblRow = append(tblRow, "") // 合并占位保列对齐
}
cellOpen = false
}
case "tr":
if len(tblRow) > 0 {
for len(tblRow) > 0 && tblRow[len(tblRow)-1] == "" {
tblRow = tblRow[:len(tblRow)-1]
}
if len(tblRow) > 0 {
tblRows = append(tblRows, tblRow)
}
tblRow = nil
}
case "tbl":
if len(tblRows) > 0 {
tables = append(tables, core.DocBlock{Type: core.BlockTable, Rows: tblRows})
}
tblRows = nil
}
}
}
finishPara()
return title, paras, tables
}
// pptxSlideNotes 读幻灯片的备注页正文;无备注返回空串。
func pptxSlideNotes(zr *zip.Reader, slidePart string) string {
rels, err := parseRels(zr, slidePart)
if err != nil {
return ""
}
for _, rel := range rels {
if !relIsType(rel, "/notesSlide") {
continue
}
data, err := zipPart(zr, resolveRelTarget(path.Dir(slidePart), rel.Target))
if err != nil {
return ""
}
_, paras, _ := pptxSlideBlocks(data)
var lines []string
for _, p := range paras {
lines = append(lines, p.Text)
}
return strings.Join(lines, "\n")
}
return ""
}
+425
View File
@@ -0,0 +1,425 @@
package ingest
// xlsx/xlsm(SpreadsheetML)内置解析器。工作表顺序取 workbook.xml 的 sheets
// 列表(target 经关系部件解析,支持相对/绝对路径);共享字符串含富文本 run
//(注音 rPh 无对应解码字段,天然排除);单元格覆盖 inlineStr/str/b/e/n,
// 公式取缓存值、无缓存回显公式;日期按数字格式(内置 ID 与自定义记号)转
// ISO,1900/1904 双日期系统;稀疏单元格按引用就位。纯标准库。
import (
"archive/zip"
"bytes"
"encoding/xml"
"fmt"
"io"
"math"
"strconv"
"strings"
"time"
core "agent-system-core"
)
// 解析上限(包级变量,测试可覆盖)。
var (
xlsxMaxCells = 200000
xlsxMaxCols = 1024
)
type xlsxSheetRef struct {
Name string
State string
Path string
IsSheet bool
}
func parseXlsx(content []byte) ([]core.DocBlock, []string) {
zr, err := openZip(content)
if err != nil {
return nil, []string{"xlsx 解析失败:" + err.Error()}
}
sheets, date1904, warns := xlsxWorkbookSheets(zr)
if len(warns) > 0 && sheets == nil {
return nil, warns
}
shared := xlsxSharedStrings(zr)
dateStyles := xlsxDateStyles(zr)
var blocks []core.DocBlock
cells := 0
for _, sh := range sheets {
if !sh.IsSheet || sh.Path == "" {
continue // chartsheet 等非工作表部件
}
data, err := zipPart(zr, sh.Path)
if err != nil {
warns = append(warns, fmt.Sprintf("工作表 %s 读取失败:%v", sh.Name, err))
continue
}
table, used, trunc, wide, sheetWarn := xlsxSheetBlocks(data, shared, dateStyles, date1904, cells)
cells = used
if sheetWarn != "" {
warns = append(warns, fmt.Sprintf("工作表 %s:%s", sh.Name, sheetWarn))
}
if wide {
warns = append(warns, fmt.Sprintf("工作表 %s 含超过 %d 列的稀疏单元格,已跳过该列(其余内容不受影响)", sh.Name, xlsxMaxCols))
}
if table == nil {
continue // 空表(读取失败/坏 XML 已单独告警)
}
title := sh.Name
if sh.State == "hidden" || sh.State == "veryHidden" {
title += "(隐藏)"
}
blocks = append(blocks,
core.DocBlock{Type: core.BlockHeading, Level: 2, Text: title},
*table,
)
if trunc {
warns = append(warns, fmt.Sprintf("单元格总数超过 %d 上限,工作表 %s 之后未解析", xlsxMaxCells, sh.Name))
break
}
}
if len(blocks) == 0 && len(warns) == 0 {
warns = append(warns, "xlsx 无可解析的工作表内容")
}
return blocks, warns
}
// xlsxWorkbookSheets 读 workbook.xml:工作表顺序、隐藏状态、1904 日期系统。
func xlsxWorkbookSheets(zr *zip.Reader) (sheets []xlsxSheetRef, date1904 bool, warns []string) {
data, err := zipPart(zr, "xl/workbook.xml")
if err != nil {
return nil, false, []string{"xlsx 解析失败:" + err.Error()}
}
rels, err := parseRels(zr, "xl/workbook.xml")
if err != nil {
return nil, false, []string{"xlsx 解析失败:" + err.Error()}
}
dec := xml.NewDecoder(bytes.NewReader(data))
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return sheets, date1904, append(warns, "workbook.xml 解码失败:"+err.Error())
}
st, ok := tok.(xml.StartElement)
if !ok {
continue
}
switch st.Name.Local {
case "workbookPr":
if v := attrLocal(st, "date1904"); v == "1" || strings.EqualFold(v, "true") {
date1904 = true
}
case "sheet":
ref := xlsxSheetRef{Name: attrLocal(st, "name"), State: attrLocal(st, "state")}
if rel, ok := rels[ridAttr(st)]; ok {
ref.Path = resolveRelTarget("xl", rel.Target)
ref.IsSheet = relIsType(rel, "/worksheet")
} else {
warns = append(warns, fmt.Sprintf("工作表 %s 的关系缺失", ref.Name))
}
sheets = append(sheets, ref)
}
}
return sheets, date1904, warns
}
// xlsxSharedStrings 读共享字符串表;富文本按 run 拼接,rPh 注音不进结果。
func xlsxSharedStrings(zr *zip.Reader) []string {
data, err := zipPart(zr, "xl/sharedStrings.xml")
if err != nil {
return nil
}
dec := xml.NewDecoder(bytes.NewReader(data))
var out []string
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return out
}
st, ok := tok.(xml.StartElement)
if !ok || st.Name.Local != "si" {
continue
}
var si struct {
T string `xml:"t"`
Runs []struct {
T string `xml:"t"`
} `xml:"r"`
}
if err := dec.DecodeElement(&si, &st); err != nil {
continue
}
if si.T != "" || len(si.Runs) == 0 {
out = append(out, si.T)
continue
}
parts := make([]string, 0, len(si.Runs))
for _, r := range si.Runs {
parts = append(parts, r.T)
}
out = append(out, strings.Join(parts, ""))
}
return out
}
// xlsxDateStyles 返回按日期呈现的样式索引集合(cellXfs 下标)。
func xlsxDateStyles(zr *zip.Reader) map[int]bool {
data, err := zipPart(zr, "xl/styles.xml")
if err != nil {
return nil
}
custom := map[int]string{}
date := map[int]bool{}
dec := xml.NewDecoder(bytes.NewReader(data))
inCellXfs := false
xfIdx := 0
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return date
}
switch t := tok.(type) {
case xml.StartElement:
switch t.Name.Local {
case "numFmt":
id, _ := strconv.Atoi(attrLocal(t, "numFmtId"))
custom[id] = attrLocal(t, "formatCode")
case "cellXfs":
inCellXfs = true
xfIdx = 0
case "xf":
if !inCellXfs {
continue
}
id, _ := strconv.Atoi(attrLocal(t, "numFmtId"))
if xlsxFmtIsDate(id, custom) {
date[xfIdx] = true
}
xfIdx++
}
case xml.EndElement:
if t.Name.Local == "cellXfs" {
inCellXfs = false
}
}
}
return date
}
// xlsxFmtIsDate:自定义格式按记号判断;内置 ID 覆盖常见日期/时间区间。
func xlsxFmtIsDate(id int, custom map[int]string) bool {
if code, ok := custom[id]; ok {
return formatCodeIsDate(code)
}
switch {
case id >= 14 && id <= 22, id >= 27 && id <= 36,
id >= 45 && id <= 47, id >= 50 && id <= 58:
return true
}
return false
}
// formatCodeIsDate 剥掉引号字面量与方括号节([Red]、[DBNum1]、[$-409] 等)后,
// 剩余字母含 y/d/h/s/m 记号即按日期处理。
func formatCodeIsDate(code string) bool {
var b strings.Builder
inQuote, inBracket := false, false
runes := []rune(code)
for i := 0; i < len(runes); i++ {
r := runes[i]
switch {
case r == '"':
inQuote = !inQuote
case inQuote:
case r == '[':
inBracket = true
case r == ']':
inBracket = false
case inBracket:
case r == '\\':
i++ // 转义字符:连同下一字符跳过
default:
b.WriteRune(r)
}
}
return strings.ContainsAny(strings.ToLower(b.String()), "ydhsm")
}
type xlsxCellEl struct {
Ref string `xml:"r,attr"`
T string `xml:"t,attr"`
S string `xml:"s,attr"`
IS *struct {
T string `xml:"t"`
} `xml:"is"`
F string `xml:"f"`
V string `xml:"v"`
}
type xlsxRowEl struct {
Cells []xlsxCellEl `xml:"c"`
}
// xlsxSheetBlocks 解析单张工作表;返回表格块、累计单元格数、是否触达单元格
// 上限、是否跳过超宽列、以及坏 XML 的告警(显式失败,不静默)。无内容行
// 返回 nil 表。
func xlsxSheetBlocks(data []byte, shared []string, dateStyles map[int]bool, date1904 bool, cellsUsed int) (*core.DocBlock, int, bool, bool, string) {
dec := xml.NewDecoder(bytes.NewReader(data))
var rows [][]string
count := cellsUsed
truncated := false
wide := false
for {
tok, err := dec.Token()
if err == io.EOF {
break
}
if err != nil {
return nil, count, truncated, wide, "XML 解码失败:" + err.Error()
}
st, ok := tok.(xml.StartElement)
if !ok || st.Name.Local != "row" {
continue
}
var re xlsxRowEl
if err := dec.DecodeElement(&re, &st); err != nil {
continue
}
cells := map[int]string{}
maxCol := -1
for _, c := range re.Cells {
if count >= xlsxMaxCells {
truncated = true
break
}
count++
col := xlsxColIndex(c.Ref)
if col < 0 {
col = maxCol + 1 // 无引用:按出现顺序接续
}
if col >= xlsxMaxCols {
// 稀疏样式单元格常落在远端列(XFD 区):跳过该格并告警,
// 不中断本表其余内容,也不阻断后续工作表。
wide = true
continue
}
cells[col] = xlsxCellText(c, shared, dateStyles, date1904)
if col > maxCol {
maxCol = col
}
}
if len(cells) == 0 {
continue
}
full := make([]string, maxCol+1)
for i, v := range cells {
full[i] = v
}
for len(full) > 0 && strings.TrimSpace(full[len(full)-1]) == "" {
full = full[:len(full)-1]
}
if len(full) > 0 {
rows = append(rows, full)
}
}
if len(rows) == 0 {
return nil, count, truncated, wide, ""
}
return &core.DocBlock{Type: core.BlockTable, Rows: rows}, count, truncated, wide, ""
}
// xlsxCellText 把单元格转成展示文本。
func xlsxCellText(c xlsxCellEl, shared []string, dateStyles map[int]bool, date1904 bool) string {
text := ""
switch c.T {
case "s":
if idx, err := strconv.Atoi(c.V); err == nil && idx >= 0 && idx < len(shared) {
text = shared[idx]
}
case "inlineStr":
if c.IS != nil {
text = c.IS.T
}
case "b":
if c.V == "1" || strings.EqualFold(c.V, "true") {
text = "TRUE"
} else {
text = "FALSE"
}
case "e":
text = c.V
default: // n 或空:数字 / 公式缓存值
text = c.V
if sIdx, err := strconv.Atoi(c.S); err == nil && dateStyles[sIdx] && c.V != "" {
if f, err := strconv.ParseFloat(c.V, 64); err == nil {
text = xlsxSerialString(f, date1904)
}
}
}
if text == "" && c.F != "" {
return "=" + c.F // 无缓存值的公式:回显公式本身
}
return text
}
// xlsxSerialString Excel 日期序列值 → ISO 字符串;带小数部分时输出到秒。
func xlsxSerialString(serial float64, date1904 bool) string {
days := math.Floor(serial)
frac := serial - days
var t time.Time
switch {
case date1904:
t = time.Date(1904, 1, 1, 0, 0, 0, 0, time.UTC).AddDate(0, 0, int(days))
case days >= 61:
t = time.Date(1899, 12, 30, 0, 0, 0, 0, time.UTC).AddDate(0, 0, int(days))
default: // 1–60:兼容 Excel 的 1900 闰年 bug 区段
t = time.Date(1899, 12, 31, 0, 0, 0, 0, time.UTC).AddDate(0, 0, int(days))
}
if frac > 0 {
return t.Add(time.Duration(frac * 86400 * float64(time.Second))).Round(time.Second).Format("2006-01-02 15:04:05")
}
return t.Format("2006-01-02")
}
// xlsxColIndex "BC12" → 0 起列号;无引用或非法返回 -1。
func xlsxColIndex(ref string) int {
if ref == "" {
return -1
}
i := 0
for i < len(ref) {
ch := ref[i]
if (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') {
i++
continue
}
break
}
if i == 0 || i == len(ref) {
return -1
}
for _, ch := range ref[i:] {
if ch < '0' || ch > '9' {
return -1
}
}
n := 0
for _, ch := range ref[:i] {
c := ch
if c >= 'a' && c <= 'z' {
c = c - 'a' + 'A'
}
n = n*26 + int(c-'A') + 1
}
return n - 1
}
+56 -1
View File
@@ -24,20 +24,74 @@ type MatchResult struct {
ID string
Slots map[string]string
Missing []SlotQuestion
// Commands 是库层展开的命令列表(I1);Router 消费它做唯一裁决。
// guard_hit/expectation_hit 被折叠为 fallback 时为空。
Commands []Command
}
// 命令类型经闸门投影(别名):生态库按边界规则只许本包导入,
// Router/runmgr 等消费方只依赖本包表面。
type (
Command = agentintent.Command
ActionRef = agentintent.ActionRef
)
const (
CommandStart = agentintent.CommandStart
ActionAuto = agentintent.ActionAuto
ActionCommand = agentintent.ActionCommand
ActionWorkflow = agentintent.ActionWorkflow
ActionBlueprint = agentintent.ActionBlueprint
ActionHandler = agentintent.ActionHandler
)
type Gate struct {
mu sync.RWMutex
reg *agentintent.Registry
rec *agentintent.FunnelRecognizer
specs int
specsByID map[string]agentintent.IntentSpec
// 慢通道(I2):nil = 不启用(默认,行为与 I1 一致)。
sem agentintent.SemanticScorer
semThresh float64
gen agentintent.CommandGenerator
}
func New() *Gate {
return &Gate{rec: agentintent.NewRecognizer(agentintent.Options{}, nil, nil)}
}
// SetSemantic 接线语义打分器(慢通道中位)。识别器随之重建;
// 传 nil 关闭语义层。阈值 ≤0 用库默认(0.8)。
func (g *Gate) SetSemantic(s agentintent.SemanticScorer, threshold float64) {
g.mu.Lock()
defer g.mu.Unlock()
g.sem, g.semThresh = s, threshold
g.rebuildRecognizerLocked()
}
// SetGenerator 接线模型命令生成器(慢通道末位)。识别器随之重建;
// 传 nil 关闭生成层。
func (g *Gate) SetGenerator(gen agentintent.CommandGenerator) {
g.mu.Lock()
defer g.mu.Unlock()
g.gen = gen
g.rebuildRecognizerLocked()
}
// rebuildRecognizerLocked 按当前慢通道配置重建识别器(调用方持写锁)。
func (g *Gate) rebuildRecognizerLocked() {
if g.sem == nil && g.gen == nil {
g.rec = agentintent.NewRecognizer(agentintent.Options{}, nil, nil)
return
}
g.rec = agentintent.NewRecognizer(agentintent.Options{
Semantic: g.sem,
SemanticThreshold: g.semThresh,
Generator: g.gen,
}, nil, nil)
}
// Rebuild 用插件意图 overlay 重建注册表;失败保留上一快照(fail-closed)并返回错误。
func (g *Gate) Rebuild(ctx context.Context, overlay []json.RawMessage) error {
if len(overlay) == 0 {
@@ -54,7 +108,7 @@ func (g *Gate) Rebuild(ctx context.Context, overlay []json.RawMessage) error {
}
specs = append(specs, s)
}
b := agentintent.NewRegistryBuilder(agentintent.Options{TopN: 3})
b := agentintent.NewRegistryBuilder(agentintent.Options{TopN: 3, NFKC: true})
if err := b.Register(specs...); err != nil {
return fmt.Errorf("intentgate: register: %w", err)
}
@@ -95,6 +149,7 @@ func (g *Gate) Recognize(ctx context.Context, input string) MatchResult {
switch res.Branch {
case agentintent.BranchMatched:
out.Branch = "matched"
out.Commands = res.Commands
case agentintent.BranchAmbiguous:
out.Branch = "ambiguous"
}
+85
View File
@@ -4,6 +4,8 @@ import (
"context"
"encoding/json"
"testing"
agentintent "agent-intent/intent"
)
func raw(t *testing.T, v any) json.RawMessage {
@@ -91,3 +93,86 @@ func TestRecognizeKeepsSlots(t *testing.T) {
t.Fatalf("Match wrapper = %s %s", branch, id)
}
}
// ---- I2 慢通道接线(默认关;SetSemantic/SetGenerator 开启)----
// stubScorer 固定返回脚本分(按 examples[0] 查表)。
type stubScorer struct{ scores map[string]float64 }
func (s stubScorer) Score(_ context.Context, _ string, examples []string) (float64, error) {
if len(examples) == 0 {
return 0, nil
}
return s.scores[examples[0]], nil
}
// stubGenerator 固定返回脚本命令。
type stubGenerator struct{ cmds []Command }
func (g stubGenerator) Generate(_ context.Context, _ string, _ []agentintent.CatalogSpec) ([]agentintent.Command, error) {
return g.cmds, nil
}
func ambiguousOverlay(t *testing.T) []json.RawMessage {
t.Helper()
mk := func(id string, ex []string) json.RawMessage {
return raw(t, map[string]any{
"id": id, "domain": "demo", "version": 1, "priority": 5,
"matcher": map[string]any{"includes_any": []string{"模糊词"}},
"examples": ex,
})
}
return []json.RawMessage{mk("demo.alpha", []string{"查运行进度"}), mk("demo.beta", []string{"闲聊问候"})}
}
// 语义层:歧义被过阈打分裁决为 matched(默认关时不启用)。
func TestGateSemanticResolvesAmbiguous(t *testing.T) {
g := New()
if err := g.Rebuild(context.Background(), ambiguousOverlay(t)); err != nil {
t.Fatal(err)
}
// 默认(未接线):规则层歧义原样透出。
if branch, _ := g.Match(context.Background(), "模糊词"); branch != "ambiguous" {
t.Fatalf("默认关:branch = %s", branch)
}
g.SetSemantic(stubScorer{scores: map[string]float64{"查运行进度": 0.95, "闲聊问候": 0.4}}, 0)
branch, top := g.Match(context.Background(), "模糊词")
if branch != "matched" || top != "demo.alpha" {
t.Fatalf("语义裁决:%s/%s", branch, top)
}
// 关闭后回到歧义。
g.SetSemantic(nil, 0)
if branch, _ := g.Match(context.Background(), "模糊词"); branch != "ambiguous" {
t.Fatalf("关闭语义层:branch = %s", branch)
}
}
// 生成层:fallback 时产出合法 start 命令 → matched + Commands。
func TestGateGeneratorProducesCommands(t *testing.T) {
g := New()
overlay := []json.RawMessage{raw(t, map[string]any{
"id": "report.generate", "domain": "report", "version": 1,
"description": "生成报告",
"matcher": map[string]any{"includes_any": []string{"生成报告"}},
"handler": "reportHandler",
})}
if err := g.Rebuild(context.Background(), overlay); err != nil {
t.Fatal(err)
}
g.SetGenerator(stubGenerator{cmds: []Command{{
Kind: CommandStart,
Action: ActionRef{Kind: ActionHandler, ID: "reportHandler"},
}}})
res := g.Recognize(context.Background(), "完全不相关的一句话")
if res.Branch != "matched" || res.ID != "report.generate" {
t.Fatalf("生成器命中:%+v", res)
}
if len(res.Commands) != 1 || res.Commands[0].Kind != CommandStart {
t.Fatalf("commands: %+v", res.Commands)
}
// 关闭后回到 fallback。
g.SetGenerator(nil)
if branch, _ := g.Match(context.Background(), "完全不相关的一句话"); branch != "fallback" {
t.Fatalf("关闭生成层:branch = %s", branch)
}
}
+129
View File
@@ -0,0 +1,129 @@
package intentsem
import (
"context"
"encoding/json"
"fmt"
"strings"
agentintent "agent-intent/intent"
agentllm "agent-native/runtime/llm"
)
// LightGenerator 用轻量模型实现 agentintent.CommandGenerator
// (Rasa 式 LLM command generator:只把相关流程交给生成器)。
// 提示词给出能力目录(spec id/描述/槽位键),要求模型输出严格的
// 命令 JSON 数组;解析失败或模型不可达都返回错误——库侧约定生成器
// 出错时识别保持 fallback(fail-closed)。
type LightGenerator struct {
// Stream 提供方边界(注入点;测试可脚本化)。
Stream agentllm.StreamFunc
// Model 是轻量档模型名。
Model string
// APIBase/APIKey 走 StreamOptions(客户端可按次覆盖)。
APIBase string
APIKey string
// MaxTokens 上限;0 = 1024。
MaxTokens int
}
// 生成输出契约:action 与 agentintent.ActionRef 同形(kind/id)。
type generatedCommand struct {
Kind string `json:"kind"`
Action generatedAction `json:"action,omitempty"`
Slots map[string]string `json:"slots,omitempty"`
}
type generatedAction struct {
Kind string `json:"kind"`
ID string `json:"id"`
}
// Generate 基于输入与能力目录产出命令列表。目录为空时不调用模型。
func (g LightGenerator) Generate(ctx context.Context, input string, catalog []agentintent.CatalogSpec) ([]agentintent.Command, error) {
if g.Stream == nil || len(catalog) == 0 || strings.TrimSpace(input) == "" {
return nil, nil
}
maxTok := g.MaxTokens
if maxTok <= 0 {
maxTok = 1024
}
msgs := []agentllm.Message{
{Role: agentllm.RoleSystem, Content: generatorSystemPrompt(catalog)},
{Role: agentllm.RoleUser, Content: input},
}
ch := g.Stream(ctx, g.Model, msgs, nil, agentllm.StreamOptions{
MaxTokens: maxTok, APIBase: g.APIBase, APIKey: g.APIKey,
})
var sb strings.Builder
for ev := range ch {
switch ev.Kind {
case agentllm.KindTextDelta:
sb.WriteString(ev.Text)
case agentllm.KindError:
if ev.Err != nil {
return nil, fmt.Errorf("intentsem: stream: %w", ev.Err)
}
return nil, fmt.Errorf("intentsem: stream: %s", ev.ErrorMessage)
}
}
return parseCommands(sb.String())
}
// parseCommands 从模型输出解析命令数组:剥代码围栏,取首个 [ 到末个 ],
// 严格 JSON。任何形状不符都报错(由库侧 fail-closed 到 fallback)。
func parseCommands(raw string) ([]agentintent.Command, error) {
text := strings.TrimSpace(raw)
text = strings.TrimPrefix(text, "```json")
text = strings.TrimPrefix(text, "```")
text = strings.TrimSuffix(text, "```")
text = strings.TrimSpace(text)
lo, hi := strings.Index(text, "["), strings.LastIndex(text, "]")
if lo < 0 || hi <= lo {
return nil, fmt.Errorf("intentsem: 输出不含命令数组")
}
var gen []generatedCommand
if err := json.Unmarshal([]byte(text[lo:hi+1]), &gen); err != nil {
return nil, fmt.Errorf("intentsem: 命令 JSON: %w", err)
}
out := make([]agentintent.Command, 0, len(gen))
for _, c := range gen {
out = append(out, agentintent.Command{
Kind: agentintent.CommandKind(c.Kind),
Action: agentintent.ActionRef{Kind: c.Action.Kind, ID: c.Action.ID},
Slots: c.Slots,
})
}
return out, nil
}
// generatorSystemPrompt 渲染能力目录。只暴露 id/描述/槽位键——
// 不暴露模式与示例(避免提示词注入借道示例文本进入慢通道)。
func generatorSystemPrompt(catalog []agentintent.CatalogSpec) string {
var sb strings.Builder
sb.WriteString("你是意图命令生成器。根据用户输入,从下面的能力目录选择并输出命令。\n")
sb.WriteString("只输出一个 JSON 数组,不要输出任何其他文字。数组元素形如:\n")
sb.WriteString(`[{"kind":"start","action":{"kind":"workflow","id":"<目录中的id>"},"slots":{"<槽位键>":"<值>"}}]` + "\n")
sb.WriteString("规则:kind 取 start/set_slot/cancel/clarify/chitchat/handoff 之一;")
sb.WriteString("start 必须带 action,action.kind 取 auto/command/workflow/blueprint/handler;")
sb.WriteString("action.id 与 slots 的键必须来自目录;用户闲聊用 chitchat(不带 action);")
sb.WriteString("无法判断时输出空数组 []。\n能力目录:\n")
for _, spec := range catalog {
sb.WriteString("- id=" + spec.ID)
if spec.Description != "" {
sb.WriteString(" 描述=" + spec.Description)
}
if spec.Handler != "" {
sb.WriteString(" handler=" + spec.Handler)
}
if len(spec.Slots) > 0 {
keys := make([]string, 0, len(spec.Slots))
for _, sl := range spec.Slots {
keys = append(keys, sl.Key)
}
sb.WriteString(" 槽位=[" + strings.Join(keys, ",") + "]")
}
sb.WriteString("\n")
}
return sb.String()
}
+149
View File
@@ -0,0 +1,149 @@
package intentsem
import (
"context"
"errors"
"strings"
"testing"
agentintent "agent-intent/intent"
agentllm "agent-native/runtime/llm"
)
// fakeEmbedder 返回脚本化向量:texts[i] → vecs[i](循环取模)。
type fakeEmbedder struct {
vecs [][]float32
err error
}
func (f fakeEmbedder) Embed(_ context.Context, texts []string) ([][]float32, error) {
if f.err != nil {
return nil, f.err
}
out := make([][]float32, len(texts))
for i := range texts {
out[i] = f.vecs[i%len(f.vecs)]
}
return out, nil
}
func TestEmbeddingScorerMaxCosine(t *testing.T) {
// input 与 example2 同向(相似度 1),与 example1 正交(0)。
s := EmbeddingScorer{Provider: fakeEmbedder{vecs: [][]float32{
{1, 0}, {0, 1}, {2, 0},
}}}
got, err := s.Score(context.Background(), "输入", []string{"例一", "例二"})
if err != nil {
t.Fatal(err)
}
if got != 1 {
t.Fatalf("期望最大余弦 1,得到 %v", got)
}
}
func TestEmbeddingScorerErrorPropagates(t *testing.T) {
s := EmbeddingScorer{Provider: fakeEmbedder{err: errors.New("boom")}}
if _, err := s.Score(context.Background(), "x", []string{"y"}); err == nil {
t.Fatal("期望错误上抛(库侧保持 Ambiguous)")
}
}
func TestEmbeddingScorerEmptyExamples(t *testing.T) {
s := EmbeddingScorer{Provider: fakeEmbedder{vecs: [][]float32{{1}}}}
got, err := s.Score(context.Background(), "x", nil)
if err != nil || got != 0 {
t.Fatalf("空示例应得 0 分无错误,得到 %v / %v", got, err)
}
}
// scriptedStream 回放固定文本增量后关闭通道。
func scriptedStream(text string, err error) agentllm.StreamFunc {
return func(_ context.Context, _ string, msgs []agentllm.Message, _ []agentllm.ToolDecl, _ agentllm.StreamOptions) <-chan agentllm.Event {
ch := make(chan agentllm.Event, 3)
if err != nil {
ch <- agentllm.Event{Kind: agentllm.KindError, Err: err}
} else {
ch <- agentllm.Event{Kind: agentllm.KindTextDelta, Text: text}
}
close(ch)
return ch
}
}
func catalogFixture() []agentintent.CatalogSpec {
return []agentintent.CatalogSpec{
{ID: "deploy.run", Description: "部署应用", Slots: []agentintent.CatalogSlot{{Key: "env", Required: true}}},
}
}
func TestLightGeneratorParsesCommands(t *testing.T) {
g := LightGenerator{
Stream: scriptedStream(" ```json\n[{\"kind\":\"start\",\"action\":{\"kind\":\"workflow\",\"id\":\"deploy.run\"},\"slots\":{\"env\":\"prod\"}}]\n``` ", nil),
Model: "lite",
}
cmds, err := g.Generate(context.Background(), "把应用部署到生产", catalogFixture())
if err != nil {
t.Fatal(err)
}
if len(cmds) != 1 || cmds[0].Kind != agentintent.CommandStart {
t.Fatalf("期望 1 条 start 命令,得到 %+v", cmds)
}
if cmds[0].Action.Kind != "workflow" || cmds[0].Action.ID != "deploy.run" {
t.Fatalf("action 不符:%+v", cmds[0].Action)
}
if cmds[0].Slots["env"] != "prod" {
t.Fatalf("槽位不符:%+v", cmds[0].Slots)
}
}
func TestLightGeneratorEmptyCatalogNoCall(t *testing.T) {
called := false
g := LightGenerator{Stream: func(_ context.Context, _ string, _ []agentllm.Message, _ []agentllm.ToolDecl, _ agentllm.StreamOptions) <-chan agentllm.Event {
called = true
ch := make(chan agentllm.Event)
close(ch)
return ch
}}
cmds, err := g.Generate(context.Background(), "你好", nil)
if err != nil || cmds != nil || called {
t.Fatalf("空目录不应调用模型:cmds=%v err=%v called=%v", cmds, err, called)
}
}
func TestLightGeneratorStreamError(t *testing.T) {
g := LightGenerator{Stream: scriptedStream("", errors.New("dial refused"))}
if _, err := g.Generate(context.Background(), "部署", catalogFixture()); err == nil ||
!strings.Contains(err.Error(), "dial refused") {
t.Fatalf("期望流错误上抛,得到 %v", err)
}
}
func TestLightGeneratorBadJSON(t *testing.T) {
g := LightGenerator{Stream: scriptedStream("我无法判断", nil)}
if _, err := g.Generate(context.Background(), "部署", catalogFixture()); err == nil {
t.Fatal("期望解析失败返回错误(库侧 fail-closed 回 fallback)")
}
}
func TestParseCommandsFencedAndChatty(t *testing.T) {
raw := "好的,命令如下:\n```json\n[{\"kind\":\"chitchat\"}]\n```\n以上。"
cmds, err := parseCommands(raw)
if err != nil {
t.Fatal(err)
}
if len(cmds) != 1 || cmds[0].Kind != agentintent.CommandChitchat {
t.Fatalf("期望 chitchat,得到 %+v", cmds)
}
}
func TestGeneratorPromptContainsCatalogOnly(t *testing.T) {
p := generatorSystemPrompt(catalogFixture())
for _, want := range []string{"deploy.run", "部署应用", "env"} {
if !strings.Contains(p, want) {
t.Fatalf("提示词缺少 %q", want)
}
}
if strings.Contains(p, "正例") || strings.Contains(p, "examples") {
t.Fatal("提示词不应包含示例文本(注入面)")
}
}
+62
View File
@@ -0,0 +1,62 @@
// Package intentsem 慢通道实现(I2 增量三):把 agent-intent 的
// SemanticScorer / CommandGenerator 端口接到宿主已有的向量与轻量模型
// 能力上。两个实现都是可插拔的——装配层默认不接线(设置开关关),
// 此时识别行为与 I1 完全一致。
package intentsem
import (
"context"
"math"
core "agent-system-core"
)
// EmbeddingScorer 用语义向量实现 agentintent.SemanticScorer:
// 对 input 与每个 example 取向量,返回最大余弦相似度。
// 任何错误都原样上抛——库侧约定 scorer 出错时识别保持 Ambiguous,
// 不会使 Recognize 失败(fail-open 到规则层结论)。
type EmbeddingScorer struct {
Provider core.EmbeddingProvider
}
// Score 返回 input 与 examples 的最高余弦相似度,取值 [0,1]。
func (s EmbeddingScorer) Score(ctx context.Context, input string, examples []string) (float64, error) {
if s.Provider == nil || len(examples) == 0 {
return 0, nil
}
texts := make([]string, 0, len(examples)+1)
texts = append(texts, input)
texts = append(texts, examples...)
vecs, err := s.Provider.Embed(ctx, texts)
if err != nil {
return 0, err
}
if len(vecs) != len(texts) || len(vecs[0]) == 0 {
return 0, nil
}
best := 0.0
for _, v := range vecs[1:] {
if sim := cosine(vecs[0], v); sim > best {
best = sim
}
}
return best, nil
}
// cosine 计算两个等长向量的余弦相似度;任一零向量或不等长返回 0。
func cosine(a, b []float32) float64 {
if len(a) == 0 || len(a) != len(b) {
return 0
}
var dot, na, nb float64
for i := range a {
fa, fb := float64(a[i]), float64(b[i])
dot += fa * fb
na += fa * fa
nb += fb * fb
}
if na == 0 || nb == 0 {
return 0
}
return dot / (math.Sqrt(na) * math.Sqrt(nb))
}
+130 -17
View File
@@ -1,8 +1,14 @@
// Package pgbus 是 PG 档的事件跨实例扇出(阶段 3):
// Package pgbus 是 PG 档的事件跨实例扇出(阶段 3 / 协作存储 U6 切片一):
// 落盘事务提交后 NOTIFY agent_events '<conv>|<lastSeq>';每实例一个 LISTEN 连接,
// 收到通知按 seq 从库取回事件并转发本地 bus(SSE 以 seq 去重,双投无害)。
// 收到通知按会话游标从库分页追赶转发本地 bus(SSE 以 seq 去重,重投无害)。
// 多实例部署契约:run 在其归属实例内存中执行,会话流量需按会话亲和路由
// (网关/客户端择点);本包只解决“其他实例的订阅者”可见性。
//
// U6(KTD6)修复的两个既有缺口:
// - 批内遗漏:通知只带末 seq,旧实现 EventAt 只取末条——批内中间事件对
// 实时订阅者不可见。现按会话游标 EventsAfter 分页追赶,末 seq 即水位;
// - 断线假重连:旧实现对原连接继续 WaitForNotification。现真正重建
// 连接→LISTEN→按全部已知会话追赶,另有周期补查覆盖通知丢失。
package pgbus
import (
@@ -10,6 +16,7 @@ import (
"log"
"strconv"
"strings"
"sync"
"time"
"agent-system-backend/internal/bus"
@@ -21,6 +28,12 @@ import (
const channel = "agent_events"
// catchUpPage 追赶分页大小(验收含 1000+ 事件分页可见)。
const catchUpPage = 500
// rescueEvery 周期补查间隔:覆盖 NOTIFY 丢失与监听窗口前后竞争(KTD6)。
const rescueEvery = 2 * time.Second
// Notifier 落盘后通知(任意连接池连接可用)。
type Notifier struct{ st *store.Store }
@@ -39,53 +52,153 @@ func (n *Notifier) Notify(conversationID string, evs []core.StreamEvent) {
}
}
// Listener 每实例一个:LISTEN → 按 seq 取回 → 本地 bus 转发。
// Listener 每实例一个:LISTEN → 按会话游标追赶 → 本地 bus 转发。
type Listener struct {
conn *pgx.Conn
dsn string
st *store.Store
bus *bus.Bus
done chan struct{}
mu sync.Mutex
cursor map[string]int64 // conv → 已转发水位(重启清零:SSE 侧重放兜底)
}
// Start 建立监听连接并开始转发循环;返回后即可用,随 ctx 取消停机。
func Start(ctx context.Context, dsn string, st *store.Store, b *bus.Bus) (*Listener, error) {
conn, err := pgx.Connect(ctx, dsn)
if err != nil {
l := &Listener{dsn: dsn, st: st, bus: b, done: make(chan struct{}), cursor: map[string]int64{}}
if err := l.connect(ctx); err != nil {
return nil, err
}
if _, err := conn.Exec(ctx, "LISTEN "+channel); err != nil {
conn.Close(ctx)
return nil, err
}
l := &Listener{conn: conn, st: st, bus: b, done: make(chan struct{})}
go l.loop(ctx)
return l, nil
}
func (l *Listener) connect(ctx context.Context) error {
conn, err := pgx.Connect(ctx, l.dsn)
if err != nil {
return err
}
if _, err := conn.Exec(ctx, "LISTEN "+channel); err != nil {
conn.Close(ctx)
return err
}
l.conn = conn
return nil
}
func (l *Listener) loop(ctx context.Context) {
defer close(l.done)
// 周期补查独立 goroutine:通知丢失场景恰恰没有通知可触发——不能只在
// 收到通知后顺带补查(KTD6)。
go l.rescueLoop(ctx)
for {
n, err := l.conn.WaitForNotification(ctx)
if err != nil {
if ctx.Err() != nil {
return // 停机
}
log.Printf("[pgbus] wait: %v(1s 后重连)", err)
time.Sleep(time.Second)
// 真重连(U6):旧实现对原连接继续 Wait——断线后永不恢复。
// 退避重建 → LISTEN → 按已知会话追赶(补断线窗口内落盘的事件)。
log.Printf("[pgbus] wait: %v(重连中)", err)
l.reconnect(ctx)
continue
}
conv, seq, ok := parsePayload(n.Payload)
if !ok {
continue
}
ev, err := l.st.EventAt(conv, seq)
if err != nil {
continue // 事件不可见(极早期窗口)——SSE 重放兜底
}
l.bus.Publish(ev)
l.catchUp(conv, seq)
}
}
// rescueLoop 周期补查心跳:随 ctx 或主循环退出(done)停止。
func (l *Listener) rescueLoop(ctx context.Context) {
t := time.NewTicker(rescueEvery)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-l.done:
return
case <-t.C:
l.rescueKnown()
}
}
}
// reconnect 重建监听连接并立即追赶全部已知会话(有界退避重试)。
func (l *Listener) reconnect(ctx context.Context) {
_ = l.conn.Close(context.Background())
for attempt := 0; attempt < 3; attempt++ {
cctx, cancel := context.WithTimeout(ctx, 5*time.Second)
err := l.connect(cctx)
cancel()
if err == nil {
l.rescueKnown()
return
}
select {
case <-ctx.Done():
return
case <-time.After(time.Duration(attempt+1) * time.Second):
}
}
}
// rescueKnown 周期补查:不经通知,按各会话游标继续分页追赶(覆盖 NOTIFY
// 丢失、监听生效窗口前后竞争——KTD6「只有落库事件才是重放依据」)。
// 调用方:重连成功后与 loop 的 rescue 心跳(消息稀疏时由 ticker 兜底)。
func (l *Listener) rescueKnown() {
l.mu.Lock()
convs := make([]string, 0, len(l.cursor))
for c := range l.cursor {
convs = append(convs, c)
}
l.mu.Unlock()
for _, conv := range convs {
l.catchUp(conv, 1<<62)
}
}
// catchUp 从会话游标之后分页取回并转发,直到取空或越过水位 toSeq。
// 首见会话从 0 开始(SSE 端按 seq 去重,重投无害;这保证批内中间事件可见)。
func (l *Listener) catchUp(conv string, toSeq int64) {
l.mu.Lock()
since := l.cursor[conv]
l.mu.Unlock()
for {
evs, err := l.st.EventsAfter(conv, since, catchUpPage)
if err != nil {
log.Printf("[pgbus] catchup %s@%d: %v", conv, since, err)
return
}
stop := false
for _, ev := range evs {
if ev.Seq > toSeq {
stop = true
break
}
l.bus.Publish(ev)
since = ev.Seq
}
l.setCursor(conv, since)
if stop || len(evs) < catchUpPage {
return
}
}
}
func (l *Listener) setCursor(conv string, seq int64) {
l.mu.Lock()
if seq > l.cursor[conv] {
l.cursor[conv] = seq
}
l.mu.Unlock()
}
// parsePayload 拆 '<conv>|<seq>'(会话 ID 不含 '|')。
func parsePayload(p string) (string, int64, bool) {
i := strings.LastIndex(p, "|")
if i < 0 {
+161 -1
View File
@@ -29,6 +29,12 @@ func pgTestStore(t *testing.T) (*store.Store, string) {
if err != nil {
t.Fatal(err)
}
// pg_trgm 预装 public(幂等),理由见 store 包 openTestStore 注释:
// 避免扩展落进用例 schema 后随 CASCADE 删除造成的并行竞态。
if _, err := admin.Exec(`CREATE EXTENSION IF NOT EXISTS pg_trgm`); err != nil {
admin.Close()
t.Fatalf("preinstall pg_trgm: %v", err)
}
if _, err := admin.Exec("CREATE SCHEMA " + schema); err != nil {
admin.Close()
t.Fatal(err)
@@ -37,7 +43,7 @@ func pgTestStore(t *testing.T) (*store.Store, string) {
admin.Exec("DROP SCHEMA IF EXISTS " + schema + " CASCADE")
admin.Close()
})
full := dsn + "&search_path=" + schema
full := store.AppendPGRuntimeParam(dsn, "search_path", schema+",public")
st, err := store.OpenPostgres(full, filepath.Join(t.TempDir(), "artifacts"))
if err != nil {
t.Fatal(err)
@@ -80,3 +86,157 @@ func TestNotifyListenerFanout(t *testing.T) {
t.Fatal("no fanout within 3s")
}
}
// ---- U6 切片一(KTD6):批内全可见、分页追赶、断线重连、通知丢失补查 ----
// drainUntil 从订阅通道收集事件直到谓词为真或超时;返回收到的全部事件。
func drainUntil(ch <-chan core.StreamEvent, done func([]core.StreamEvent) bool, timeout time.Duration) []core.StreamEvent {
var got []core.StreamEvent
deadline := time.After(timeout)
for {
select {
case e := <-ch:
got = append(got, e)
if done(got) {
return got
}
case <-deadline:
return got
}
}
}
// 一批 64 条事件(单条 NOTIFY 只带末 seq)对实时订阅者全部可见——
// 旧实现 EventAt 只取末条,批内 1..63 全部遗漏。
func TestBatchEventsAllVisible(t *testing.T) {
st, dsn := pgTestStore(t)
b2 := bus.New()
listener, err := Start(context.Background(), dsn, st, b2)
if err != nil {
t.Fatal(err)
}
defer listener.Close(context.Background())
ch, stop := b2.Subscribe("conv_batch")
defer stop()
notifier := NewNotifier(st)
evs := make([]core.StreamEvent, 64)
for i := range evs {
evs[i] = core.StreamEvent{ConversationID: "conv_batch", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)}
}
if _, err := st.AppendEvents("conv_batch", evs); err != nil {
t.Fatal(err)
}
notifier.Notify("conv_batch", evs)
got := drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= 64 }, 5*time.Second)
if len(got) != 64 {
t.Fatalf("批 64 条应全部可见,收到 %d(旧实现只末条可见)", len(got))
}
// 序号连续(首见会话从 0 追赶,SSE 去重语义下的完整性)。
for i, e := range got {
if e.Seq != int64(i+1) {
t.Fatalf("序号不连续:got[%d].Seq=%d", i, e.Seq)
}
}
}
// 1200 条(跨 500/页 的三页)分页追赶全部可见。
func TestPagedCatchUpBeyondOnePage(t *testing.T) {
st, dsn := pgTestStore(t)
b2 := bus.New()
listener, err := Start(context.Background(), dsn, st, b2)
if err != nil {
t.Fatal(err)
}
defer listener.Close(context.Background())
ch, stop := b2.Subscribe("conv_page")
defer stop()
notifier := NewNotifier(st)
const total = 1200
evs := make([]core.StreamEvent, 0, total)
for i := 0; i < total; i++ {
evs = append(evs, core.StreamEvent{ConversationID: "conv_page", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)})
}
if _, err := st.AppendEvents("conv_page", evs); err != nil {
t.Fatal(err)
}
notifier.Notify("conv_page", evs)
got := drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= total }, 15*time.Second)
if len(got) != total {
t.Fatalf("1200 条应分页全部可见,收到 %d", len(got))
}
}
// 断线重连:底层连接被杀后,新落盘+通知的事件仍可达(真重连路径)。
func TestListenerReconnectsAfterConnLoss(t *testing.T) {
st, dsn := pgTestStore(t)
b2 := bus.New()
listener, err := Start(context.Background(), dsn, st, b2)
if err != nil {
t.Fatal(err)
}
defer listener.Close(context.Background())
ch, stop := b2.Subscribe("conv_re")
defer stop()
notifier := NewNotifier(st)
first := []core.StreamEvent{{ConversationID: "conv_re", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)}}
if _, err := st.AppendEvents("conv_re", first); err != nil {
t.Fatal(err)
}
notifier.Notify("conv_re", first)
drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= 1 }, 3*time.Second)
// 杀掉监听连接(模拟网络断开;旧实现会对此连接永远 Wait 下去)。
if err := listener.conn.Close(context.Background()); err != nil {
t.Logf("close conn: %v", err)
}
// 断线窗口内落盘一批(重连后应经追赶补上)。
during := []core.StreamEvent{{ConversationID: "conv_re", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)}}
if _, err := st.AppendEvents("conv_re", during); err != nil {
t.Fatal(err)
}
notifier.Notify("conv_re", during)
// 第二段 drain 从零计数:断线窗口内的一条事件到达即证明重连+追赶生效。
got := drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= 1 }, 10*time.Second)
if len(got) < 1 {
t.Fatalf("重连后应补见断线窗口内事件:got=%d", len(got))
}
}
// 通知丢失补查:事件落盘但**不**发 NOTIFY,周期补查应在 rescueEvery 内送达。
func TestRescueCoversLostNotifications(t *testing.T) {
st, dsn := pgTestStore(t)
b2 := bus.New()
listener, err := Start(context.Background(), dsn, st, b2)
if err != nil {
t.Fatal(err)
}
defer listener.Close(context.Background())
ch, stop := b2.Subscribe("conv_lost")
defer stop()
// 先用一次正常通知建立该会话的游标(补查只覆盖已知会话)。
notifier := NewNotifier(st)
seed := []core.StreamEvent{{ConversationID: "conv_lost", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)}}
if _, err := st.AppendEvents("conv_lost", seed); err != nil {
t.Fatal(err)
}
notifier.Notify("conv_lost", seed)
drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= 1 }, 3*time.Second)
// 只落盘、不通知(模拟 NOTIFY 丢失)。
lost := []core.StreamEvent{{ConversationID: "conv_lost", RunID: "r1", TS: time.Now().UTC().Format(time.RFC3339Nano), V: 1, Type: core.EvRunPhase, Payload: []byte(`{}`)}}
if _, err := st.AppendEvents("conv_lost", lost); err != nil {
t.Fatal(err)
}
got := drainUntil(ch, func(g []core.StreamEvent) bool { return len(g) >= 1 }, 3*rescueEvery+2*time.Second)
if len(got) < 1 {
t.Fatalf("周期补查应送达无通知事件:got=%d", len(got))
}
}
+41
View File
@@ -0,0 +1,41 @@
package plugins
import (
"bytes"
"encoding/json"
"fmt"
core "agent-system-core"
)
// agentBannedKeys 是插件代理声明禁带的键:钩子、MCP 与权限模式归宿主与
// 基座,插件画像只能声明身份与工具面(N1 WP1 验收条款)。
var agentBannedKeys = map[string]bool{
"hooks": true,
"mcp": true,
"permissionMode": true,
}
// parseAgentAsset 解析 agents/ 资产:禁带键定位到字段、未知键拒绝、
// id 必填。任一违规由调用方整批拒绝该插件(fail-closed)。
func parseAgentAsset(raw []byte) (core.PluginAgentDecl, error) {
var probe map[string]json.RawMessage
if err := json.Unmarshal(raw, &probe); err != nil {
return core.PluginAgentDecl{}, fmt.Errorf("agent json: %w", err)
}
for key := range probe {
if agentBannedKeys[key] {
return core.PluginAgentDecl{}, fmt.Errorf("插件代理不得声明 %s", key)
}
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
var item core.PluginAgentDecl
if err := dec.Decode(&item); err != nil {
return core.PluginAgentDecl{}, fmt.Errorf("agent json: %w", err)
}
if item.ID == "" {
return core.PluginAgentDecl{}, fmt.Errorf("agent missing id")
}
return item, nil
}
+72
View File
@@ -0,0 +1,72 @@
package plugins
import (
"os"
"path/filepath"
"strings"
"testing"
core "agent-system-core"
)
// writeAgentPlugin 写一个带 agents/ 资产的插件目录。
func writeAgentPlugin(t *testing.T, dir, id, agentJSON string) {
t.Helper()
root := filepath.Join(dir, id)
os.MkdirAll(filepath.Join(root, "agents"), 0o755)
write := func(rel, content string) {
if err := os.WriteFile(filepath.Join(root, rel), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
write("plugin.json", `{"id":"`+id+`","name":"代理插件","version":"0.1.0","apiVersion":"`+core.PluginAPIVersion+`",
"provides":{"agents":["reviewer"]}}`)
write("agents/reviewer.json", agentJSON)
}
func TestAgentAssetLoads(t *testing.T) {
h, _, dir := newHost(t)
writeAgentPlugin(t, dir, "agentplug", `{"id":"rdm-reviewer","title":"章节评审","instructions":"逐条核对来源","tools":["read_file","grep"]}`)
if err := h.LoadDir()["agentplug"]; err != nil {
t.Fatal(err)
}
for _, ld := range h.List() {
if ld.Manifest.ID != "agentplug" {
continue
}
if len(ld.Agents) != 1 || ld.Agents[0].ID != "rdm-reviewer" || len(ld.Agents[0].Tools) != 2 {
t.Fatalf("agents = %+v", ld.Agents)
}
return
}
t.Fatal("plugin not loaded")
}
// TestAgentAssetBansHostOnlyKeys:插件代理声明 hooks/mcp/permissionMode 时
// 整批拒绝且错误定位到字段(N1 WP1 验收条款)。
func TestAgentAssetBansHostOnlyKeys(t *testing.T) {
for _, key := range []string{"hooks", "mcp", "permissionMode"} {
h, _, dir := newHost(t)
writeAgentPlugin(t, dir, "badplug", `{"id":"x","`+key+`":{}}`)
err := h.LoadDir()["badplug"]
if err == nil {
t.Fatalf("%s accepted", key)
}
if !strings.Contains(err.Error(), "不得声明 "+key) {
t.Fatalf("%s error 未定位到字段: %v", key, err)
}
}
}
func TestAgentAssetRejectsUnknownKeysAndMissingID(t *testing.T) {
h, _, dir := newHost(t)
writeAgentPlugin(t, dir, "unknownkey", `{"id":"x","bogus":1}`)
if err := h.LoadDir()["unknownkey"]; err == nil {
t.Fatal("unknown key accepted")
}
h2, _, dir2 := newHost(t)
writeAgentPlugin(t, dir2, "noid", `{"title":"无 id"}`)
if err := h2.LoadDir()["noid"]; err == nil {
t.Fatal("missing id accepted")
}
}
+11 -2
View File
@@ -9,6 +9,7 @@ import (
"path/filepath"
"strings"
"agent-system-backend/internal/skills"
"agent-system-backend/internal/store"
"agent-system-backend/internal/tools"
core "agent-system-core"
@@ -59,6 +60,10 @@ func (h *Host) loadDeclared(dir string, m core.PluginManifest, ld *Loaded) error
if err != nil {
return fmt.Errorf("plugins: %s: skill %s: %w", m.ID, name, err)
}
// 技能按开放规范校验(N2 WP4):不合规整批拒载,字段定位到键。
if err := skills.ValidateForPlugin(m.ID, name, body); err != nil {
return err
}
if ld.Skills == nil {
ld.Skills = map[string]string{}
}
@@ -98,8 +103,12 @@ func (h *Host) loadDeclared(dir string, m core.PluginManifest, ld *Loaded) error
ld.Parsers = append(ld.Parsers, item)
}
for _, name := range m.Provides.Agents {
var item core.PluginAgentDecl
if err := readAsset(dir, "agents", name, &item); err != nil {
raw, err := readPluginFile(dir, filepath.Join("agents", name+".json"))
if err != nil {
return fmt.Errorf("plugins: %s: agent %s: %w", m.ID, name, err)
}
item, err := parseAgentAsset(raw)
if err != nil {
return fmt.Errorf("plugins: %s: agent %s: %w", m.ID, name, err)
}
ld.Agents = append(ld.Agents, item)
+32 -1
View File
@@ -3,6 +3,7 @@ package plugins
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -33,7 +34,7 @@ func TestV2AssetsAndNetwork(t *testing.T) {
"provides":{"tools":["echo"],"skills":["note"],"mcpServers":["local"],"knowledge":["regs"],"parsers":["pdf"],"agents":["writer"],"channels":["feishu"],"pages":["home"]}
}`)
write("tools/echo.json", `{"name":"v2_echo","description":"e","parameters":{"type":"object"},"http":{"method":"GET","url":"https://api.example.com/x"}}`)
write("skills/note.md", "# note\n")
write("skills/note.md", "---\nname: note\ndescription: 笔记技能\n---\n\n# note\n")
write("mcp/local.json", `{"name":"local","command":"echo"}`)
write("knowledge/regs.json", `{"id":"regs","title":"法规"}`)
write("parsers/pdf.json", `{"id":"pdf","title":"PDF","mime":"application/pdf"}`)
@@ -95,3 +96,33 @@ func TestSidecarTouch(t *testing.T) {
}
t.Fatal("sidecar did not run")
}
// N2 WP4:不合规插件技能整批拒载,错误带插件上下文与字段定位。
func TestV2InvalidSkillRejected(t *testing.T) {
host, _, dir := newHost(t)
root := filepath.Join(dir, "badskill")
if err := os.MkdirAll(filepath.Join(root, "skills"), 0o755); err != nil {
t.Fatal(err)
}
write := func(rel, body string) {
t.Helper()
if err := os.WriteFile(filepath.Join(root, rel), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
write("plugin.json", `{
"id":"badskill","name":"Bad","version":"0.1.0","apiVersion":"`+core.PluginAPIVersionExperimental+`",
"backend":{"schemaVersion":1},
"provides":{"skills":["note"]}
}`)
// 缺 name(开放规范必填)。
write("skills/note.md", "---\ndescription: 没有名字\n---\n正文\n")
err := host.LoadDir()["badskill"]
if err == nil {
t.Fatal("不合规技能必须拒载")
}
if !strings.Contains(err.Error(), "plugins: badskill: skill note:") ||
!strings.Contains(err.Error(), "name") {
t.Fatalf("错误须带插件上下文与字段定位:%v", err)
}
}
+35
View File
@@ -0,0 +1,35 @@
// Package projectscope resolves host retrieval filters using service-provided
// lifecycle metadata. Stored/client settings never supply an authority mapping.
package projectscope
import (
"agent-system-backend/internal/store"
"agent-system-backend/pkg/extension"
core "agent-system-core"
"context"
)
func Resolve(ctx context.Context, host extension.Host, p extension.ProjectLifecycle, id core.ID) (core.ID, error) {
if id == "" {
return id, nil
}
if p.List == nil {
return "", store.ErrNotFound
}
projects, err := p.List(ctx, host)
if err != nil {
return "", err
}
for _, project := range projects {
domainID, _ := project.Settings["rdm.projectId"].(string)
if project.ID != id && domainID != id {
continue
}
state, _ := project.Settings["rdm.state"].(string)
if domainID == "" || state != "ready" {
return "", store.ErrNotFound
}
return core.ID(domainID), nil
}
return "", store.ErrNotFound
}
@@ -1,7 +1,7 @@
---
id: agent.sub.explore
layer: agent
version: 1
version: 2
title: 探索子代理(只读)
description: 只读检索专家——定位文件/内容/事实,不做任何修改
tools: knowledge_search, web_search, tavily_search, time_now, list_conversations, read_conversation, fs_read_file, fs_list_dir
@@ -9,6 +9,7 @@ model: lightweight
---
你是只读探索子代理,职责是高效且彻底地定位目标信息。
- 汇报发现时给出定位(文件/页/段落);数字必须来自材料原文并注明出处,没有出处的不要写。
- 严格只读:不写入、不修改、不删除任何文件或数据;只执行检索与读取类操作。
- 检索策略:从任务给出的线索出发,多角度查询(不同关键词、不同路径);一次不中就换查询,不重复同一查询。
- 返回结论时给出定位(文件/路径/段落/链接),让派生方能直接复核。
@@ -1,14 +1,16 @@
---
id: agent.sub.general
layer: agent
version: 1
version: 3
title: 通用子代理
description: 独立完成子任务并返回结构化结果(默认子代理提示词)
tools: knowledge_search, web_search, tavily_search, time_now, list_conversations, read_conversation, fs_read_file, fs_list_dir, write_artifact, quality_review
tools: knowledge_search, web_search, tavily_search, time_now, list_conversations, read_conversation, fs_read_file, fs_list_dir, write_artifact, quality_review, generative-ui.present
model: lightweight
---
你是被派生的子代理,负责独立完成一个子任务。
- 结论中的关键数字与事实附来源定位(材料/文件与页/段落);没有出处的数字不要写,无法回溯时说明来源缺失。
- 把任务完整做完:既不过度修饰,也不半途而废;遇到障碍先自行尝试解决,确实无法解决时在结果中说明卡点与已尝试的手段。
- 只使用与任务相关的信息,不臆测派生方未提供的上下文。
- 完成时返回结构化结果:做了什么、关键发现/结论、产出物位置(如有)、遗留问题(如有)。分点列出,关键内容不省略;摘要长度服从内容,不为简短而丢信息。
- 页面化产物(分析页/对比页等)用 `generative-ui.present` 提交;工具不可用或提交被拒时改用 `write_artifact` 落盘并在结果中说明。
@@ -1,7 +1,7 @@
---
id: agent.sub.research
layer: agent
version: 1
version: 3
title: 调研子代理
description: 围绕问题多源调研,输出带依据的结论
tools: knowledge_search, web_search, tavily_search, time_now, list_conversations, read_conversation, fs_read_file, fs_list_dir
@@ -9,6 +9,7 @@ model: lightweight
---
你是调研子代理,围绕给定问题收集信息并给出结论。
- 结论里的关键数字与事实必须附原文引用(材料/文件与页/段落定位,重要处引原句);没有出处的数字不要写,无法回溯时明确说明「来源缺失」而不是估一个。检索/读取结果带【src_N|工具】标记时,引用同时给出该标记编号。
- 先把问题拆成可回答的子问题,再逐一检索;优先项目内知识库与已给材料,其次按任务指定的外部来源。
- 每条结论标注依据来源;多来源冲突时不和稀泥,列出各方说法与差异点,说明你采信哪个及理由。
- 覆盖正反两面:支持与不支持该结论的证据都要呈现。
@@ -1,7 +1,7 @@
---
id: base.deliverables
layer: base
version: 5
version: 7
title: 交付物工作流
description: 生成文档、表格、演示稿的工作流、样式决策与技能入口(常驻基座)
---
@@ -11,11 +11,13 @@ description: 生成文档、表格、演示稿的工作流、样式决策与技
1. **定样式**,按下面的优先级只取第一个命中项:本轮用户的明确指令(例如改成公文、换成品牌色)→ 本轮上传的模板(.docx/.dotx/.pptx/.potx/.xlsx/.xltx)→ 本轮要求仿制的参考文档(`from housestyle.extract import extract`,抽取字体、字号、色板、页边距,得到自定义主题)→ 本会话已经确定的样式 → 设置中的默认模板 → 设置中的 deliverables.style(modern / formal / minimal)和强调色。未指定时用环境段给出的默认样式。
2. **规划版面**:先定读者、结构和每一节的内容,再写脚本。不要边生成边补空节。
3. **生成**:`import housestyle`,用 docx_kit / pptx_kit / xlsx_kit 的构件写文件。文件落在会话工作区。模板文件和参考原件在 `inputs/`(环境段列出相对路径)。
4. **自检**:`python -m housestyle.lint <文件>`。输出为 JSON;存在 severity=error 时必须修好再继续,不要带着 error 登记。然后用 `housestyle.summarize` 回读标题、表格和页数,核对与规划一致。
3. **生成(Word 交付物优先走源码化)**:先写 Markdown 源文件(`write_file 方案.md`,约定见 `python3 -m housestyle.build` 的 docstring:`#`标题/`##`-`####` 三级、`-`/`1.` 列表、`| |` 表格、`>` 引用、`!!! 标签 文本` 强调块、首行 `<!-- housestyle: theme=modern -->` 可选主题),再 `python3 -m housestyle.build 方案.md` 确定性渲染成 docx——build 自动跑 lint 并回读结构(写入即诊断),有 error 就改 .md 源的对应段落再重建,不要整段重写。修改已有交付物 = `edit_file` 改 .md 源 + 重新 build。Excel/PPT/图片等其他类型仍用 `run_python` + 对应 kit。文件都在会话工作区;模板和参考原件在 `inputs/`(环境段列出相对路径)。
4. **自检**:`python3 -m housestyle.lint <文件>`。输出为 JSON;存在 severity=error 时必须修好再继续,不要带着 error 登记。然后用 `python3 -m housestyle.outline_cli <文件>` 回读标题、表格和页数,核对与规划一致。
5. **登记**:`register_artifact`(path 传相对路径)。登记后对话里出现可预览的卡片。
6. **收尾**:回复只写结论摘要,并说明这次用的样式(主题、是否模板、是否仿制)以及用户还能怎么调。不要在回复里复述全文,也不要只口头说文件名。
检索与读取的工具结果末尾带【src_N|工具名】来源标记:成文时数字与关键事实的出处就是这些带标记的结果,逐项核对;交付物里的每一个百分比、日期和大额数字,都必须能在本轮上传材料或你自己的工具/计算输出里找到出处;找不到出处的,要么删除,要么用「目标 / 预计 / 假设」等词在文中显式标注为规划量(如「目标 85%」「预计三个月」)。终轮会对无出处且未标注的数字拦截两轮;仍无法回溯又拒绝标注的,会被系统如实记为未核实。
没有调用工具、没有登记工件时,禁止说「已生成 / 已保存 / 已导出」。回复里不要写工件 ID(art_…)。工具只能通过函数调用触发,不要在回复里书写 `[调用工具 …]`、`[工具结果 …]`、`[产出工件 …]` 这类文字。
交付物族(所有任务共用,不限于会议):
+17
View File
@@ -0,0 +1,17 @@
---
id: base.method
layer: base
version: 1
title: 工作方法
description: 把任务做完的方法论:计划、换路、查 API、坚持到完成、回到材料(常驻基座)
---
按以下方法工作:
- **先计划再动手**:预计三步以上、或要产出文件的任务,开始前用 update_plan 列出步骤;每完成一步就更新状态,同一时间只有一步进行中。计划是给自己看的进度账,不是给用户看的排场。
- **一个方法失败两次就换路**:同样的调用连续失败两次,停下来想第三条路(换工具、换参数结构、换实现方式),不要原样重试第三次。
- **写脚本前先查 API**:调用不熟悉的库/模块前,先用 grep 查它的函数签名(`inspect.signature`/`dir`)或读 SKILL.md 的示例,确认参数形态后再写完整脚本。猜 API 是失败的最大来源。
- **编辑粒度**:修改文件时先想清楚要改哪几处,用 multi_edit 或大块 edit 一次改完,不要一行一个 edit_file 连发十几次;改完立即验证(build/运行),验证通过再继续下一处。
- **坚持到完成**:任务没做完不要停。收尾前检查最后一段:如果还是计划、承诺或"接下来我将",就先把承诺的事做掉。只有用户输入才能解决的阻塞才允许停下来问。
- **重任务走快慢双速**:预计要跑几分钟的任务(整篇文档深度解析、多文件批量处理、成套材料整理),先用当前回复给用户即时反馈——材料的相关性判断、概要或执行计划——再用 delegate_task 的 background=true 把深度工作放到后台执行;后台进度与结果会进收件箱卡片,完成后系统自动汇报。简单任务仍同步做完,不要为快而快把小事也丢后台。
- **结论回到材料**:写进交付物的数字、百分比、日期、人名必须能在本轮材料(附件/工具输出/会议原文)里找到出处;找不到的改成不写,或明确标注"推测"。材料里的日期基准与你理解的不一致时,以材料为准或向用户确认。
- **终稿如实**:最后的回复先给结论,再列交付物与要点;没做完、没验证的部分如实说明,不用模糊措辞掩盖。
+1 -1
View File
@@ -7,7 +7,7 @@ description: 何时用工具、如何并行调用、如何处理工具结果
---
使用工具时遵循:
- 优先专用工具而非通用手段:检索知识用检索工具、写产物用工件工具、查时间用时间工具;能用结构化工具完成的,不绕道自由文本。
- 优先专用工具而非通用手段:读文件用 read_file(行号分页)、找内容用 grep、找文件用 glob、改文件用 edit_file/multi_edit(须先读过、old_string 唯一);会话工作区内的文件操作一律用这些工具,不用 run_shell 的 cat/sed 代替。检索知识用检索工具、查时间用时间工具;能用结构化工具完成的,不绕道自由文本。
- 相互独立的调用并行发起;有依赖的调用按依赖顺序串行。
- 调用前确认参数完整、类型正确;不确定参数含义时先查阅工具描述,不猜测字段。
- 工具结果是事实来源:基于结果作答并如实转述;结果为空或报错时说明情况,不假装拿到了数据。
+1 -1
View File
@@ -12,4 +12,4 @@ description: 动态注入的当前环境事实(占位符随运行填充)
- 本轮附带文档:{{attachments}}。文档全文已注入上下文(超长时会标注截断)。
- 交付物默认样式:{{deliverableStyle}}。本轮的明确指令、上传的模板或参考文档优先于该默认值。
- 技能目录:{{skillsDir}}。环境变量 AGENT_SKILLS_DIR 指向这里;Python 已能 `import housestyle`。
- 本轮原件(工作区相对路径):{{inputs}}。模板和参考文档用这些路径,不要向用户索取已经上传的文件。
- 会话工作区文件(相对路径,跨轮有效):{{inputs}}。原件旁的 <原名>.md 是解析出的纯文本,需要原始信息时优先读它(不要只依赖你上一轮生成的产物,也不要向用户索取已经上传的文件)。模板和参考文档用这些路径。
@@ -0,0 +1,8 @@
---
id: tool.edit_file
layer: tool
version: 1
title: edit_file 工具描述
description: 工作区文件精确编辑的模型侧说明文案
---
对工作区文件做唯一匹配替换:old 必须与文件内容逐字一致(含缩进)且在文件中唯一,不唯一时带上更多上下文行。改前必须先 read 过该文件;文件在读取后被外部修改会拒绝,重读即可。多处相关修改用 multi_edit 一次提交,不要一行一个调用连发。
+8
View File
@@ -0,0 +1,8 @@
---
id: tool.glob
layer: tool
version: 1
title: glob 工具描述
description: 文件名模式匹配的模型侧说明文案
---
按通配模式找文件路径(** 跨目录),用于"这类文件有哪些/在哪个目录"。找内容用 grep,不要用 glob 后逐个 cat。
+8
View File
@@ -0,0 +1,8 @@
---
id: tool.grep
layer: tool
version: 1
title: grep 工具描述
description: 工作区内容检索的模型侧说明文案
---
在工作区文件内容里做正则检索(支持 include 按文件名过滤、ignore_case)。用于定位章节、函数、数据点;配合 read_file 的 offset 直达相关段落,替代整文件通读。
@@ -0,0 +1,8 @@
---
id: tool.read_file
layer: tool
version: 1
title: read_file 工具描述
description: 工作区文件读取的模型侧说明文案
---
读取会话工作区内的文本文件,输出带行号(1-based)。用法:先不带 offset/limit 读前 2000 行看全貌;文件大时用 offset/limit 分页,不要反复从头读。编辑前必须先读过目标文件。长输出会被截断并提示分页方式。
@@ -0,0 +1,8 @@
---
id: tool.run_python
layer: tool
version: 1
title: run_python 工具描述
description: Python 执行原语的模型侧说明文案
---
执行完整 Python 脚本(python3,工作目录=会话工作区)。适用:数据处理、生成 Excel/PPT/图片(housestyle kit)、批量操作。Word 交付物优先写 .md 源后用 run_shell 跑 `python3 -m housestyle.build`,不要在脚本里手工拼 docx。报错先读退出码与 traceback 定位行号再改,同一错误不要原样重跑。生成带排式的文件(xlsx/pptx)先读 `$AGENT_SKILLS_DIR/house-style/SKILL.md` 的构件表,不确定签名先 `inspect.signature`。
@@ -0,0 +1,8 @@
---
id: tool.run_shell
layer: tool
version: 1
title: run_shell 工具描述
description: bash 执行原语的模型侧说明文案
---
在会话工作区内执行 bash 脚本(环境最小化:PATH/HOME/LANG;AGENT_SKILLS_DIR 与 PYTHONPATH 已注入)。适用:housestyle build/lint、pip install --user、文件批操作。读文件用 read_file、找内容用 grep——不要用 cat/sed/grep 命令行代替专用工具。同一命令失败两次换方法。
+33
View File
@@ -0,0 +1,33 @@
package prompts
import (
"io/fs"
"regexp"
"strings"
"testing"
)
var barePythonRe = regexp.MustCompile(`(?:^|[^0-9A-Za-z_.-])python -(?:m|c)\b`)
// 提示词库里的命令必须能在执行环境跑起来:解释器只有 python3。
// 实测模型照提示词执行 `python -m housestyle.lint` 得到 exit 127。
func TestLibraryUsesRunnablePythonCommands(t *testing.T) {
names, err := fs.Glob(libraryFS, "library/*.md")
if err != nil {
t.Fatalf("glob: %v", err)
}
var problems []string
for _, name := range names {
raw, err := libraryFS.ReadFile(name)
if err != nil {
t.Fatalf("read %s: %v", name, err)
}
body := string(raw)
for _, loc := range barePythonRe.FindAllStringIndex(body, -1) {
problems = append(problems, name+": bare `python -` command near: "+strings.TrimSpace(body[loc[0]:min(loc[0]+60, len(body))]))
}
}
if len(problems) > 0 {
t.Fatalf("unrunnable python commands in prompt library:\n%s", strings.Join(problems, "\n"))
}
}
+3 -3
View File
@@ -73,9 +73,9 @@ func TestResolveSystemDefaultBase(t *testing.T) {
if err != nil {
t.Fatal(err)
}
// 默认:base 七块(identity/behavior/caution/style/tools/deliverables/quality)+ env 一块,无技能注入。
if len(blocks) != 8 {
t.Errorf("blocks = %d, want 8 (7 base + env)", len(blocks))
// 默认:base 八块(identity/behavior/method/caution/style/tools/deliverables/quality)+ env 一块,无技能注入。
if len(blocks) != 9 {
t.Errorf("blocks = %d, want 9 (8 base + env)", len(blocks))
}
var envBody string
for _, b := range blocks {
+54
View File
@@ -0,0 +1,54 @@
package quality
import (
"os"
"path/filepath"
"strings"
"testing"
)
// P5/裁定 4:拦截路径只认硬事实(lint 实际运行且零 error);
// 关键词维度(栏目式标题/可执行性等)不得再把合格交付物拦下来。
func TestHardFactsIgnoresKeywordDimensions(t *testing.T) {
dir := t.TempDir()
// 评分整体会因「结论先行 20/可读性 25」判负的文本——硬事实口径必须放行
//(无 lint 或 lint 通过时)。
path := filepath.Join(dir, "报告.md")
if err := os.WriteFile(path, []byte("# 概述\n\n正文内容。\n"), 0o644); err != nil {
t.Fatal(err)
}
full := ScoreFile(path)
if full.Passed {
t.Fatalf("keyword dimensions should still fail the full score (fixture sanity): %+v", full)
}
hard := HardFacts("报告.md", []byte("# 概述\n\n正文内容。\n"))
if !hard.Passed {
t.Fatalf("hard facts must ignore keyword dimensions: %+v", hard.Violations)
}
if len(hard.Lint) > 0 {
for _, issue := range hard.Lint {
if issue.Severity == "error" {
t.Fatalf("lint errors must still block: %+v", issue)
}
}
}
}
// lint 不可运行(技能根已配置)仍是硬事实失败——fail-visible 保持。
func TestHardFactsKeepsLintUnavailableBlocking(t *testing.T) {
SetLintEnvRoot(t.TempDir())
t.Cleanup(func() { SetLintEnvRoot("") })
hard := HardFacts("报告.md", []byte("# 概述\n正文。\n"))
if hard.Passed {
t.Fatalf("lint-not-run must stay blocking: %+v", hard)
}
found := false
for _, v := range hard.Violations {
if strings.Contains(v, "自检未运行") {
found = true
}
}
if !found {
t.Fatalf("expected 自检未运行, got %v", hard.Violations)
}
}
+194 -12
View File
@@ -26,12 +26,15 @@ type Dimension struct {
}
// Report 是一次评分。Passed 要求总分不低于 70,且没有任何一维低于 40。
// LintRan 为假表示 housestyle.lint 没有实际执行(环境不可用);技能根已注入
// (SetLintEnvRoot)时这本身计为一条 Violations,不再把「没跑」静默当「没问题」。
type Report struct {
Passed bool `json:"passed"`
Score int `json:"score"`
Dimensions []Dimension `json:"dimensions"`
Violations []string `json:"violations,omitempty"`
Lint []LintIssue `json:"lint,omitempty"`
LintRan bool `json:"lintRan,omitempty"`
}
// LintIssue 是 housestyle.lint 的一条结果。lint 不存在时留空,不因此判失败。
@@ -41,6 +44,17 @@ type LintIssue struct {
Message string `json:"message"`
}
// lintEnvRoot 由组合根在启动时注入(<data>/skills)。零值保持旧语义:
// 继承进程环境,lint 不可用时静默跳过(测试与无技能部署)。
var lintEnvRoot string
// SetLintEnvRoot 注入技能根:lint 子进程获得与 run_python 一致的
// AGENT_SKILLS_DIR / PYTHONPATH,否则 housestyle 在后端进程里 import 失败。
// 非空根同时把「lint 必须可运行」设为强约束:跑不起来即 Violations(fail-visible)。
func SetLintEnvRoot(skillsRoot string) {
lintEnvRoot = skillsRoot
}
// Judge 用模型给同一份正文打分。返回错误时调用方回落启发式。
type Judge func(ctx context.Context, text string) (Report, error)
@@ -82,20 +96,27 @@ func ScoreText(text string) Report {
// ScoreFile 读取文件正文再打分。docx 取段落文本;其他文件按 UTF-8 文本读。
// 能跑 housestyle.lint 时,error 级问题会使 Passed 变为 false。
// lint 无法执行时:技能根已注入(SetLintEnvRoot)→ 记 Violations「自检未运行」
// 并判不通过(fail-visible);未注入(测试/无技能部署)→ 保持旧的静默跳过。
func ScoreFile(path string) Report {
text, err := readDeliverable(path)
if err != nil {
return Report{Passed: false, Violations: []string{"无法读取交付物:" + err.Error()}}
}
report := ScoreText(text)
issues := lintFile(path)
issues, lintErr := lintFile(path)
report.Lint = issues
report.LintRan = lintErr == nil
for _, issue := range issues {
if issue.Severity == "error" {
report.Passed = false
report.Violations = append(report.Violations, "版式 "+issue.Rule+":"+issue.Message)
}
}
if lintErr != nil && lintEnvRoot != "" {
report.Passed = false
report.Violations = append(report.Violations, "自检未运行(housestyle.lint 不可用:"+lintErr.Error()+")")
}
return report
}
@@ -176,11 +197,22 @@ func scoreActionable(text string) Dimension {
return Dimension{ID: "可执行性", Score: 35, Note: "没有可交给人的行动", Evidence: snippet(text, 40)}
}
// scoreReadable 只检查真正的标题行(# 开头;docx 路径已把标题样式标成 #)。
// 此前对每一行做「栏目名词 + 短行」判断,表格单元格(如「项目概述撰写」)
// 会被当成标题误报,终轮检查因此对合格文档连续判负(实测死循环源之一)。
// 命中时 Note 带行号,模型能定位到要改的那一行。
func scoreReadable(text string) Dimension {
for _, line := range strings.Split(text, "\n") {
line = strings.TrimSpace(strings.TrimLeft(line, "#"))
if bannedTitle(line) && utf8.RuneCountInString(line) <= 12 {
return Dimension{ID: "可读性", Score: 25, Note: "使用了栏目式标题", Evidence: line}
for i, line := range strings.Split(text, "\n") {
trimmed := strings.TrimSpace(line)
if !strings.HasPrefix(trimmed, "#") {
continue
}
title := strings.TrimSpace(strings.TrimLeft(trimmed, "#"))
if title == "" {
continue
}
if bannedTitle(title) && utf8.RuneCountInString(title) <= 12 {
return Dimension{ID: "可读性", Score: 25, Note: fmt.Sprintf("第 %d 行标题是栏目名,不是判断,改成一句结论", i+1), Evidence: title}
}
}
if strings.Contains(text, "。。") {
@@ -251,6 +283,9 @@ func readDeliverable(path string) (string, error) {
return string(data), nil
}
// readDocxText 抽取 docx 纯文本,并把标题样式(Title / HeadingN)的段落标成
// Markdown # 前缀——评分卡据此区分「标题行」与「正文/表格单元格」,也让
// scoreLead 等维度拿到真实的标题信息。
func readDocxText(path string) (string, error) {
zr, err := zip.OpenReader(path)
if err != nil {
@@ -274,6 +309,22 @@ func readDocxText(path string) (string, error) {
defer rc.Close()
decoder := xml.NewDecoder(rc)
var b strings.Builder
var para strings.Builder
style := ""
inPara := false
flush := func() {
text := strings.TrimSpace(para.String())
if text != "" {
if lvl := headingLevel(style); lvl > 0 {
b.WriteString(strings.Repeat("#", min(lvl, 6)))
b.WriteByte(' ')
}
b.WriteString(text)
b.WriteByte('\n')
}
para.Reset()
style = ""
}
for {
tok, err := decoder.Token()
if err != nil {
@@ -281,23 +332,70 @@ func readDocxText(path string) (string, error) {
}
switch el := tok.(type) {
case xml.StartElement:
if el.Name.Local == "p" && b.Len() > 0 {
b.WriteByte('\n')
switch el.Name.Local {
case "p":
inPara = true
para.Reset()
style = ""
case "pStyle":
if inPara {
for _, a := range el.Attr {
if a.Name.Local == "val" {
style = a.Value
}
}
}
}
case xml.EndElement:
if el.Name.Local == "p" {
inPara = false
flush()
}
case xml.CharData:
b.Write(bytes.TrimSpace(el))
if inPara {
para.Write(bytes.TrimSpace(el))
}
}
}
return b.String(), nil
flush()
return strings.TrimRight(b.String(), "\n"), nil
}
// headingLevel 把 docx styleId 映射为标题层级:Heading1..9 → 1..9,
// Title → 1,Subtitle → 2;其余(含表格单元格正文)返回 0。
func headingLevel(style string) int {
switch {
case style == "Title":
return 1
case style == "Subtitle":
return 2
case strings.HasPrefix(style, "Heading"):
n := 0
for _, c := range style[len("Heading"):] {
if c < '0' || c > '9' {
return 0
}
n = n*10 + int(c-'0')
}
if n >= 1 && n <= 9 {
return n
}
}
return 0
}
var lintJSON = regexp.MustCompile(`(?s)\[.*\]`)
func lintFile(path string) []LintIssue {
// lintFile 运行 housestyle.lint。返回 (issues, nil) 表示 lint 实际执行;
// 非 nil err 表示没有跑起来(解释器缺失、模块不可导入等),调用方决定是否
// 按「自检未运行」暴露。env 与 run_python 的 execEnv 同源语义:继承最小
// 环境并附加 AGENT_SKILLS_DIR / PYTHONPATH,否则后端进程里 import 不了 housestyle。
func lintFile(path string) ([]LintIssue, error) {
cmd := exec.Command("python3", "-m", "housestyle.lint", path)
cmd.Env = lintCommandEnv()
out, err := cmd.CombinedOutput()
if err != nil && !bytes.Contains(out, []byte("\"rule\"")) {
return nil
return nil, fmt.Errorf("%s", firstLine(string(out)))
}
raw := out
if loc := lintJSON.Find(out); loc != nil {
@@ -305,7 +403,91 @@ func lintFile(path string) []LintIssue {
}
var issues []LintIssue
if json.Unmarshal(raw, &issues) != nil {
return nil, fmt.Errorf("lint 输出无法解析:%s", firstLine(string(out)))
}
return issues, nil
}
// lintCommandEnv 继承最小环境(PATH/HOME/LANG 等,与 execEnv 的继承面一致)。
// 技能根注入后附加 AGENT_SKILLS_DIR 与 PYTHONPATH=<root>/house-style;进程已有
// PYTHONPATH 时前置合并(测试与二次包装场景不丢自己的路径)。
// lintEnvRoot 为空时返回 nil(不设 cmd.Env,完整继承,保持旧行为)。
func lintCommandEnv() []string {
if lintEnvRoot == "" {
return nil
}
return issues
var out []string
var existingPath string
for _, kv := range os.Environ() {
if strings.HasPrefix(kv, "PYTHONPATH=") {
existingPath = strings.TrimPrefix(kv, "PYTHONPATH=")
continue
}
for _, keep := range []string{"PATH=", "HOME=", "LANG=", "LC_", "TZ=", "TERM="} {
if strings.HasPrefix(kv, keep) {
out = append(out, kv)
break
}
}
}
pp := filepath.Join(lintEnvRoot, "house-style")
if existingPath != "" {
pp = pp + string(os.PathListSeparator) + existingPath
}
out = append(out,
"AGENT_SKILLS_DIR="+lintEnvRoot,
"PYTHONPATH="+pp,
)
return out
}
func firstLine(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[:i]
}
if utf8.RuneCountInString(s) > 160 {
s = string([]rune(s)[:160]) + "…"
}
return s
}
// HardFacts 只评可机械判定的硬事实:lint 实际运行且零 error。
// 关键词维度(ScoreText 六维)从拦截路径移出(裁定 4/P5):它们描述的是
// 写作质量偏好,实测会把正常中文章节标题判成栏目名、把没有"未指明"字样的
// 行动表判成不可执行,造成终轮死循环;评分整体仍由 ScoreBytes 供评测侧使用。
func HardFacts(name string, content []byte) Report {
report := ScoreBytes(name, content)
report.Passed = true
report.Violations = nil
for _, issue := range report.Lint {
if issue.Severity == "error" {
report.Passed = false
report.Violations = append(report.Violations, "版式 "+issue.Rule+":"+issue.Message)
}
}
if !report.LintRan && lintEnvRoot != "" {
report.Passed = false
report.Violations = append(report.Violations, "自检未运行(housestyle.lint 不可用)")
}
return report
}
// DeliverableText 供事实核查抽取工件文本(docx 与 ScoreBytes 同源口径;
// 二进制返回空)。导出给 runtime 包使用。
func DeliverableText(name string, content []byte) string {
dir, err := os.MkdirTemp("", "factcheck")
if err != nil {
return ""
}
defer os.RemoveAll(dir)
path := filepath.Join(dir, name)
if err := os.WriteFile(path, content, 0o644); err != nil {
return ""
}
text, err := readDeliverable(path)
if err != nil {
return ""
}
return text
}
+132
View File
@@ -0,0 +1,132 @@
package quality
import (
"archive/zip"
"os"
"path/filepath"
"strings"
"testing"
)
func TestLintCommandEnvCarriesSkillsRoot(t *testing.T) {
root := t.TempDir()
SetLintEnvRoot(root)
t.Cleanup(func() { SetLintEnvRoot("") })
env := lintCommandEnv()
var sawSkills, sawPath bool
for _, kv := range env {
if kv == "AGENT_SKILLS_DIR="+root {
sawSkills = true
}
if kv == "PYTHONPATH="+filepath.Join(root, "house-style") {
sawPath = true
}
}
if !sawSkills || !sawPath {
t.Fatalf("lint env missing skills root entries: %v", env)
}
}
// 技能根已注入但 housestyle 不可导入时,评分必须显式报「自检未运行」,
// 不能把没跑当成没问题(实测:后端进程无 PYTHONPATH,lint 静默失效)。
func TestLintUnavailableFailsVisibleWhenSkillsRootSet(t *testing.T) {
SetLintEnvRoot(t.TempDir())
t.Cleanup(func() { SetLintEnvRoot("") })
good := "# 上传文档应先判断相关性再入库\n\n## 先校验,再生成\n- 说话人1:上传后第一步是判断是否与项目相关。\n一句话:没有记忆库,后面的生成和追溯都没有依据。\n\n## 下一步\n- 事项:写出第一版需求。负责人:陈威。期限:周五前;说不清的写未指明。\n"
rep := ScoreBytes("note.md", []byte(good))
if rep.Passed {
t.Fatalf("lint unavailable must fail visibly when skills root is set: %+v", rep)
}
found := false
for _, v := range rep.Violations {
if strings.Contains(v, "自检未运行") {
found = true
}
}
if !found {
t.Fatalf("expected 自检未运行 violation, got: %v", rep.Violations)
}
if rep.LintRan {
t.Fatalf("LintRan should be false when lint could not run")
}
}
// docx 标题样式要被标成 # 前缀;表格单元格/正文不参与「栏目式标题」判定
// (实测误报:单元格「项目概述撰写」被当成标题,检查连续判负)。
func TestDocxHeadingMarkedAndTableCellNotFlagged(t *testing.T) {
docx := buildDocx(t, []paraSpec{
{style: "Heading1", text: "概述"},
{style: "", text: "项目概述撰写"},
{style: "Heading2", text: "先把需求写清楚再开发"},
})
text, err := readDocxText(docx)
if err != nil {
t.Fatalf("readDocxText: %v", err)
}
if !strings.Contains(text, "# 概述") || !strings.Contains(text, "## 先把需求写清楚再开发") {
t.Fatalf("headings not marked with #: %q", text)
}
if !strings.Contains(text, "项目概述撰写") {
t.Fatalf("body paragraph missing: %q", text)
}
// 标题是栏目名 → 仍要被判低分。
withBadHeading := ScoreText(text)
if withBadHeading.Passed {
t.Fatalf("banned heading should fail: %+v", withBadHeading)
}
// 同一个「概述」字样只出现在非标题行时,不应触发栏目式标题。
cellOnly := buildDocx(t, []paraSpec{
{style: "Heading1", text: "先把需求写清楚再开发"},
{style: "", text: "项目概述撰写"},
{style: "", text: "一句话:先写需求。负责人:陈威。期限:周五前。"},
})
cellText, err := readDocxText(cellOnly)
if err != nil {
t.Fatalf("readDocxText: %v", err)
}
for _, d := range ScoreText(cellText).Dimensions {
if d.ID == "可读性" && d.Score < 60 {
t.Fatalf("cell text must not be flagged as column heading: %+v", d)
}
}
}
type paraSpec struct {
style string
text string
}
// buildDocx 构造最小 docx:只含 word/document.xml(readDocxText 的唯一输入)。
func buildDocx(t *testing.T, paras []paraSpec) string {
t.Helper()
var b strings.Builder
b.WriteString(`<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body>`)
for _, p := range paras {
b.WriteString(`<w:p>`)
if p.style != "" {
b.WriteString(`<w:pPr><w:pStyle w:val="` + p.style + `"/></w:pPr>`)
}
b.WriteString(`<w:r><w:t>` + p.text + `</w:t></w:r></w:p>`)
}
b.WriteString(`</w:body></w:document>`)
path := filepath.Join(t.TempDir(), "test.docx")
f, err := os.Create(path)
if err != nil {
t.Fatalf("create docx: %v", err)
}
defer f.Close()
w := zip.NewWriter(f)
entry, err := w.Create("word/document.xml")
if err != nil {
t.Fatalf("zip entry: %v", err)
}
if _, err := entry.Write([]byte(b.String())); err != nil {
t.Fatalf("zip write: %v", err)
}
if err := w.Close(); err != nil {
t.Fatalf("zip close: %v", err)
}
return path
}
+6
View File
@@ -48,6 +48,12 @@ type embeddingsResponse struct {
} `json:"error,omitempty"`
}
// Model 当前 embedding 模型身份(U6-3:写入与检索同源;未配置返回空)。
func (e *HTTPEmbedder) Model() string {
route := e.resolve(context.Background())
return route.Model
}
func (e *HTTPEmbedder) Embed(ctx context.Context, texts []string) ([][]float32, error) {
if len(texts) == 0 {
return nil, nil
+3
View File
@@ -13,6 +13,9 @@ const hashDim = 32
// 由 settings models.embedding.provider=hash 显式打开;未设置时仍按 D23 退化为全文。
type HashEmbedder struct{}
// Model 本地哈希嵌入的身份(U6-3:同源写入/检索,禁止与真实模型混查)。
func (HashEmbedder) Model() string { return "hash:" + fmt.Sprint(hashDim) }
func (HashEmbedder) Embed(_ context.Context, texts []string) ([][]float32, error) {
out := make([][]float32, len(texts))
for i, text := range texts {

Some files were not shown because too many files have changed in this diff Show More