9.3 KiB
原件归属：独立审阅者 /root/fast_slow_final_review；原始记录在
/tmp/fast-slow-u2-u3-20261013/review/。以下为质量判定，未复制完整评测材料或凭据。
Independent verification — explicitly approved extra A01
Reviewer /root/fast_slow_final_review, 2026-10-13. Read-only inspection and fake-provider tests only. No model calls, credentials, source edits or further sample retries. The original six-sample failures remain recorded.
Object, authorization and artifacts
The parent records explicit user approval for exactly one additional A01, at most two paid execution runs, reported-token target60000 and no automatic retry. This reviewer did not infer further authorization from leftover budget.
Sample manifest fixes Backend 31efafa4790843b39ea15c4c0cd99e1495d89199, Dev 316b5a5a56c4cf1201003865448c613cbb2a12e6; binary SHA256 1ce5c7961b3544bfe337d4495f3bdded776d8b4afd71afcbe56a35bbfb6cd2d5. 31efafa's delta from6a2d85c changes only live evaluation fixtures/consumer tests, not production behavior. Original evidence remains in ../model-output/candidate-extra-a01.json and ../approved-extra-a01-data/provider-records.json.
Paid run count and usage — independently reconciled
| Actual RunID | Kind/state | Provider calls | Reported tokens |
|---|---|---|---|
| run_b05e2ef14df2 | conversation/completed | 2 | 10706+11013=21719 |
| run_8cc08e1fb3fe | subagent/failed | 2 | 9362+11122=20484 |
| run_05d9df58ee30 | conversation/completed, test-only notice | 0 | 0 |
Exactly two paid run IDs, four real provider requests, 42203 reported tokens. Provider total equals summed persisted Run.Usage, with usageUnknown=false. The two foreground requests match exact non-process persisted text; the two child requests match tool-call IDs. Mapping is saved in extra-a01-call-run-map.json. Provider fingerprint groups=3 is not a paid-run count.
Every actual request's outputCap is4000; neither child call began with a remaining run allowance below4000. After its second request, reported20484 exceeded the20000 run threshold, so further child requests were denied. This sample does not exhibit the old fixture cap override. No final child answer was produced. Observed extra usage is below its separately approved60000 target by17797; that difference is not new-run authorization.
Original batch211974 + separately authorized extra42203 = 254177 reported tokens across both authorizations. Keep those two budgets distinct; the extra authorization does not rewrite the original240000 batch target.
Checkpoint rejection and source validation
First tool call call_0VotHi8NLwSGVP42wwmeLq2K:
- Valid JSON syntax, invalid candidate schema: findings contains unknown
$text, source fields are misplaced at finding level, one source has no quote, another blockIndex is a string. - Observed result:
json: unknown field "$text"from strict decoding. It never reaches the publishing sink. - This is a schema/model-output failure, not a source-store failure or fabricated success.
Second tool call call_H3g1poeMJTa4bBhzfBRu3S1r:
- Schema nesting/types repaired; same checkpoint ID is legal because the first attempt was rejected and not committed.
- Twelve source links in the candidate were independently matched to host source IDs and source paragraphs. Eleven quotes are literal substrings. One review quote stitches two source portions with inserted
……: “评审意见要求上线七月以留出接口联调时间……外部接口的负责人尚未确认。” The supplied paragraph contains intervening budget text and no such ellipsis, so this is not a literal quote. - Observed result:
checkpoint.source: quote not in provided block. The rejection is correct. Details inextra-a01-rejected-source-checks.json. - Manager validates all candidate sources before committing any finding. This later invalid quote rejects the entire publication, including earlier valid records; no partial leak occurs. Neither call consumes an accepted-checkpoint count.
Persistent transactionality and truthful state
Read the isolated SQLite DB with mode=ro and extracted exact card events to extra-a01-persisted-events.json:
| Seq | Revision | State | Accepted checkpoints/findings |
|---|---|---|---|
| 1 | 1 | queued | 0/0 |
| 3 | 2 | running | 0/0 |
| 418 | 3 | running activity | 0/0 |
| 422 | 4 | running activity | 0/0 |
| 426 | 5 | failed | 0/0 |
| 429 | 6 | failed/reported | 0/0 |
Registration05:09:53.281418215UTC; failed projection05:10:42.845780033UTC, approximately49.56s later. Rejected tool attempts only coexist with real activity updates; there is no findings publication in any persistent revision. Card outcomeCompleteness=none, executionState=failed, phase/focus state “执行已停止.” Exact run error is retained as code run.subagent_failed, message budget_exceeded: run token budget exhausted. Foreground remains completed. No malformed or stitched-quote finding was displayed as an accepted result.
A truthful-copy edge remains on this sample's production SHA: conclusion says “已有发现已保留” and nextStep references existing findings even though there are none; remaining still says “待任务进展后更新” after execution stopped. Main state is honest, but these subordinate strings imply saved findings/future progression. Reported to executor for a host-owned zero-findings branch: state that no accepted phase result exists, clear obsolete process Remaining, and suggest inspecting materials/new request only when desired. Preserve this historical raw sample; do not rewrite it after correction.
The material notice contains the correctly failed summary but retains generic deliverable-selection prompt text. Because the controlled notice model is disabled, no actual notification-model behavior is established by this sample. Do not claim it would correctly offer follow-up actions based on nonexistent results.
Visible answer quality
There is no child final result or accepted checkpoint to grade as useful. The foreground provides factual file overviews and preserves missing approval/date/acceptance information, but repeats known unsupported interpretations: “分期vs一次性” implies the requirements demanded one-phase delivery; “剩余4项（含3项依赖接口）” assigns external dependencies to phase2 without evidence; “后续…需按会议版执行” chooses authority not supplied by the materials. It again duplicates the immediate answer after the stop-hook reminder. Questions later asking whether those assignments are correct do not remove earlier assertions.
The semantic failures in rejected candidate text were not promoted to material records. Literal-source canonicalization is therefore not exercised by a successful actual publication in this extra sample. It remains mechanism-tested, not real-model demonstrated here.
Fixture-only code review and independent replay
31efafa adds explicit authorizedA01 selection/output name, refuses to overwrite an existing completed sample file, and installs limitExtraA01Runs before the shared provider wrapper. The guard reads authoritative persisted run records in the isolated default-owner evaluation space and rejects a third run or query failure before forwarding a paid request. It deliberately does not rely on prompt fingerprints. The zero-token notice follows real work and makes no provider request.
Independent REPLAY on exact31efafa before/after: go test ./internal/server -tags live -run '^(TestExtraA01LimitsActualRunIDs|TestEvaluationPreservesProductionOutputCap)$' -count=1 -v — exit0. Raw output extra-a01-guard-replay.log. These tests use fake streams/httptest, not paid models. The third-run guard and output-cap780 preservation both pass.
Incoming settings/SSE compatibility notes
Read upstream Core0f94e4b and Backend741da4a changes without altering refs. Effective settings now merge persisted system scope and strip managed user/project overrides. Models and runtime.mock become system-managed; materialIntake, materialIntakeChars and context.maxAttachmentChars remain ordinary scoped values. U2 material hashing already consumes effective attachment cap. Current eval fixture's trusted model configuration must be seeded through system override/system scope after merge rather than now-rejected MergeUser. No route credential was read or printed.
Merge must retain TaskSnapshot/DocumentOrigin while adding SettingDesc.Scope, and retain source visibility + subscribe-before-ready alongside upstream SSE session rechecks. These are compatibility observations, not a completed review/replay of the not-yet-merged composition. No broad audit of the upstream settings feature is claimed.
Verdict / declined scope
Extra A01: FAIL. Zero accepted useful checkpoints, no child final answer, child ended on budget. Strict argument/source rejection, whole-publication consistency, run isolation/count and observed usage accounting behave as expected; those safety properties do not satisfy output quality acceptance. The two-run/60000 observed limit is met, not a reason for further retries.
Overall U3 quality gate remains not passed. No more paid samples are authorized in this review. Declined: monetary billing (not provided); normal notification-model quality (suppressed); new merged component/runtime behavior (await exact composition); final platform CI/merge (executor gate). Retain the failure as concrete, reviewable evidence.