merge: candidate jev-dev-assist-on into local_dev

This commit is contained in:
weichen
2026-09-23 18:02:02 +00:00
7 changed files with 648 additions and 3 deletions
+1
View File
@@ -18,6 +18,7 @@
| owner | task | branch | worktree | mutex | status |
|---|---|---|---|---|---|
| Cursor | integrate-jev-assist-land-2026-09-24 | feature/integrate-jev-assist-land-2026-09-24 | .worktrees/integrate-jev-assist-land-2026-09-24 | git:dev | done |
| Cursor | integrate-codegraph-fixup2-2026-09-24 | feature/integrate-codegraph-fixup2-2026-09-24 | (removed) | git:dev | done |
| Cursor | integrate-codegraph-governance-2026-09-24 | feature/integrate-codegraph-governance-2026-09-24 | (removed) | git:dev | done |
| Cursor | codegraph-governance-r2-2026-09-24 | feature/codegraph-governance-2026-09-24 | .worktrees/codegraph-governance-2026-09-24 | agent-workflow-governance+codegraph | done |
@@ -0,0 +1,29 @@
---
status: done
write_scope: configs/.env.dev(local),env:dev,.doc
mutex: env:dev+jevdecide
base_dev_sha: 156c1130
review_state: self-reviewed
integration_state: docs_pending
deployment: local_dev_assist_on
cleanup: pending
remote_sync: not_pushed
owner: Cursor
task_id: jev-dev-assist-on-2026-09-24
---
# local_dev 启用 jev-service(assist)并多样本对话验收
## 目标
将 `JEV_DECIDE_MODE` 从 shadow 升为 **assist**,用 ≥20 条真实 `/api/v1/chat` 验证:规则漏检时能否加成打开实时检索,且业务问句不被误伤。
## 做法
1. `configs/.env.dev`:`JEV_DECIDE_MODE=assist`(URL/KEY 不变)
2. `env:dev` 租约下 force-recreate app → 日志 `jevdecide wired mode=assist`
3. 20 轮真实对话;证据 [assets](../../test-reports/assets/2026-09-24-jev-dev-assist-enable/);报告 [test-report](../../test-reports/2026-09-24-jev-dev-assist-enable-test-report.md)
## 回退
`JEV_DECIDE_MODE=shadow` 或 `off` 后 recreate app。
+1 -1
View File
@@ -49,7 +49,7 @@
| record_id | 模块 | 声明范围 | 状态 | 发布关系 | 证据 | 剩余工作 / 关闭条件 | mutex |
|------|----------|------|----------|------|---------------------|-------|-------|
| jevdecide-shadow | jev 产品决策 shadow | jevdecide 客户端 + coordinator realtime/ProfileTask shadow;compose 默认 off;**local_dev 已开 shadow**(`configs/.env.dev`,容器经 172.17.0.1:8767);assist 未放量 | partial | deferred_enhancement | [产品报告](../test-reports/2026-09-24-jev-product-shadow-test-report.md);[启用报告](../test-reports/2026-09-24-jev-dev-shadow-enable-test-report.md);[启用短文](../ops/short-term/2026-09-24-jev-dev-shadow-enable.md) | 关闭条件:真实轮浸泡≥200 + 用户授权 assist;intent/risk 另立任务。本行不升 done | `jevdecide` |
| jevdecide-shadow | jev 产品决策 shadow/assist | jevdecide + coordinator realtime/ProfileTask;compose 默认 off;**local_dev 已开 assist**(`configs/.env.dev` → 172.17.0.1:8767);决策层漏检加成已对话验收 | partial | deferred_enhancement | [产品报告](../test-reports/2026-09-24-jev-product-shadow-test-report.md);[shadow 启用](../test-reports/2026-09-24-jev-dev-shadow-enable-test-report.md);[assist 启用+对话](../test-reports/2026-09-24-jev-dev-assist-enable-test-report.md);[assist 短文](../ops/short-term/2026-09-24-jev-dev-assist-enable.md) | 关闭条件:真实轮浸泡≥200 + 体验层检索空结果收敛;intent/risk 另立。本行不升 done | `jevdecide` |
| chat-semantic-replace | 语义摘要 REPLACE | 以 shadow 叙事可选替换 served deterministic Narrative(fail-closed 灰度) | deferred | deferred_enhancement | [Wave5 X-2](../ops/short-term/2026-09-18-wave5-deadcode-decisions.md);[pu6](../ops/long-term/2026-09-16-pu6-semantic-summary-contract.md);[summary_flags](../../internal/ctxmgr/summary_flags.go) | 关闭条件:用户独立授权 + fail-closed 灰度设计;`SEMANTIC_SUMMARY_REPLACE` 当前密封不接线。本行不自动升 done | `chat` |
| gov-agent-worktree | Agent worktree 治理自动化 | 主目录保护、任务 worktree、机器认领、候选集成、dev 部署与安全清理 | partial | non_blocker | [方案](../remediation/2026-09-21-agent-workflow-governance-plan.md);[L1 验收](../test-reports/2026-09-21-agent-governance-test-report.md);[W1-B macOS](../ops/short-term/2026-09-22-agentctl-macos.md);[W4](../ops/short-term/2026-09-22-governance-automation.md);residual `modular-decouple-residual-2026-09-23` | L1 与 macOS `with/docs apply/validate/integrate` 已合入。W4 landed:durable queue + 完整 docs projection apply(行/整文 CAS)+ deploy/recover 真实收据闭环 + Darwin cwd 可见性测试。仍 deferred:PI 源码镜像 compose、跨机隔离、automation/w4 实现收敛。本行不升 done | `agent-workflow-governance` |
| intent-l1 | 意图识别 L1 | 18 类 Intent、三分支消解、写/只读分流、双标签 `DOC_BUSY`、intent 纯依赖边界 | done | non_blocker | [intent-recognition](../system-design/intent-recognition/);[工作流作用域](../test-reports/2026-09-21-intent-workflow-scoping-test-report.md);[审计](../audit/2026-09-13-conversation-intent-runtime-state.md);[报告](../test-reports/2026-09-13-conversation-intent-runtime-state-test-report.md);[双标签](../test-reports/assets/2026-09-15-conversation-intent-dual-tab/live.json);[U1](../test-reports/2026-09-15-multi-intent-u1-contracts-test-report.md)–[U3](../test-reports/2026-09-15-multi-intent-u3-intentflow-pg-test-report.md) | **边界 before**:含 U9–U12 叙事混入 → **after**:U9–U12 迁至「多意图扩面」行。关闭:纯依赖边界回归即重开 | `intent` |
@@ -0,0 +1,109 @@
# local_dev jev-service assist 启用与对话加成 测试报告
## 一、 测试背景 (Background)
shadow 已在 local_dev 观察一致率。用户授权启用 **assist**,并用多样本真实对话检验:规则词典漏检的时效问句能否被 jev 高置信加成打开实时检索,同时业务问句不被误开。
## 二、 测试目标 (Test Objectives)
- 启动接线 `mode=assist`
- 20 条真实 chat:A 规则命中基线 / B 漏检候选 / C 应保持非实时
- 决策层:`rule=false ∧ jev=true ∧ mode=assist` 计为加成信号
- 体验层:回复是否出现可核验的实时信息(气温/汇率/AQI/金价等)
- C 组零错误加成
## 三、 测试方案与环境 (Test Plan & Environment)
- OS:Linux;`shuimu-agent-app-dev`;jev `172.17.0.1:8767`;`JEV_DECIDE_MODE=assist`
- 层级:真实 Agent(pi-agent + 实时检索)端到端对话
- 样本矩阵:A×4 + B×10 + C×6(见 assets `batch-results.json`)
- **全量 `go test ./...`:NOT REQUIRED**(运维开关 + 对话验收;产品单元/soak 已有既有报告)
## 四、 预期结果 (Expected Results)
| 组 | 预期 |
|----|------|
| A | rule=true,jev≈true,agree;行为与 shadow 时一致(assist 不压低规则) |
| B | 多数 rule=false、jev=true → 决策加成;部分回复含实时事实 |
| C | rule=false、jev=false,无 assist 加成 |
## 五、 实际测试结果与判定 (Actual Results & Evaluation)
### 5.1 启动接线
- **输入**:`.env.dev` → assist;force-recreate app
- **实际输出**:`msg="jevdecide wired" mode=assist endpoint=http://172.17.0.1:8767`
- **判定**:PASS
### 5.2 批量对话汇总(20/20 HTTP 200)
证据:[`assets/2026-09-24-jev-dev-assist-enable/batch-summary.json`](./assets/2026-09-24-jev-dev-assist-enable/batch-summary.json)
| 指标 | 结果 | 判定 |
|------|------|------|
| 决策层加成(B) | **10/10**(B1–B10 全部 `rule=false jev=true`) | PASS |
| A 组 agree | **4/4** | PASS |
| C 组错误加成 | **0/6** | PASS |
| UX 可见实时事实 | B3/B4/B6/B7/B8 明确;B9 部分;B1/B2/B5/B10 仍偏项目话术 | 决策 PASS;体验 **部分 PASS** |
### 5.3 典型用例
#### A1 规则基线
- **输入**:`今天上海天气怎么样`
- **日志**:`mode=assist rule=true jev=true confidence=0.997951 agree=true`
- **判定**:PASS(assist 未破坏规则命中)
#### B1 决策加成(体验弱)
- **输入**:`外面冷不冷啊今天`
- **日志**:`mode=assist rule=false jev=true confidence=0.990937 agree=false`
- **回复**:偏项目 V0.5 话术(检索可能空/模型未用 snippet)
- **判定**:决策 PASS;体验弱
#### B3 决策+体验加成
- **输入**:`上海现在热不热`
- **日志**:`rule=false jev=true confidence=0.997512`
- **回复**:给出约 29°C、湿度等实时表述
- **判定**:PASS(整链加成可见)
#### B4 汇率漏检加成
- **输入**:`美元兑人民币现在多少了`(无「汇率」词典字)
- **日志**:`rule=false jev=true`
- **回复**:给出约 6.697 汇率
- **判定**:PASS
#### B6 英文天气
- **输入**:`What's the weather like in Beijing today`
- **日志**:`rule=false jev=true`(中文 marker 全漏)
- **回复**:北京气温 24.6°C 等
- **判定**:PASS(对英文问句加成明显)
#### C2 业务防误伤
- **输入**:`型检温度指标怎么定`(含「温度」但强业务词)
- **日志**:`rule=false jev=false confidence=0.994535 agree=true`
- **判定**:PASS(未错误加成)
### 5.4 耗时
单轮 chat 约 6–29s(含 pi-agent / 检索);jev decide 本身亚秒级,不主导端到端。
## 六、 自动化测试套件汇总 (Test Suite Execution)
NOT REQUIRED(运维启用 + 真实对话批次)。不将本报告记为产品全量回归 PASS。
## 七、 测试结论与后续优化建议 (Conclusion & Next Steps)
**结论**:assist 在决策层对规则漏检的时效问句有明确加成(B 组 10/10),且未误伤 C 组业务问句;A 组规则路径保持。端到端体验上约半数 B 样本能把实时事实写进回复,其余仍受检索空结果/对话模板牵引限制——**加成在「是否打开检索」上已成立,在「答复一定用上检索」上仍有缺口**。
**建议**:
1. 保持 local_dev assist;继续统计 AssistHit / 检索空结果率
2. 检索失败时对话提示可更明确,避免回落纯项目话术
3. compose 默认仍 `off`;test/main 不放量直至浸泡与体验稳定
回退:`JEV_DECIDE_MODE=shadow|off` + recreate app。
@@ -0,0 +1,402 @@
[
{
"id": "A1",
"bucket": "rule_true",
"input": "今天上海天气怎么样",
"rule_oracle": true,
"http": 200,
"err": null,
"duration_s": 11.74,
"reply_prefix": "好的,今天(9月24日)上海天气**小雨转多云**,气温约24~29℃,东风微风。 出门记得带把伞~ 如有其他需要随时说。",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:53:10.890Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=true jev=true confidence=0.997951 margin=0.99590182 agree=true",
"mode": "assist",
"rule": "true",
"jev": "true",
"confidence": "0.997951",
"margin": "0.99590182",
"agree": "true",
"assist_additive_signal": false,
"log_profile": "time=2026-09-23T17:53:22.464Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.990003 agree=true"
},
{
"id": "A2",
"bucket": "rule_true",
"input": "美元汇率多少",
"rule_oracle": true,
"http": 200,
"err": null,
"duration_s": 13.46,
"reply_prefix": "根据实时数据,当前(2026年9月23日)美元兑人民币汇率约为 **1 USD = 6.6977 CNY**(买入价/卖出价约 6.6975 / 6.6978)。 供参考,当前项目「医疗器械研发项目」处于 V0.1 摸底期,若 BOM 中涉及进口光学/电气元器件,汇率波动会直接影响成本边界,建议在预算评估时预留 1%~2% 的汇率缓冲空间。 如需我帮您按",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:53:22.966Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=true jev=true confidence=0.998322 margin=0.9966430399999999 agree=true",
"mode": "assist",
"rule": "true",
"jev": "true",
"confidence": "0.998322",
"margin": "0.9966430399999999",
"agree": "true",
"assist_additive_signal": false,
"log_profile": "time=2026-09-23T17:53:36.239Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.973338 agree=true"
},
{
"id": "A3",
"bucket": "rule_true",
"input": "今天有什么新闻",
"rule_oracle": true,
"http": 200,
"err": null,
"duration_s": 16.71,
"reply_prefix": "好的,为您梳理一下今天的要点新闻: **【国际热点】** 1. **川习会**:习近平主席9月23-25日访美,9月24日将访白宫与特朗普会面并出席国宴,双方就贸易及地区冲突议题展开磋商。 2. **霍尔木兹海峡局势**:一艘油轮21日遭袭、两名水手受伤;美军称近两周该海峡能源运输量达近六个月新高。 **【科技/商业】** 3. **AI竞争**:Ant",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:53:37.315Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=true jev=true confidence=0.996222 margin=0.99244306 agree=true",
"mode": "assist",
"rule": "true",
"jev": "true",
"confidence": "0.996222",
"margin": "0.99244306",
"agree": "true",
"assist_additive_signal": false,
"log_profile": "time=2026-09-23T17:53:53.269Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.980692 agree=true"
},
{
"id": "A4",
"bucket": "rule_true",
"input": "明天会下雨吗",
"rule_oracle": true,
"http": 200,
"err": null,
"duration_s": 15.79,
"reply_prefix": "您好! 「明天」是否下雨取决于您所在的具体城市——检索到的实时预报显示,不同地区差异较大: - **大阪(大阪府)**:明天(9/25,周五)晴时多云,降雨概率 0%~20%,基本不下雨。 - **新加坡**:明天小雨,气温 29℃/25℃。 - **纽约**:明天有较高降雨概率。 - **香港、广州**:需查询具体地点预报。 请问您关心的是哪个城市或",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:53:53.765Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=true jev=true confidence=0.997117 margin=0.9942339200000001 agree=true",
"mode": "assist",
"rule": "true",
"jev": "true",
"confidence": "0.997117",
"margin": "0.9942339200000001",
"agree": "true",
"assist_additive_signal": false,
"log_profile": "time=2026-09-23T17:54:09.370Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.986274 agree=true"
},
{
"id": "B1",
"bucket": "assist_candidate",
"input": "外面冷不冷啊今天",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 13.81,
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册分类路径,并预估型检成本,便于与客户深入沟通; • 若先放一放 / 多做调研: 可先补齐预算、目标市场或使用场景等信息,降低后续返工风险。 💡 如果您接下来更希望先有一份可审阅的材料,我可以先整",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:54:11.130Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.990937 margin=0.9818732400000001 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.990937",
"margin": "0.9818732400000001",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:54:23.498Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.996736 agree=true"
},
{
"id": "B2",
"bucket": "assist_candidate",
"input": "明天出门要不要带伞",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 13.98,
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册分类路径,并预估型检成本,便于与客户深入沟通; • 若先放一放 / 多做调研: 可先补齐预算、目标市场或使用场景等信息,降低后续返工风险。 💡 如果您接下来更希望先有一份可审阅的材料,我可以先整",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:54:25.319Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.996626 margin=0.99325288 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.996626",
"margin": "0.99325288",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:54:37.765Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.967817 agree=true"
},
{
"id": "B3",
"bucket": "assist_candidate",
"input": "上海现在热不热",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 15.75,
"reply_prefix": "上海现在还挺热的~ 🌡️ 据实时数据显示,上海当前气温约 **29°C** 左右,天气多云,湿度较高(约80%),紫外线指数较强。夜间虽然会凉快一些,但整体仍维持夏季炎热的体感。 建议出行时做好防暑防晒,适当补水。 --- *注:以上天气信息来源于实时网络查询,供您参考。如您有其他问题需要协助,随时告诉我~*",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:54:39.587Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.997512 margin=0.99502314 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.997512",
"margin": "0.99502314",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:54:53.857Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.98938 agree=true"
},
{
"id": "B4",
"bucket": "assist_candidate",
"input": "美元兑人民币现在多少了",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 11.06,
"reply_prefix": "好的,帮您查了一下—— **截至2026年9月22日,美元兑人民币汇率约 6.697。** 近期人民币有所走强,若您后续涉及进口元器件或海外采购预算,这算是个相对友好的窗口。 如果您接下来需要做BOM成本换算或预算边界梳理,我可以帮您一起核算。",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:54:54.433Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.998392 margin=0.99678406 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.998392",
"margin": "0.99678406",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:55:05.237Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.980923 agree=true"
},
{
"id": "B5",
"bucket": "assist_candidate",
"input": "苹果股票今天表现如何",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 15.0,
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册分类路径,并预估型检成本,便于与客户深入沟通; • 若先放一放 / 多做调研: 可先补齐预算、目标市场或使用场景等信息,降低后续返工风险。 💡 如果您接下来更希望先有一份可审阅的材料,我可以先整",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:55:07.072Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.998564 margin=0.99712808 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.998564",
"margin": "0.99712808",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:55:20.555Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.97285 agree=true"
},
{
"id": "B6",
"bucket": "assist_candidate",
"input": "What's the weather like in Beijing today",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 14.14,
"reply_prefix": "北京今日天气(2026年9月24日凌晨): - 🌡️ 气温:**24.6°C** - 🌫️ 天气:烟霾 + 部分多云 - 💨 风向:东北风 - 🌧️ 降雨概率:低 - 🌬️ 空气质量:中等 如您接下来有项目推进或文档整理方面的需求,随时告诉我~",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:55:21.773Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.998316 margin=0.99663272 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.998316",
"margin": "0.99663272",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:55:35.016Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.984659 agree=true"
},
{
"id": "B7",
"bucket": "assist_candidate",
"input": "黄金现在贵不贵",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 18.83,
"reply_prefix": "**结论:贵,且处于历史高位区间。** 根据实时数据(2026-09-23): | 指标 | 价格 | |------|------| | 国际黄金现货 | **$4,333.25 / 盎司** | | 折合人民币 | 约 **¥934 / 克** | | 上海黄金现货 | 约 **¥936 / 克** | **参考背景:** - 年内峰值 $5,60",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:55:36.251Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.99545 margin=0.9908997799999999 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.99545",
"margin": "0.9908997799999999",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:55:54.154Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.964275 agree=true"
},
{
"id": "B8",
"bucket": "assist_candidate",
"input": "北京空气质量怎么样最近",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 13.45,
"reply_prefix": "好的,帮您查了一下北京最近的空气质量: **截至 2026-09-23**,北京 AQI 约 **109**,处于**轻度污染**级别。PM2.5 浓度超出世卫组织指导值,对敏感人群可能有一定影响。 **小建议**:如果您或团队成员对空气质量比较敏感,建议减少长时间、高强度的户外活动,适当做好防护。 --- 这个时间点咱们项目组应该都在休息了吧?有什",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:55:54.702Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.997111 margin=0.9942215000000001 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.997111",
"margin": "0.9942215000000001",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:56:07.921Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.948139 agree=true"
},
{
"id": "B9",
"bucket": "assist_candidate",
"input": "今晚适合户外跑步吗",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 17.21,
"reply_prefix": "好的,我帮您查了一下—— 当前时间是**凌晨1:56**,严格来说还没到\"今晚\"呢,是今晨 :) 如果您指的是**夜间跑步**的话,依据检索到的建议:夜跑可选,但建议注意以下几点: 1. **时间选择**:黄昏(17:00~19:00)人体运动状态最佳,氧气浓度高、感觉灵敏;若只能在深夜跑,建议选择照明充足的场所,避免运动损伤。 2. **空气质量**",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:56:08.454Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.992417 margin=0.98483312 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.992417",
"margin": "0.98483312",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:56:25.451Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.964401 agree=true"
},
{
"id": "B10",
"bucket": "assist_candidate",
"input": "现在要穿羽绒服吗上海",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 17.08,
"reply_prefix": "您好,用户!我是您的项目助手,很高兴协助您推进医疗器械立项与研发协同。 针对「医疗器械研发项目」,我可以帮您: • 查询进展、成熟度与风险; • 沉淀客户需求与技术指标; • 起草沟通话术与方案要点; • 推进方案论证与报价核算。 如果您愿意,我们可以先从您当前最关心的一块开始——您想先看进展、补事实,还是直接聊方案?",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:56:26.005Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=true confidence=0.987759 margin=0.9755179199999999 agree=false",
"mode": "assist",
"rule": "false",
"jev": "true",
"confidence": "0.987759",
"margin": "0.9755179199999999",
"agree": "false",
"assist_additive_signal": true,
"log_profile": "time=2026-09-23T17:56:42.849Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.958358 agree=true"
},
{
"id": "C1",
"bucket": "should_false",
"input": "项目进度怎么样",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 11.87,
"reply_prefix": "您好!帮您梳理一下当前项目进度: **项目状态概览** - **阶段**:V0.1(初始摸底期) - **成熟度**:65/100 - **工作流**:intake(信息采集/补充阶段) - **进展**:基准产品画像已梳理完成 **当前待办** | 槽位 | 状态 | 说明 | |------|------|------| | 设备类型(devic",
"realtime_note": "reply_suggests_live_or_search",
"log_realtime": "time=2026-09-23T17:56:44.121Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.987928 margin=0.97585502 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.987928",
"margin": "0.97585502",
"agree": "true",
"assist_additive_signal": false,
"log_profile": ""
},
{
"id": "C2",
"bucket": "should_false",
"input": "型检温度指标怎么定",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 29.19,
"reply_prefix": "您好!型检温度指标的确定通常遵循以下逻辑: ## 核心依据 1. **强标 GB 9706.1-2020** 〔引用键〕——有源医械的通用安全要求,规定了工作气候条件(通常 +10℃~+40℃)和贮存气候条件(通常 -40℃~+70℃)。 2. **GB/T 14710-2009《医用电器环境要求及试验方法》** 〔引用键〕——明确温度试验分组(Ⅰ/Ⅱ",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:56:56.232Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.994535 margin=0.98906928 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.994535",
"margin": "0.98906928",
"agree": "true",
"assist_additive_signal": false,
"log_profile": ""
},
{
"id": "C3",
"bucket": "should_false",
"input": "帮我推进 V0.5 方案",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 26.67,
"reply_prefix": "好的,我来帮您梳理一下推进V0.5方案的情况。 根据项目当前状态,V0.5方案推进需要关注以下几点: **当前阶段分析:** - 项目处于**V0.1阶段**(intake工作流) - 基准产品画像已梳理完成 ✓ - 成熟度:65/100 **推进到V0.5需要的关键信息:** | 槽位 | 状态 | 说明 | |------|------|----",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:57:25.749Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.994602 margin=0.98920304 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.994602",
"margin": "0.98920304",
"agree": "true",
"assist_additive_signal": false,
"log_profile": ""
},
{
"id": "C4",
"bucket": "should_false",
"input": "注册路径需要注意什么",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 20.98,
"reply_prefix": "您好!项目「医疗器械研发项目」当前处于【V0.1】阶段(成熟度 65/100)。 核心研判:已完成「医疗器械研发项目」的初始事实沉淀,当前具备进入方案与法规初筛的基准条件 ⚠️ 关键风险预警: • 若客户宣称中包含实质性疗效或治疗作用,可能触发器械分类升档(如二类转三类),研发与注册周期将增加 6-12 个月 • 时间与预算尚缺客户书面口径,直接对",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:57:52.630Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.998821 margin=0.9976410200000001 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.998821",
"margin": "0.9976410200000001",
"agree": "true",
"assist_additive_signal": false,
"log_profile": ""
},
{
"id": "C5",
"bucket": "should_false",
"input": "你好",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 6.21,
"reply_prefix": "您好,用户!我是您的项目助手,很高兴协助您推进医疗器械立项与研发协同。 针对「医疗器械研发项目」,我可以帮您: • 查询进展、成熟度与风险; • 沉淀客户需求与技术指标; • 起草沟通话术与方案要点; • 推进方案论证与报价核算。 如果您愿意,我们可以先从您当前最关心的一块开始——您想先看进展、补事实,还是直接聊方案?",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:58:13.380Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.998913 margin=0.9978259 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.998913",
"margin": "0.9978259",
"agree": "true",
"assist_additive_signal": false,
"log_profile": "time=2026-09-23T17:58:19.348Z level=INFO msg=\"jevdecide profile shadow\" mode=assist rule_bucket=low jev_bucket=low score=0 confidence=0.99835 agree=true"
},
{
"id": "C6",
"bucket": "should_false",
"input": "这份报价方案文档写得怎么样",
"rule_oracle": false,
"http": 200,
"err": null,
"duration_s": 13.07,
"reply_prefix": "用户,您好!针对「肌骨超声AI辅助诊断系统」的系统验证方案(SVP)与型式检验要求,我为您梳理以下核心难点项目与特别注意事项: ### 一、 核心测试难点项目 1. **声学输出安全与探头表面温升极限(GB 9706.237-2020)** • **难点分析**:肌骨超声常需配合高频线阵探头(10-18MHz)进行浅表滑膜微血流彩色多普勒(CDFI",
"realtime_note": "",
"log_realtime": "time=2026-09-23T17:58:20.485Z level=INFO msg=\"jevdecide realtime shadow\" mode=assist rule=false jev=false confidence=0.995574 margin=0.99114838 agree=true",
"mode": "assist",
"rule": "false",
"jev": "false",
"confidence": "0.995574",
"margin": "0.99114838",
"agree": "true",
"assist_additive_signal": false,
"log_profile": ""
}
]
@@ -0,0 +1,104 @@
{
"n": 20,
"http_ok": 20,
"assist_additive_hits": 10,
"additive_ids": [
"B1",
"B2",
"B3",
"B4",
"B5",
"B6",
"B7",
"B8",
"B9",
"B10"
],
"additive_inputs": [
{
"id": "B1",
"input": "外面冷不冷啊今天",
"confidence": "0.990937",
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册"
},
{
"id": "B2",
"input": "明天出门要不要带伞",
"confidence": "0.996626",
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册"
},
{
"id": "B3",
"input": "上海现在热不热",
"confidence": "0.997512",
"reply_prefix": "上海现在还挺热的~ 🌡️ 据实时数据显示,上海当前气温约 **29°C** 左右,天气多云,湿度较高(约80%),紫外线指数较强。夜间虽然会凉快一些,但整体仍"
},
{
"id": "B4",
"input": "美元兑人民币现在多少了",
"confidence": "0.998392",
"reply_prefix": "好的,帮您查了一下—— **截至2026年9月22日,美元兑人民币汇率约 6.697。** 近期人民币有所走强,若您后续涉及进口元器件或海外采购预算,这算是"
},
{
"id": "B5",
"input": "苹果股票今天表现如何",
"confidence": "0.998564",
"reply_prefix": "完全理解您想再掂量一下。针对「医疗器械研发项目」目前的节点,我帮您快速对照一下: • 若推进 V0.5(方案成型与法规对齐): 我可帮您梳理初步架构与注册"
},
{
"id": "B6",
"input": "What's the weather like in Beijing today",
"confidence": "0.998316",
"reply_prefix": "北京今日天气(2026年9月24日凌晨): - 🌡️ 气温:**24.6°C** - 🌫️ 天气:烟霾 + 部分多云 - 💨 风向:东北风 - 🌧️ 降雨概率"
},
{
"id": "B7",
"input": "黄金现在贵不贵",
"confidence": "0.99545",
"reply_prefix": "**结论:贵,且处于历史高位区间。** 根据实时数据(2026-09-23): | 指标 | 价格 | |------|------| | 国际黄金现货 |"
},
{
"id": "B8",
"input": "北京空气质量怎么样最近",
"confidence": "0.997111",
"reply_prefix": "好的,帮您查了一下北京最近的空气质量: **截至 2026-09-23**,北京 AQI 约 **109**,处于**轻度污染**级别。PM2.5 浓度超出世"
},
{
"id": "B9",
"input": "今晚适合户外跑步吗",
"confidence": "0.992417",
"reply_prefix": "好的,我帮您查了一下—— 当前时间是**凌晨1:56**,严格来说还没到\"今晚\"呢,是今晨 :) 如果您指的是**夜间跑步**的话,依据检索到的建议:夜跑可"
},
{
"id": "B10",
"input": "现在要穿羽绒服吗上海",
"confidence": "0.987759",
"reply_prefix": "您好,用户!我是您的项目助手,很高兴协助您推进医疗器械立项与研发协同。 针对「医疗器械研发项目」,我可以帮您: • 查询进展、成熟度与风险; • 沉淀客户需求"
}
],
"agree": 10,
"disagree": 10,
"missing_log": [],
"should_false_wrong_assist": [],
"B_hit_rate": 1.0,
"failures": [],
"ux_visible_live_ids": [
"B3",
"B4",
"B6",
"B7",
"B8"
],
"ux_partial_search_ids": [
"B9"
],
"ux_weak_project_fallback_ids": [
"B1",
"B2",
"B5",
"B10"
],
"decision_layer_additive_rate": "10/10 B candidates",
"safety_false_assist_on_C": "0/6",
"baseline_A_agree": "4/4"
}
+2 -2
View File
@@ -113,8 +113,8 @@ INTENT_V2_DECISION=false
INTENT_V2_DECISION_TIER=none
# jev-service typed decide (product shadow/assist; default off)
# local_dev enable: set JEV_DECIDE_MODE=shadow in configs/.env.dev (gitignored) + rebuild app.
# From Docker app → host jev: use http://172.17.0.1:8767 (host.docker.internal may be unreachable).
# local_dev: set JEV_DECIDE_MODE=shadow|assist in configs/.env.dev (gitignored) + recreate/rebuild app.
# assist = rule || high-conf jev yes (additive recall only). From Docker → host: http://172.17.0.1:8767.
JEV_DECIDE_MODE=off
JEV_BASE_URL=http://127.0.0.1:8767
JEV_API_KEY=