Agent-System-Dev/docs/superpowers/plans/2026-10-18-regulations-knowledge-base-plan.md

title, date, type, artifact_contract, product_contract_source, execution, origin

title	date	type	artifact_contract	product_contract_source	execution	origin
feat: Build a traceable medical-device regulations knowledge base	2026-10-18	feat	ce-unified-plan/v1	ce-plan-bootstrap	code	../Agent-System-RDM/docs/requirements/01-用户需求与系统需求.md, ../Agent-System-RDM/docs/requirements/02-产品概念与形态定义.md, ../Agent-System-RDM/docs/requirements/03-产品设计方案.md

Build a Traceable Medical-Device Regulations Knowledge Base

Goal Capsule

• Objective: RDM users can rely on relevant medical-device regulations, standards, and review guidance with precise, reproducible citations in conversations and documents, backed by complete audit evidence.
• Means: Create an independently versioned regulations knowledge component and integrate it with RDM's mandatory knowledge-base plus real-web-search verification flow (KTD1, KTD2).
• Authority: Source documents and issuing authorities govern legal text and status; the versioned corpus governs the text indexed for knowledge-base retrieval; verified network captures provide the second source; RDM project/resource authorization governs access to project work and audit data.
• Stop conditions: Do not publish uncertain or unlicensed corpus material; do not report dual-source verification when either channel, version mapping, or excerpt comparison is incomplete; stop on access-control failure or an unreconciled source conflict.
• Execution profile: Cross-repository work in Agent-System-KB-Regulations, Agent-System-RDM, and the base repository when a public shared-regulations retrieval or real-web-search extension is required.
• Finishing responsibility: The implementation owner completes component tests, RDM integration, exact-composition verification, review, and authorized release steps. Runtime deployment and business acceptance remain separate from this plan.

---

Product Contract

Summary

Build a controlled, versioned medical-device regulations knowledge base that RDM can query alongside a real network search. Every generated regulatory claim must carry structured citations that resolve to exact source text, clause location, edition, verification status, and immutable audit evidence.

Problem Frame

RDM already models a regulatory dual-channel check, but its current network leg is based on model-proposed URLs and excerpts followed by fetching and excerpt matching. This does not establish that an actual search occurred. Precise citations also require more than vector retrieval: source editions, clause hierarchy, paragraph anchors, parsed text, and exact evidence must survive re-indexing and later audit.

The current requirements already call for local regulations knowledge, knowledge-base plus network cross-checking, citations, and project isolation (UR-14, SR-F3, SR-N4; see origin documents above). The regulations directory currently exposes a manifest and a PostgreSQL dump; the manifest reports 2,907 documents, 312,919 chunks, and 279,184 nodes. These are inventory claims, not validation of the dump contents, source-file availability, legal status, permissions, or parsing accuracy. Those must be verified before corpus publication.

Requirements

• R1. Regulations, standards, and review guidance relevant to RDM work must use both the controlled regulations knowledge base and a real network search. The system must distinguish actual search from model-generated URLs or recollection.
• R2. Every regulatory citation must resolve to a specific regulation identity and edition, clause path, exact excerpt, source locator, and verification batch. The system must separately check whether the cited excerpt supports the generated claim; authentic text and matching dual sources alone do not establish claim support or legal applicability. Unsupported/uncertain claims must be marked unverified or routed for review. Generated summaries may aid retrieval but must never be presented or audited as legal source text.
• R3. The corpus must preserve original assets or an authorized immutable source representation, extracted text, hierarchy, source provenance, effective status, and content hashes independently from rebuildable search indexes.
• R4. Ingestion must preserve meaningful legal structure: regulation, part/chapter/section, clause, subclause/item, paragraph, list, and table relationships when present. Retrieval chunks must retain heading context and map back to one or more exact source nodes.
• R5. Parsing, normalization, chunking, embedding, and ranking versions must be recorded. Re-indexing must not rewrite evidence used by prior answers, documents, or audits.
• R6. A dual-source result is verified only when both sources are fetched or retrieved from pinned corpus/network evidence, mapped to the same regulation edition and clause, and their relevant text is compared. Missing, stale, disputed, inaccessible, or conflicting evidence must be explicit and must not be labeled verified.
• R7. Conversation and document citations must use one structured citation contract, preserve the edition and excerpt used at creation time, and provide an authorized path to inspect the source evidence.
• R8. Audit records must reconstruct the question, corpus revision, search provider/query/results, retrieved clause IDs, source snapshots and hashes, comparison result, generated claims and citations, actor, authorization outcome, and any human resolution. Published evidence batches must detect unauthorized modification independently of mutable database rows.
• R9. Corpus access and audit disclosure must follow resource-level and project-level authorization. Public or shared regulations do not grant access to private project documents, conversations, or audit records.
• R10. MinerU may be used as a replaceable parsing adapter for supported assets. Its token is server-side only; parsing limits, task lifecycle, failures, and output provenance must be handled without exposing credentials or treating parser output as authoritative legal truth.
• R11. The regulations knowledge component must be independently versioned and testable. Its first deployment may remain local and use PostgreSQL/pgvector; a separate always-on network service is not required unless performance or operational evidence justifies it.
• R12. Outbound search queries must use the minimum regulatory context needed and exclude confidential project/user content by default. Source fetching must enforce SSRF protections at DNS resolution, connection, and every redirect.
• R13. Corpus and evidence retention must follow source-specific rights, retention, revocation, and takedown obligations. Withdrawal or expiry must not silently rewrite historical evidence; its current availability and legal retention basis must be auditable.

Key Decisions

• K1. Independent knowledge component: use a dedicated regulations knowledge repository rather than placing corpus ingestion, parsing, and indexing inside RDM. This was accepted after considering the tighter RDM-only alternative; it gives the corpus pipeline and retrieval index an independent lifecycle while leaving RDM workflows in RDM. Governs R3, R4, R5, R10, R11. (session-settled: user-approved — chosen over keeping the entire corpus pipeline in RDM: ingestion, corpus versioning, and index rebuilds have a distinct lifecycle.)
• K2. Source assets are not Git source: large source documents, database dumps, runtime indexes, and secrets remain in controlled asset/storage systems or immutable releases. Git stores schemas, parsers, metadata contracts, migration/configuration, test fixtures that are distributable, and corpus manifests. Governs R3, R5, R10, R13.
• K3. PostgreSQL-first retrieval: begin with PostgreSQL relational source records, full-text search, and pgvector where current capacity tests support it. Keep vector indexes derived and rebuildable; move them to a dedicated vector service only if measured needs require it. Governs R3-R5, R11.
• K4. Structure-first chunking: retain exact paragraph/clause nodes and construct retrieval chunks from legal structure rather than fixed-width text windows. Include heading breadcrumbs in retrieval text, but keep headings distinguishable from quoted source text. Governs R2-R5.
• K5. Separate source and summary: store document/section summaries as generated retrieval aids with their model/prompt provenance. Do not require paragraph summaries by default, and never use any summary as the evidence for a legal citation. Governs R2, R4, R6.
• K6. Fail closed on evidence state: only mark dual-source verified when both legs have valid evidence and matching edition/clause/text. A single source, inaccessible channel, unresolved edition, or substantive disagreement produces an explicit degraded/disputed state. Governs R1, R6-R8.
• K7. RDM owns product integration: the knowledge repository owns corpus, retrieval, and ingestion; RDM owns relevance routing, dual-source orchestration, user/project authorization, conversation/document citation rendering, and business audit projection. Governs R1, R6-R9, R11-R13.

Actors

• A1. RDM user asks a regulatory question or requests work that implicates regulations, standards, or review guidance.
• A2. RDM agent retrieves and uses verified evidence but cannot invent legal identity, version, clause, or quotation.
• A3. Corpus maintainer reviews source provenance, legal identity/status, parsing quality, licensing, and publication.
• A4. RDM reviewer resolves legal applicability or substantive conflicts that automation cannot safely decide.
• A5. Knowledge component and web-search provider return independently attributable evidence.
• A6. Base-platform owner publishes compatible shared-regulations retrieval and web-search capabilities when the current public extension contract cannot provide them safely.

Key Flows

• F1. Dual-source regulatory answer: a regulatory-relevant task is detected; RDM queries a pinned corpus revision and executes a real web search; source documents are fetched/parsed and mapped to clauses; evidence is compared; the model produces claims bound to citation objects; citation validation runs before the answer is shown; the evidence batch is persisted for audit.
• F2. Corpus publication: an authorized asset batch is inventoried; exact source and rights are recorded; documents are parsed; structure and metadata pass quality gates or enter quarantine; stable nodes and retrieval chunks are generated; full-text/vector projections and evaluation results are built; a reviewed corpus revision is published without deleting prior revisions.
• F3. Document citation lifecycle: a generated document stores citation IDs and pinned evidence/version references; preview/export resolves those references; later source changes flag potentially stale citations but never silently rewrite the historical document.
• F4. Audit replay: an authorized auditor opens a regulatory claim and sees the exact evidence available at the time, retrieval/search provenance, comparison, citation mapping, and any human decision, subject to disclosure controls.

Acceptance Examples

• AE1. Given both channels return the same current edition and exact clause text, when the RDM agent answers, then each regulatory assertion has a citation that opens the matching clause excerpt and both evidence records.
• AE2. Given the corpus has a clause but real web search is unavailable, when a regulatory task runs, then the result is explicitly single-source/degraded and cannot be labeled dual-source verified.
• AE3. Given web search finds an amended edition that conflicts with an older corpus edition, when comparison runs, then the system records both editions and the conflict, prevents a green verified state, and exposes the item for human review.
• AE4. Given a vector index is rebuilt with a new chunking or embedding version, when an auditor opens an older answer, then the original citation still resolves to the prior corpus revision, source node, excerpt, and hashes.
• AE5. Given an unauthorized user requests another project's regulatory audit detail, when authorization is evaluated, then the request is denied or redacted without exposing project content or restricted source text.
• AE6. Given MinerU rejects a file, times out, or returns structurally incomplete parsing, when ingestion runs, then the job remains failed/quarantined with a reason and no unreviewed chunks enter the published corpus.
• AE7. Given a real clause citation whose text does not support the generated claim, when citation validation runs, then citation authenticity may pass but claim support fails; the claim is marked unsupported/unverified or routed for review, and is not presented as a supported regulatory conclusion.

Success Criteria

• Every displayed regulatory quotation resolves to a fixed source node and edition; citation verification can confirm the excerpt against the recorded source text/hash.
• No scenario with one unavailable source, unresolved edition, or unresolved conflict is shown as dual-source verified.
• Rebuilding an index does not alter previously stored audit batches, document citations, or source snapshots.
• Reviewers can reconstruct each accepted claim from corpus revision, exact web evidence, comparison state, and citation links without relying on model conversation history.
• Retrieval and parsing thresholds are established from a labeled representative sample before production publication; no unsupported precision/recall threshold is invented in advance.

Scope Boundaries

In scope: independent regulations knowledge component; corpus inventory and ingestion; legal-document metadata and version relationships; MinerU adapter; structure-aware parsing and chunking; full-text/vector retrieval; real web search evidence; dual-channel verification contract; structured citations in RDM conversation/document paths; immutable audit projection; evaluation, security, and recovery tests.

Deferred for Later

• Advanced audit anchoring beyond the independently verifiable tamper-evidence baseline required for published evidence batches.

Deferred to Follow-Up Work

• Dedicated distributed vector database or always-on remote KB service unless measured scale, availability, or consumer requirements demand it.
• Automated legal applicability decisions or unattended legal conflict adjudication.
• Broad monitoring dashboards beyond the operational signals needed to run ingestion, search, and index publication.
• Continuous regulatory change surveillance and automatic bulk re-ingestion after publication; initial release supports controlled, explicit corpus updates.

Outside This Plan

• Modifying or deploying production services, importing the full corpus, publishing protected standards, or approving legal interpretations.
• Moving all RDM project knowledge, templates, or audit storage into the regulations component.
• Using MinerU as a source of legal authority, network search engine, or legal-status classifier.

Dependencies and Open Questions

• The actual dump schema, row quality, source-file availability, and database-to-file provenance in ../Agent-System-Regulations must be verified before choosing a production import path. The local manifest is not sufficient evidence.
• Rights and permitted excerpt/display/export limits must be established for every source class, especially standards and third-party review guidance. This is a publication blocker, not an implementation default.
• A real web-search provider and authoritative-source allowlist must be selected. Existing RDM code can fetch model-proposed URLs but has no verified real-search capability in the inspected flow. If no existing base extension supports search, raise and implement a base extension through its approved process.
• The initial local deployment and backup topology, corpus owner, update approval role, and recovery objective must be fixed before production publication.
• Sample-driven thresholds for OCR fidelity, clause extraction, retrieval recall, citation accuracy, and stale-edition suppression are set after a representative labeled corpus is available.

---

Planning Contract

Key Technical Decisions

• KTD1. Repository boundary: Agent-System-KB-Regulations owns corpus ingestion, legal identity/version metadata, parsing, structure, chunking, retrieval indexes, corpus release manifests, and retrieval APIs/contracts. RDM retains its current plugin and owns user-facing workflows, authorization, citations, and audits. A repository boundary does not mandate a network-service boundary.
• KTD2. Retrieval source of truth: store canonical source text and legal structure in a versioned relational corpus store; store original binaries in controlled file/object storage where appropriate; treat full-text/vector indexes as disposable projections. A vector payload may cache a short retrieval excerpt plus stable IDs and hashes but cannot be the only copy of source text.
• KTD3. Chunk model: preserve the smallest stable source nodes (paragraph/list/table/clause) with parent hierarchy, ordering, page/layout locator, original and normalized text, and hashes. Build retrieval chunks by combining complete adjacent legal nodes, normally targeting approximately 300-800 tokens and allowing longer complete clauses when needed. Never split inside a sentence or legal condition solely to satisfy a fixed token limit. Tune limits on the labeled sample set.
• KTD4. Heading context: persist titles as distinct structured fields and include a breadcrumb (regulation > part/chapter/section > clause number/title) in the embedding/search representation. Do not merge synthetic breadcrumbs into the text labeled as a quotation.
• KTD5. Summary policy: keep generated regulation/part/section summaries separate from source text, with model, prompt, generation time, and summary hash. Paragraph summaries are optional and excluded from citation evidence and dual-source text comparison.
• KTD6. Immutable corpus publication: every release pins source hashes, parsed representation hashes, metadata review state, parser/chunker/index versions, and evaluation result. Rebuild creates a new index/corpus revision; it does not mutate past evidence or citations.
• KTD7. MinerU is an adapter in the ingestion pipeline, not a hard dependency for every file type. The adapter records task ID/state, limits, retry outcome, parser version, output hashes, and page/block locators. Secrets are server-side. At the consulted docs snapshot, API documentation described asynchronous task submission/result retrieval; limits stated 50 submitted files/minute, 5,000 files/user/day, and up to 200 MB/200 pages per file. Reconfirm these values during implementation because provider limits can change.
• KTD8. Credential lifecycle: MinerU and search credentials are provisioned through protected runtime secret injection, separated by environment, limited to the component that needs them, and revocable/rotatable without source changes. They must not appear in source, fixtures, browser assets, command output, logs, or evidence exports.
• KTD9. Search provider contract: network search must return provider, minimized query, timestamp, ranked results, URL/title, and fetch evidence; prioritize official publishers/regulators. Search queries contain only minimum necessary regulatory terms and exclude confidential project/user context by default. Search snippets discover sources but are not authoritative excerpts. Fetch and preserve a permitted source snapshot before it can count as verified evidence.
• KTD10. Safe source fetch: validate scheme, host, port, resolved address, response type/size, and each redirect target; reject loopback, private, link-local, multicast, and reserved destinations after DNS resolution and at connection time. Revalidation must prevent DNS rebinding between validation and connection.
• KTD11. Citation contract: one immutable citation object is shared by conversation, document, and audit views. It identifies claim-to-excerpt mapping, regulation/version/clause IDs, corpus revision, web source, exact offsets/locator, hashes, and verification state.
• KTD12. Verification state: distinguish verified, single_source, degraded, edition_unresolved, conflict, unverified, and not_applicable; do not collapse them into a Boolean or infer compliance from clause-number equality alone.
• KTD13. Audit persistence: append evidence batches and human resolutions. Store full snapshots only when rights permit; otherwise retain legally allowed excerpts, source locator, hashes, fetch metadata, and an explicit reason the full source is unavailable. Published batch manifests must be tamper-evident through a separately verifiable digest/signature or append-only anchor, not only hashes stored beside mutable rows. Apply source-specific retention, withdrawal, and takedown rules while preserving a non-content audit event where permitted. Never let re-indexing overwrite evidence.
• KTD14. Shared integration contract: define a public, read-only regulations retrieval surface that pins corpus revision and returns stable regulation/edition/node IDs, permitted excerpts, locators, hashes, and disclosure metadata. The base contract must not route through owner-scoped project KnowledgeSearch. RDM may use only released public extensions; if absent, submit the required base proposal and compatibility tests before RDM integration. Real web search follows the same public extension boundary.

High-Level Technical Design

flowchart LR
  Asset[Controlled legal assets] --> Intake[Inventory and rights gate]
  Intake --> Parse[Parser adapters including MinerU]
  Parse --> Structure[Versioned legal structure and source text]
  Structure --> Build[Chunk and index builder]
  Build --> Corpus[(PostgreSQL corpus and full-text index)]
  Build --> Vector[(Rebuildable vector projection)]
  RDM[RDM regulatory workflow] -->|query pinned revision| Corpus
  RDM --> Search[Real web-search provider]
  Search --> Fetch[Fetch and snapshot permitted evidence]
  Corpus --> Compare[Clause/version/text comparison]
  Fetch --> Compare
  Compare --> Citation[Immutable citation/evidence batch]
  Citation --> Chat[Conversation citations]
  Citation --> Docs[Document citations]
  Citation --> Audit[Authorized audit projection]

The relations are versioned rather than overwrite-in-place:

source_asset -> regulation_identity -> regulation_edition -> legal_nodes
                                                |              |
                                                |              +-> retrieval_chunks -> full-text/vector projections
                                                +-> status/relations (amends, replaces, repeals)

verification_batch -> corpus_revision + legal_node/excerpt + web_capture + comparison + generated_claims
document_version -> citation_ids -> verification_batch/source hashes

Data and Chunking Contract

The canonical legal node is the smallest accurately locatable source unit. Typical node kinds are document title, part/chapter/section, clause, subclause, paragraph, list item, table, table row/cell, footnote, and annex. The source may lack some levels; preserve actual structure and record extraction gaps rather than inventing empty hierarchy.

Each node records regulation and edition IDs, parent and sibling order, node kind, displayed clause number/title, original text, normalized search text, source asset hash, page range and parser locator, extraction confidence/review state, and original/normalized text hashes. Table content retains header-to-cell relationships. A missing or ambiguous clause number is represented as such and cannot be fabricated for citation display.

Retrieval chunks are a separate projection over one or more nodes. They carry ordered node IDs, full heading breadcrumbs, exact composed source text with component spans, chunking version, and embedding model/version. They may overlap only by referencing adjacent nodes where evaluation proves context helps; duplicated overlap text is not independently cited. At citation time, the system resolves the generated claim to exact source-node spans, not just the retrieval chunk.

Generated summaries are stored in a separate table/object and indexed as lower-trust retrieval hints. Every summary states its coverage, target regulation revision, model/prompt version, and generation provenance. Ranking must not cause a summary to outrank an exact applicable clause without displaying that it is a summary. Summaries never satisfy the source-evidence requirement.

Assumptions

• The source corpus may be imported into a new controlled PostgreSQL database rather than modifying the source dump in place.
• First release favors a locally deployable component and a simple supported retrieval interface over a distributed multi-service topology.
• The RDM repository remains the owner of project-specific claim records and user-visible conversation/document flows; the regulations repository is the owner of canonical legal corpus and reusable evidence/search contracts.
• Some source classes may not permit full-text retention or redistribution; their ingestion may be metadata-only or blocked until rights are resolved.

Sequencing and Dependencies

U1 corpus/rights audit -> U2 repository contracts and schema -> U3 parsing/structure/chunking -> U4 retrieval and evaluation -> U6 RDM dual-source and audit integration -> U7 conversation/document citations -> U8 composition, security, recovery, and acceptance. After U2 stabilizes the identifiers/evidence contract, U3, U5, and U9 may proceed in parallel; U4 depends on U3; U6 depends on U4, U5, and U9. RDM must not claim dual-source verification until the public contracts and both evidence legs are integrated.

---

Implementation Units

U1. Verify corpus provenance, content, and publication rights

• Goal: Establish a trustworthy import inventory and identify material that can legally and technically be published.
• Requirements: R3, R5, R10.
• Dependencies: None.
• Files: docs/proposals/regulations-kb/corpus-inventory.md; source repository's manifest.json and database dump are read-only inputs. U2 carries the reviewed inventory into the new repository after bootstrap.
• Approach: Inspect database schema and representative rows; map documents, editions, chunks, nodes, aliases, relations, and source paths; validate dump integrity and source-file availability; classify rights and disclosure constraints; produce a provenance manifest. Keep original data untouched and quarantine records with ambiguous identity, status, or rights.
• Patterns to follow: Existing RDM TemplateAsset provenance fields and immutable source/output hashes; controlled asset batches rather than copying runtime data into Git.
• Test Scenarios:
  • A valid sample asset maps to exactly one source hash and an explicit source/rights record.
  • Duplicate binary content is deduplicated without merging distinct legal identities or editions.
  • Missing source, invalid dump row, ambiguous edition, or unknown rights is quarantined and excluded from publishable counts.
• Verification: Inventory totals reconcile with the imported source snapshot or discrepancies are enumerated; a reviewer can trace each accepted sample back to its original asset and provenance. No product source assets are copied into the new repository during this discovery unit.

U2. Establish the independent repository, domain contract, and versioned schema

• Goal: Create the separately versioned regulations knowledge component and its stable source/retrieval contract.
• Requirements: R3, R5, R11.
• Dependencies: U1.
• Files: Agent-System-KB-Regulations/README.md; Agent-System-KB-Regulations/AGENTS.md; Agent-System-KB-Regulations/PROJECT.md; Agent-System-KB-Regulations/go.mod; Agent-System-KB-Regulations/internal/domain/; Agent-System-KB-Regulations/internal/store/; Agent-System-KB-Regulations/migrations/; Agent-System-KB-Regulations/tests/.
• Approach: Define stable regulation identity, edition/status, source asset, legal node, retrieval chunk, corpus revision, and evidence provenance entities. Separate canonical data from search projections. Provide local deployment configuration and schema migration/recovery ownership. Keep corpus bytes out of Git; use controlled object/file references and immutable corpus release manifests.
• Patterns to follow: RDM domain/store separation, immutable hashes, explicit revision contracts, fail-closed validation, and existing PostgreSQL deployment practices.
• Test Scenarios:
  • A regulation with multiple editions and amends/replaces/repeals relations persists and reloads without identity collision.
  • A corpus revision cannot publish when required rights, source hash, status review, or node provenance is missing.
  • Schema migration and restore tests preserve edition/node identities and immutable historical revisions.
• Verification: Domain and persistence contracts round-trip against the supported local database, and a clean restore reproduces the same corpus revision hashes.

U3. Build structure-aware parsing, chunking, and MinerU adapter

• Goal: Convert approved source assets into locatable legal nodes and retrieval chunks without losing source boundaries.
• Requirements: R2-R5, R10.
• Dependencies: U2.
• Files: Agent-System-KB-Regulations/internal/ingest/; Agent-System-KB-Regulations/internal/parser/; Agent-System-KB-Regulations/internal/chunk/; Agent-System-KB-Regulations/internal/mineru/; Agent-System-KB-Regulations/tests/ingest/; Agent-System-KB-Regulations/tests/chunk/; Agent-System-KB-Regulations/tests/mineru/.
• Approach: Add parser adapters that produce a common block/tree representation. Preserve titles, clause numbers, paragraphs, list structure, tables, page and layout locators, parser version, and confidence. Build chunks from complete legal nodes using the versioned structure-first policy; include heading breadcrumbs in search text while keeping synthetic context separate from quote text. Store document/section summaries separately with model/prompt provenance. Treat MinerU as an asynchronous optional adapter with server-side credentials and explicit limit/retry/error states.
• Patterns to follow: Existing RDM parsed-document contracts, source/content hash validation, parser adapter boundaries, and quarantine states for incomplete assets.
• Test Scenarios:
  • A structured regulation preserves regulation > chapter > clause > paragraph hierarchy and exact source text through parse/chunk/reload.
  • A long clause is split only at legal paragraph boundaries; every chunk maps back to ordered source nodes and no condition is dropped.
  • A table retains headings and cell relationships; citations resolve to the correct row/cell source location.
  • Missing titles, missing clause numbers, low-confidence OCR, and absent page locators are represented as gaps rather than invented metadata.
  • MinerU timeout, provider rejection, rate limit, invalid result, or output hash mismatch leaves the job non-published and resumable/reviewable without leaking the token.
  • Summary generation failure does not block exact-source storage and cannot mark a citation as verified.
• Execution note: Characterize parsing on representative structured, scanned, table-heavy, and long-clause samples before fixing token targets.
• Verification: Golden parse fixtures prove text/structure/locator fidelity; all published chunks have a source-node map, and parser/chunker versions are reproducible.

U4. Implement hybrid retrieval, corpus revisions, and retrieval evaluation

• Goal: Retrieve exact, applicable clause candidates across exact identifiers, full-text, and vector semantics while keeping indexes rebuildable.
• Requirements: R1-R5, R11.
• Dependencies: U2, U3.
• Files: Agent-System-KB-Regulations/internal/retrieval/; Agent-System-KB-Regulations/internal/index/; Agent-System-KB-Regulations/internal/publish/; Agent-System-KB-Regulations/eval/; Agent-System-KB-Regulations/tests/retrieval/; Agent-System-KB-Regulations/tests/publish/.
• Approach: Implement exact standard/number/clause lookup, metadata/effectivity filters, PostgreSQL full-text and vector candidate retrieval, and explainable reranking. Return stable regulation/edition/node IDs, exact excerpts, heading paths, source locators, hashes, and corpus/index versions. Use document/section summaries only as lower-trust aids. Build a labeled retrieval set and compare clause recall, edition selection, quote localization, and stale-edition suppression before publication.
• Patterns to follow: Stable snapshot/revision APIs, evidence provenance enums, current RDM query/retrieval adapters, and immutable publish manifests.
• Test Scenarios:
  • Exact standard number plus clause query selects the matching edition and node even when semantic similarity ranks another edition highly.
  • Natural-language query returns relevant clause nodes with heading context and exact excerpts, not only summaries.
  • A superseded or future edition is excluded or clearly labeled when the query requests current requirements.
  • Rebuilding an embedding or full-text index creates a new revision and leaves old retrieval evidence readable.
  • Empty results, malformed filters, unavailable vector index, and partial index build return explicit degraded states without fabricated hits.
• Verification: Evaluation report includes dataset provenance and per-slice retrieval metrics; publication is blocked if configured acceptance thresholds regress from the reviewed baseline.

U5. Provide real web search and immutable web evidence capture

• Goal: Implement the knowledge component's independently attributable network search and evidence-capture provider behind the public contract.
• Requirements: R1, R6, R8, R12, R13.
• Dependencies: U2 for the stable regulation/evidence contract; may run in parallel with U3 and U4 after that contract is agreed.
• Files: Agent-System-KB-Regulations/internal/websearch/; Agent-System-KB-Regulations/internal/webcapture/; Agent-System-KB-Regulations/internal/evidence/; Agent-System-KB-Regulations/tests/websearch/; Agent-System-KB-Regulations/tests/webcapture/.
• Approach: Implement the provider adapter and fetch/evidence capture in the knowledge component. Record provider, minimized query, timestamp, ranked results, URL/title, and official-source classification. Exclude confidential project/user context from outbound queries by default; allow only a separately approved policy to add specific sensitive context. Fetch selected pages/files with DNS-resolution and connection-time private/reserved-address rejection, host/scheme/port/size controls, and validation of every redirect target. Retain only rights-permitted snapshots or excerpts plus hashes and fetch metadata, and parse clauses into candidates. Do not treat search snippets, model-generated URLs, or inaccessible pages as verified evidence. U9 owns only the public base contract/compatibility boundary; U6 owns the RDM consumer adapter and workflow integration.
• Patterns to follow: Existing URL fetch validation/snapshot code in RDM as a starting point only; preserve the distinction between candidate discovery and verified source fetch.
• Test Scenarios:
  • A provider response records the exact query/provider/rank/time and returns stable result evidence.
  • Search timeout, rate limit, invalid URL, redirect to a disallowed host, non-text body, oversized content, or fetch failure cannot count as a verified web source.
  • A search snippet that is not present in fetched source text remains unverified.
  • A query containing a confidential project identifier is minimized or rejected before reaching the provider unless an explicit approved policy allows it.
  • An authentic citation whose excerpt does not support the generated claim passes source-integrity validation but fails claim-support validation and is marked unsupported/unverified or routed for review.
  • An official source page fetch produces a content hash and a reproducible clause excerpt/locator within applicable retention rules.
  • DNS answers resolving to private/reserved addresses, a DNS rebinding attempt, or a redirect to a disallowed address are rejected before connection and cannot count as evidence.
  • An expired or revoked source snapshot is withheld from normal display and records a rights/retention event without silently changing prior batch identity.
• Verification: Integration tests demonstrate an actual provider search-to-fetch path and prove failure states remain non-verified; provider secrets never appear in logs, storage exports, or client responses.

U6. Integrate mandatory dual-source verification and immutable audit batches in RDM

• Goal: Replace model-suggested network candidates as the search mechanism and bind each regulatory conclusion to immutable knowledge-base and web evidence.
• Requirements: R1, R6, R8, R9, R11-R13.
• Dependencies: U4, U5, U9.
• Files: Agent-System-RDM/internal/domain/regcheck.go; Agent-System-RDM/internal/domain/payloads.go; Agent-System-RDM/internal/domain/replay.go; Agent-System-RDM/internal/rdmapp/regcheck.go; Agent-System-RDM/internal/rdmapp/regfetch.go; Agent-System-RDM/internal/rdmapp/; Agent-System-RDM/plugins/rdm/plugin.go; Agent-System-RDM/internal/domain/regcheck_test.go; Agent-System-RDM/internal/rdmapp/regcheck_test.go; Agent-System-RDM/internal/rdmapp/regfetch_test.go; Agent-System-RDM/plugins/rdm/ tests.
• Approach: Add a versioned evidence-batch model and persist complete source references, edition/node IDs, exact excerpts, source snapshots/hashes, query and fetch metadata, state transitions, and human resolutions. Call shared regulations retrieval through U9 and real web search through its released public contract as separate legs. Compare regulation identity, edition, clause mapping, and relevant text; replace clause-number-only agreement with evidence-level comparison. Retain existing project authorization and audit boundaries. Keep historical batches replayable after corpus re-indexing. Add a separately verifiable tamper-evidence record for published batches and rights-aware retention/withdrawal behavior.
• Patterns to follow: RDM event-log replay, content-addressed network snapshots, project-authorized snapshot reads, and resource-level authorization.
• Test Scenarios:
  • Same-edition, same-clause, matching excerpts from both legs produce a verified batch with both immutable evidence references.
  • A citation that resolves to authentic matching text but whose excerpt does not support the associated claim is recorded as unsupported/unverified or routed for review; source integrity cannot upgrade claim support.
  • Knowledge-only, web-only, unavailable search, unknown edition, and mismatched text produce distinct non-verified states.
  • Same clause number but different edition/text produces conflict, not agreement.
  • Restart/replay restores all batch metadata, exact citation refs, human resolution, and snapshot hashes.
  • Mutation of a stored batch or its linked source data is detected by audit verification when mutable database rows and adjacent hashes are changed together.
  • Source withdrawal/takedown updates current access according to its retention rule while retaining a permitted non-content history event and preserving document/version references.
  • Unauthorized actor cannot read another project's evidence batch or retrieve private source text through an audit command.
  • Existing malformed/model-generated network output cannot satisfy the real-search requirement.
• Verification: Domain and application tests prove state invariants, persistence replay, source validation, and project isolation; the command-bus integration proves the public RDM command path enforces them.

U7. Add structured citations to conversation, documents, and audit views

• Goal: Present one stable citation object consistently in generated answers, document content, preview/export, and audit inspection.
• Requirements: R2, R7-R9.
• Dependencies: U6.
• Files: Agent-System-RDM/internal/domain/; Agent-System-RDM/internal/rdmapp/; Agent-System-RDM/plugins/rdm/plugin.go; Agent-System-RDM/web/app.js; Agent-System-RDM/web/index.html; Agent-System-RDM/web/app.css; Agent-System-RDM/web/workbench_test.mjs; Agent-System-Frontend/src/; Agent-System-Frontend/test/; Agent-System-Documents/docs/proposals/ if a shared renderer/API extension is required.
• Approach: Bind claim spans to citation IDs; render concise regulation/edition/clause/status marks with expandable exact excerpt and authorized source details. Persist citation IDs and evidence revisions in document versions and exports. Mark citations stale when a later corpus revision changes status or text, but preserve the original basis and require explicit review before refreshing it. Use existing RDM audit workspace/read-model patterns for source history rather than introducing a new write authority.
• Patterns to follow: Existing RDM command-bus UI, three-pane audit workspace and shared snapshot rules, document citation/trace contracts, and UI design tokens/components in the consuming frontend.
• Test Scenarios:
  • An answer with two regulatory claims renders two distinct citation marks linked to their claim spans and correct clause excerpts.
  • A generated document stores the citation and pinned edition; preview/export preserves it across reload.
  • A later corpus update marks the citation for review without mutating the saved document or historic audit batch.
  • A user without source-text disclosure permission sees the permitted metadata/redaction reason but not restricted excerpt or original content.
  • Missing or invalid citation IDs are surfaced as unverified and cannot be rendered as valid legal references.
• Verification: Browser and document round-trip tests prove citation persistence, navigation, redaction, stale marking, and export consistency.

U8. Validate composed deployment, recovery, privacy, and release readiness

• Goal: Prove the complete local RDM + regulations component works with recoverable data/index lifecycle and protected evidence.
• Requirements: R1-R13.
• Dependencies: U1-U7, U9.
• Files: Agent-System-KB-Regulations/deploy/; Agent-System-KB-Regulations/docs/operations/; Agent-System-KB-Regulations/tests/integration/; Agent-System-RDM/tests/integration/; Agent-System-Dev/tests/; Agent-System-Dev/PROJECT.md; Agent-System-Dev/ component pins and composition configuration.
• Approach: Pin compatible component revisions; provision a disposable local corpus/database; build a new index revision; run dual-source and citation flows end-to-end; exercise backup/restore, source/index rebuild, provider/parser outages, and authorization. Keep legal source data and secrets outside test output and source-control history. Separate local test acceptance from deployment and business/legal approval.
• Patterns to follow: Existing Dev composition verification, RDM PostgreSQL and audit replay tests, fixed component pins, and resource/project authorization tests.
• Test Scenarios:
  • A full regulatory question returns both channel records, exact cited clause, claim mapping, and audit-batch replay through the composed product.
  • One channel is deliberately disabled and the UI/agent reports degraded verification without presenting a green dual-source result.
  • Backup/restore followed by index rebuild preserves corpus revision hashes and historical citation resolution.
  • Cross-project and restricted-source access cases are denied/redacted through the real command/API composition.
  • MinerU/search credentials are absent from client assets, logs, command output, test fixtures, and audit exports.
  • Credentials are injected per environment, scoped to the required component, and can be rotated/revoked without changing source or leaving old credentials usable.
• Verification: Exact composition tests and security/recovery checks pass on the pinned candidate; release, deployment, and user/legal acceptance are explicitly recorded as separate later gates.

U9. Publish the shared regulations retrieval and search extension contract

• Goal: Provide an authorized public integration surface for RDM to query the shared read-only legal corpus and perform real web search without misusing owner-scoped project knowledge APIs.
• Requirements: R1, R6, R8, R9, R11, R12.
• Dependencies: U2 for stable identifiers and evidence schema; can be designed alongside U3-U5.
• Files: Agent-System-Documents/docs/proposals/; applicable base-repository extension contracts under Agent-System-Backend/pkg/extension/; base-repository contract tests.
• Approach: First verify whether an existing public Host or API contract can express shared corpus retrieval with revision pinning, stable source-node references, excerpt disclosure policy, and audit provenance, plus access to the knowledge component's real-search provider with minimized query input and evidence provenance. Do not use the existing owner-scoped project KnowledgeSearch as a substitute. If it cannot represent these capabilities, document the scenario, required public surface, reason existing registration points do not compose, and compatibility plan in the base proposal process; implement/release the base contract before RDM consumes it. No backend-internal import or private credential bypass is allowed. U5 owns search-provider behavior; U9 owns only the public contract and compatibility tests; U6 owns the RDM consumer adapter and orchestration.
• Patterns to follow: RDM's extension boundary and approved base-extension proposal process; versioned public command/Host contracts and cross-repository compatibility tests.
• Test Scenarios:
  • A consumer queries a pinned shared corpus revision and receives stable regulation/edition/node IDs, permitted excerpt, locator, hashes, and disclosure state.
  • An unauthorized or unsupported request cannot access restricted source text or use a project ID to widen shared corpus access.
  • Search receives only permitted minimized query fields and returns provider/query/result provenance; provider failure remains a non-verified result.
  • Compatibility tests prove the released base surface is consumable by the pinned RDM adapter without importing backend internals.
• Verification: A versioned public contract and compatibility tests exist before U6; if base changes are needed, RDM integration uses only the released compatible contract. This unit does not implement a second provider, fetcher, or RDM adapter.

---

Verification Contract

Gate	Evidence required	Applies to
Corpus admission	Source hashes, source/rights inventory, schema/data reconciliation, quarantined-record report	U1, U2
Parser/chunk quality	Golden fixtures for clauses, paragraphs, headings, lists, tables, OCR, locators, and provenance; no lost or invented source text	U3
Retrieval quality	Labeled set with exact/semantic queries; report retrieval, edition, locator, stale-source, and summary-vs-source behavior	U4
Search evidence	Provider search, minimized query/result metadata, fetched source, source policy, SSRF negative cases, hash and excerpt/locator verification	U5, U9
RDM invariants	Dual-source state matrix, event replay, authorization denial/redaction, tamper-evidence verification, rights-aware withdrawal, command-bus tests	U6
User-visible citations	Browser/document round trip, stable source navigation, permissions, stale citation display, export	U7
Composition/recovery	Fixed component pins; full-path tests; disposable DB backup/restore and rebuild; secret lifecycle and leak checks	U8, U9
Delivery trust	Current target synchronization, independent review, and required exact-head CI/REPLAY evidence under each repository's delivery policy	All release candidates

Planning does not run these tests. Implementers should use each target repository's current authoritative test commands and policies when the work starts. The three uninitialized Dev submodules reported by precheck must be safely materialized before a future composition build; no submodules were changed for this plan.

Definition of Done

• U1-U9 are complete or explicitly deferred with owner and rationale; no unresolved launch-blocking rights, source identity, or extension-contract decision remains.
• The published corpus release is reproducible from approved source assets, versioned metadata, parser/chunker versions, and evaluation record.
• Every retrieval result contains stable source-node and revision references; exact excerpts resolve independently of vector-index contents.
• Real web search and knowledge-base evidence are independently recorded, and dual-source status cannot be inferred from model output or clause-number equality alone.
• Conversation, document, and audit citations share the same immutable evidence identities and enforce RDM project/resource permissions.
• Required component and composition checks pass on exact pinned revisions; review, CI, deployment, and business acceptance are reported as distinct states.
• No source assets, secrets, or abandoned experimental ingestion/index outputs are left in Git or the final product artifact.
• Published evidence tampering is detectable; confidential project context is not sent to external search by default; fetch redirects and DNS resolution cannot reach private/reserved network targets.

Appendix

Existing Evidence and Integration Anchors

• RDM requirements: ../Agent-System-RDM/docs/requirements/01-用户需求与系统需求.md (UR-14, SR-F3, SR-N4); ../Agent-System-RDM/docs/requirements/02-产品概念与形态定义.md (shared pgvector regulations library); ../Agent-System-RDM/docs/requirements/03-产品设计方案.md (M11 dual-channel requirements and local regulations storage).
• Current RDM flow: ../Agent-System-RDM/internal/rdmapp/regcheck.go uses host knowledge search and model-proposed URLs; ../Agent-System-RDM/internal/rdmapp/regfetch.go fetches, compares excerpts, and stores content-addressed network snapshots; ../Agent-System-RDM/internal/domain/regcheck.go records dual-channel claims but currently compares clause identity more narrowly than this plan requires.
• Current RDM integration boundary: ../Agent-System-RDM/AGENTS.md requires extension Host use and prohibits importing backend internals; missing base capabilities require a base proposal and public extension.
• Prior platform direction: modules/documents/docs/proposals/2026-09-29-system-skeleton-design-and-plan.md envisioned a future regulations plugin and a pluggable local/vector RAG pipeline. This plan narrows ownership: the new repository owns reusable corpus/retrieval; RDM retains its product plugin and audit/UI flows.
• MinerU references: https://mineru.net/apiManage/docs and https://mineru.net/apiManage/limit. Provider quota and file limits are time-sensitive and must be rechecked before implementation.
• Current regulations input candidate: ../Agent-System-Regulations/manifest.json and ../Agent-System-Regulations/regulations_db.dump. Presence and manifest counts do not verify rows, licensing, raw-source completeness, or legal effectivity.