Decision Log (ADRs)
The Architecture Decision Records are the reasoning behind every seam in the framework —
each recording the context, the choice, and the alternatives rejected. They
are the authoritative source; the Concepts and
Architecture pages are navigational summaries over them.
Note
ADRs are numbered by decision order. Number 0040 was never issued (no gap in reasoning —
the sequence simply skipped it), so the log holds 65 records numbered 0001–0066.
Foundations & scope
ADR
Decision
0001 Open-source core, personally owned, published before the Deloitte start date
0002 Clean-history monorepo; private repos are quarries, not ancestors
0005 v1 scope: the AI-ready-data spine, nothing else
0015 Python core, declarative-YAML Pipeline contract, Rust only for proven hot paths
0055 Relicence to PolyForm Noncommercial — free for research, commercial by agreement — SUPERSEDED by 0063 ; kept as the dated record of a decision that was made and reversed
0062 Distribution is signed GitHub Releases only — no PyPI — rationale SUPERSEDED by 0063 , decision REOPENED ; the pypi-publish job stays disabled pending a new decision
0063 Relicence to Apache-2.0 — the licence the intended adopter is allowed to install (supersedes 0055, restores 0001's grant; reopens 0062)
The core seams
ADR
Decision
0003 Stage library with pluggable Runners, not an orchestration control plane
0004 Capability protocols with provider plugins as the model-agnosticity seam
0006 One schema, dual serialization (JSONL + Parquet/Arrow)
0009 fsspec-backed Storage seam for cloud-agnostic IO
0010 Local Runner state is files-on-Storage, no database
0016 Thin core plus per-Provider packages, managed as a uv workspace
0017 The AI-ready deliverable is a RAG corpus and a knowledge graph, both as files
0033 Streaming, memory-bounded local Runner
0034 Observability: typed error taxonomy and a thin, OTel-free tracing seam
0047 A metrics EXPORT seam (Prometheus / OpenTelemetry) layered on ADR-0034, behind an optional extra
0032 Strict mypy is enforced on tests too, and the whole-tree conftest module collision is resolved
Providers, licensing & the ecosystem
ADR
Decision
0007 CPU-first reference Providers; GPU serving is an upgrade path
0012 Reference Providers ship permissive weights only; restricted models are opt-in
0035 A second CPU reference Provider for each single-provider Capability
0036 Provider ecosystem: adapters, device profiles, conformance, and the bake-off harness
0030 Performance path is endpoint Providers; the second Runner reuses the Stage engine
0045 IBM Granite Embedding r2 is the default learned Embedder
0065 The H-G1 dependency cap is a workspace-wide default over every third-party pin (extras included), with machine-read, reason-bearing exemptions
0066 Publication and release are separate gates — the source is published as a pre-1.0 preview while Gate 1 (measured end-to-end numbers) stays open; a v* tag still requires every gate (amends RELEASE-GATES.md)
The Stages (contracts per Stage)
ADR
Decision
0014 Source Capability seam; v1 ships only filesystem/object-store connectors
0019 Parse consumes a raw ParserInput, not a pre-decoded DocumentRecord
0020 Parse carries a page map and an error disposition on the record contract
0021 Chunk preserves offsets; Screening is two checkpoints with Quarantine off the DAG
0022 Entity mentions are first-class Records; the zero-shot label set lives in config
0023 A RelationMention is a first-class Record; a fused Stage emits both carriers
0024 Redaction is one RedactionRecord per document, keyed off Classification.sensitivity
0025 Profiling is a corpus-level Stage emitting scoped FeatureRecords over a fixed set
0026 Disambiguation emits scoped records over a CPU-viable resolver cascade
0027 Graph Assembly emits scoped GraphRecords; Export writes the KG as files
0031 Entity Extraction emits TRUE original offsets via a carried strip offset map
0042 Redaction is a chunk→chunk transform (fixes the document-truncation PII leak)
0044 A universal financial-PII redaction floor, and an optional mask-before-extract mode
0060 Page boundaries cross the OCR handoff in a sidecar; an absent page map is a value, not a page 1
0064 GLiNER 2.5 is a NEW Provider package behind an enforced gliner2 major boundary; chunk.page is page-preferred, 1024 otherwise
Optional Stages
ADR
Decision
0037 Graph Completion is an OPTIONAL post-Assembly Capability; predicted edges are inferred-marked
0038 Schema Induction is an OPTIONAL LLM Stage; induced labels are additive
0039 Context Enrichment projects the KG onto chunks into the EMBEDDING input only
0054 Type-vocabulary canonicalization is its own Capability in the resolution phase (per-chunk induction's downstream fix)
The Quality Report, deltas & operations
ADR
Decision
0008 Quality Report as a first-class run artifact
0028 The Quality Report is promoted; the gold-set harness is a Runner-side evaluation
0011 Demo: one command plus a live provider swap
0018 Incremental corpus deltas: affected-set recompute, drift-triggered reconcile, dual-mode deletion
0029 Delta orchestration: DeltaRunner wraps the LocalRunner; affected-set via blocking keys
0041 Guided setup is a deterministic wizard emitting ONE opinionated, quality-baked config
0043 Incremental extraction: a frozen induced schema + doc-level cross-run reuse
0061 Incremental ingest: exact corpus-level recompute over an order-preserving reuse union (enrichment pipelines included)
Empirical choices
ADR
Decision
0013 Relation Extraction Provider is chosen empirically; a Provider may fulfil fused Capabilities
0046 Enterprise GLinker entity-linking + audited resolver, ported from dataset-intelligence
The stateless, downstream layer that feeds an adopter's retrieval engine without being one (design;
see Retrieval-quality tooling ).
ADR
Decision
0048 Retrieval tooling is strictly stateless — it generates signals and transforms candidates, holds no index
0049 Clock-split: index-time signals extend the pipeline; query-time is a downstream library with its own latence.retrieval seam
0050 Multi-vector / MUVERA-FDE ships real-but-experimental, off by default, behind a stable seam
0051 Zero-ops multi-hop graph retrieval via DuckDB over the emitted graph parquet; Neo4j optional
0052 Retrieval MCP ports memory_engine hardening; bring-your-own backend; config-driven stack-adaptive copy
0053 SPLADE sparse is a structured composition (model ⊕ confidence-gated NER ⊕ metadata), redacted by default
0056 Hypergraph Evidence sidecar is read per call over a point-lookup KV; no global edge-adjacency matrix
0057 A Hyperedge is a third GraphScope emitted by Graph Assembly; tau_text follows the corpus redaction variant
0058 token_cost is an approximate index-time weight; the budget guarantee is an exact query-time verification pass
0059 Query-side entry is tiered: zero-model default over the alias KV; GLiNER2 query NER strictly opt-in