Skip to content

Performance path is endpoint Providers behind the existing Capability seam; the second Runner reuses the local Runner's Stage-execution engine and swaps only state/scheduling

ADR-0004 (Capabilities with Provider plugins), ADR-0007 (CPU-first reference Providers, endpoint as the performance path), and ADR-0003/0010 (pluggable Runners; local Runner state is files-on-Storage) fixed the what. S12 proves both seams generalise, and records four implementation trade-offs that were not obvious from those ADRs.

1. The heavy-model performance path is one endpoint Provider per Capability, not a new seam. Each heavy Capability (Parse, Entity Extraction, Embedder) gains a *.endpoint Provider in its own package that targets any OpenAI-compatible / vLLM / Triton endpoint; a Pipeline swaps CPU↔endpoint by changing the provider name and a base_url — one config line, no pipeline change, output identical modulo the model (ADR-0007). The Providers emit the same typed contracts as the CPU references (an endpoint EntityMention carries the identical offset+page Provenance the S4 round-trip needs; an endpoint Parser degrades a failed call to the same PARSE_ERROR record the CPU parser emits for a corrupt file), so nothing downstream can tell which Provider ran. Relation Extraction already had its endpoint Provider (relation.llm, ADR-0013), so S12 added only the missing three. Rejected: a single "remote inference" Provider parameterised by task (the OCR/NER/embedding request+response shapes are too heterogeneous for one class — the same reason ADR-0004 rejected a LiteLLM-style router).

2. Endpoint lifecycle + retry is a ported, vendor-neutral asset in core, not per-Provider glue. The private pipeline-worker/endpoint_lifecycle.py (ref-counted lease) and retry_utils.py (backoff+jitter, 5xx/429-retryable error-classing) are ported into latence_core.endpoint, generalised off RunPod: the lease's activate/deactivate are caller-supplied hooks (a vLLM autoscaler, a Triton load, or a no-op for an always-on endpoint), and the retry classes failures by HTTP status. Every endpoint Provider leans on this one implementation — co-located Stages against the same served model share one lease (pay the cold-start once), and retries are deterministic under test via injected clock/rng. The count of transient failures a Stage absorbed surfaces on StageMetrics.endpoint_retries in the Quality Report (S12 observability). It lives in the thin core (pure control-plane, zero network deps) precisely so the isolated Provider packages don't each re-port it.

3. The second Runner reuses the local Runner's Stage-execution engine verbatim; only state and scheduling are swapped. The Airflow adapter does NOT re-implement Stage dispatch: it holds a LocalRunner and calls its _execute_stage / _build_report / _read_checkpoint_text seams, so the Stage semantics are byte-identical (ADR-0003 "same Stages, pluggable Runner", proven by a test asserting the two Runners produce identical exported records). What the adapter replaces is exactly the layer ADR-0010 says an adopter-grade Runner owns itself: run-state and scheduling. One Airflow task per Stage wired by depends_on; Airflow owns which tasks ran and their retries; Stage outputs flow over Airflow XCom (re-validated through the same typed checkpoint decode) — there is no _latence/runs/.../checkpoints file state. StageMetrics.runner records which Runner ran each Stage. Rejected: forking the streaming orchestrator into an Airflow-native operator library (re-implements Stage semantics, the exact drift ADR-0003 rejects); and persisting the local checkpoint files under Airflow too (two state substrates for one run, the split-state failure ADR-0010 rejected).

4. Airflow is an optional extra and the proof runs Airflow-free. apache-airflow is a [airflow] extra, not a hard dep (ADR-0016), and the adapter's build_dag imports it lazily; the end-to-end proof runs the same per-Stage task callable in process (a topological driver + an XCom stand-in) with no Airflow install, so CI stays CPU-first, offline, and deterministic. Fault tolerance rides two layers — the Provider's ported retry inside a Stage and Airflow's task retry around it. The run's data artifacts are committed by an explicit report/commit task wired downstream of every sink Stage (issue #44.4): it builds and persists the Quality Report only after every Stage has succeeded, so a mid-DAG endpoint failure fails the DAG run before that task and leaves the last committed state intact (never torn). The adapter commits the run's Quality Report as its data artifact; a full transactional Corpus Version bump is the Delta Runner's concern, not this seam's. (Correction, issue #44.4: build_dag originally emitted one operator per Stage and no report/commit task, so the real adopter DAG produced exports but never a Quality Report and had no explicit commit — the "commit downstream of every Stage" this ADR asserted held only on the in-process run() path. The report/commit task now makes it true on the real DAG path too, with a real-airflow integration lane asserting the DAG structure, gated so CI stays Airflow-free.)