Observability: a typed error taxonomy and a thin, OTel-free tracing seam¶
The Hardening bar (DEFINITION-OF-DONE → "Observability: tracing spans, per-stage latency/quality metrics, typed error taxonomy", issue #73) is the cross-cutting foundation the per-stage H-issues build on. Three decisions:
A typed error taxonomy, additive over the builtins it replaces. latence-core gains a shallow hierarchy — one root LatenceError plus ContractError (mis-wired DAG / carrier mismatch), ConfigError (bad operator config: a disallowed Storage scheme, a ReDoS-signature pattern, a malformed gold set), ProviderError (a Provider/endpoint failed at runtime, with a retryable flag), StorageError (a Storage/Corpus-Version inconsistency), and ScreeningQuarantineError (an opt-in fatal-Screening disposition carrying record-id + reason, never content). Each category multiply-inherits the builtin it replaces (ContractError(LatenceError, TypeError), ConfigError(LatenceError, ValueError), the two runtime categories over RuntimeError), so every pre-taxonomy except ValueError/except TypeError/except RuntimeError call site — and the tests that pin them — keeps working: the taxonomy narrows the type without breaking the old contract. The framework's previously-bare exceptions are re-homed onto it (SchemeNotAllowedError → ConfigError, CorpusStoreError → StorageError, EndpointError → ProviderError) while staying subclasses of their original builtin. A stable lowercase category tag ("contract"/"config"/"provider"/"storage"/"screening"/"unknown") is what the report + spans record, so an operator greps one field, not a refactorable Python type name; error_category() maps any caught exception to that tag (a stray builtin → "unknown") so the Runner labels a failure without importing every Provider's error type.
A thin, dependency-free tracing seam — deliberately not OpenTelemetry. latence_core.tracing emits a structured Span per Stage begin/end (and per checkpoint batch), each stamped with a run-scoped correlation id (the run_id — the run is the trace). The default LoggingSpanSink writes one structured logging record on the latence.trace logger at DEBUG — silent unless an operator turns it on, so a normal run's output is unchanged — and a caller who wants the events passes a CollectingSpanSink or their own SpanSink. This keeps latence-core near-zero-dependency (ADR-0016): an adopter who wants a real tracer fans the spans into OTel (or a log pipeline) at the edge, rather than the core carrying opentelemetry-* for every laptop install. Rejected: an OTel dependency in core (breaks the CPU-first, offline, thin-core promise; forces a heavyweight transitive tree on every adopter), and a global tracer singleton (hidden state, un-testable, no clean per-run correlation).
Counts only, never content — the same no-PII discipline as the Quality Report. A span's fields are the Stage name, capability, provider, record counts, duration, and — on failure — the typed error category + message; the new per-Stage StageMetrics.error_count/error_category (Quality Report schema v14) are likewise counts + a category tag. A record's text, a PII surface, a secret, or a document body never enters a span or a metric. This is asserted adversarially: a run whose document body is a unique PII-shaped sentinel must produce that sentinel in no emitted span and no persisted report field. On a Stage failure the Runner surfaces the category in the failed span, records the failed Stage's StageMetrics (error_count=1 + the category tag, counts only — never the exception message), persists a FAIL Quality Report (the completed Stages plus the failed one), and then re-raises — so #73's "surfaced in the report + logs" is met on both halves, not only the transient span. It never swallows a Stage failure: the failed Stage's checkpoint is still never finalized, so a mid-run crash leaves no torn write and a resume re-runs that Stage exactly and overwrites the FAIL report atomically on success (the report is not a resume gate — resume keys off per-Stage checkpoints). Persisting the FAIL report never masks the real failure (a report-build error there is swallowed so the original Stage exception propagates). Observability makes the failure legible, it does not change the determinism/resume contract.