Skip to content

CPU-first reference Providers; GPU serving is an upgrade path, not the default

Every Capability ships with a CPU-viable, in-process reference Provider (small GLiNER, MiniLM/e5-class embeddings, lightweight parser, hybrid rule+model PII detector) so the entire spine runs on a laptop via docker compose — first contact is git clone → working KG, no CUDA. A second "performance" Provider set targets any OpenAI-compatible/vLLM/Triton endpoint for the heavy models (PaddleOCR-VL, GLiNER-large, DeBERTa linker); switching is a config/URL swap, never a pipeline change. This deliberately demotes the private stack's most sophisticated engineering (superpod's colocated-vLLM GPU serving) to an optional deployment recipe: that architecture existed to amortize RunPod serverless pricing, a problem framework adopters don't have. Rejected: vLLM container as the canonical path (CUDA requirement is a brutal OSS adoption filter), remote-endpoints-only (breaks the "your data never leaves" pitch in regulated industries), and GPU-primary demos (community adoption comes from laptop users).