Thalamus is a vendor-neutral, Kubernetes-native inference service for sovereign LLM deployments. Weights, prompts, and context are protected and stay in the deployment perimeter.
Thalamus is built on llm-d, vLLM, the Gateway API inference extension, and Cortex.
Supports Nvidia, AMD, and Intel infrastructure.
Full documentation can be found on our website covering architecture, getting started, API references, and more.
Thalamus integrates natively with CobaltCore, Greenhouse and the broader Apeiro Reference Architecture ecosystem.
⚠️ Status. Thalamus is under active development. The value propositions and features below reflect the project vision. Not all of them are implemented today. APIs, CRDs, and chart values are subject to change.
- Regulatory unblock. Serves regulated and public-sector workloads that cannot use hyperscaler AI providers.
- No vendor lock-in. No dependency on a single hyperscaler, LLM vendor, or GPU infrastructure provider. Swap models and hardware behind a stable API.
- IP and data protection. Confidential computing keeps model weights and customer context inside attested GPU boundaries.
- One stack, many targets. The same stack runs in datacenters, sovereign cloud, and air-gapped satellite sites.
- OpenAI-compatible inference API with streaming
- Declarative model management via a Kubernetes Custom Resources including full lifecycle management
- Model-aware routing through the Gateway API Inference Extension
- KV-cache aware load balancing via the llm-d Endpoint Picker
- Autoscaling driven by time-to-first-token and queue-depth metrics
- Multi-vendor GPU support (Nvidia, AMD, Intel)
- Observability via Prometheus, OpenTelemetry, and GPU exporters. Integrates natively with Greenhouse.
- Ready for air-gapped satellite deployments
This project is open to feature requests/suggestions, bug reports etc. via GitHub issues.
Contribution and feedback are encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our Contribution Guidelines.
If you find any bug that may be a security problem, please follow our instructions at in our security policy on how to report it. Please do not create GitHub issues for security-related doubts or problems.
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone. By participating in this project, you agree to abide by its Code of Conduct at all times.
Copyright 2026 SAP SE or an SAP affiliate company and Thalamus contributors. Please see our LICENSE for copyright and license information. Detailed information including third-party components and their licensing/copyright information is available via the REUSE tool.