LiteLLM
OSS proxy that normalizes 100+ LLMs to the OpenAI format. Add routing, fallbacks, caching, and cost tracking in one layer.
NeMo Guardrails
NVIDIA's open-source toolkit for adding programmable safety, topic, and dialog rails to LLM applications. Define rules in Colang DSL to prevent off-topic responses, enforce output formats, and guard against prompt injection. Standard in enterprise LLM deployments.