LiteLLM
OSS proxy that normalizes 100+ LLMs to the OpenAI format. Add routing, fallbacks, caching, and cost tracking in one layer.
Guardrails AI
Open-source framework for adding input/output validation to LLM applications. Define validators (Pydantic-style) for LLM responses to catch hallucinations, PII leaks, bias, and schema violations. Supports automatic retry and reask on validation failure, with any LLM provider.