vLLM

LiteLLM connects to a self-hosted vLLM endpoint via its OpenAI-compatible API, treating it as any other provider.

→ Self-hosted GPU inference via vLLM accessible through the same LiteLLM interface as cloud providers — one config for everything.

LlamaIndex connects to a vLLM-hosted endpoint via its OpenAI-compatible API, treating self-hosted vLLM as a generation provider.

→ LlamaIndex RAG pipelines backed by self-hosted GPU inference — enterprise-grade retrieval and generation with full data residency.

vLLM runs on RunPod GPU pods as a Docker container, exposing an OpenAI-compatible inference endpoint.

→ Self-hosted high-throughput LLM inference on rented GPUs — cheaper than managed APIs at scale.

Axolotl-fine-tuned models are saved in HuggingFace format and loaded directly by vLLM for serving.

→ Complete OSS fine-tuning-to-production pipeline: train with Axolotl, serve with vLLM.

Unsloth exports fine-tuned models in GGUF or HuggingFace format, both of which vLLM serves natively.

→ Train fast with Unsloth, serve fast with vLLM — same model file, no conversion required.

LlamaFactory outputs HuggingFace-compatible checkpoints that vLLM loads directly for production serving.

→ Full fine-tuning workflow from dataset to vLLM deployment within one cohesive ecosystem.

Torchtune exports fine-tuned weights as HuggingFace safetensors, compatible with vLLM loaders.

→ PyTorch-native fine-tuning with the same vLLM deployment path as the broader HuggingFace ecosystem.

Predibase LoRA adapters can be exported and served via vLLM multi-LoRA serving mode.

→ Swap fine-tuned adapters at inference time without model reload overhead using vLLM.

vLLM supports Qwen-VL as a multi-modal model, serving vision and language requests via OpenAI API.

→ Production-grade multimodal serving for Qwen-VL with continuous batching and high throughput.

vLLM multi-modal pipeline supports InternVL2, enabling batched vision-language inference at scale.

→ High-throughput InternVL2 serving with the same OpenAI-compatible API used for text models.