vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

93.8k

Stars

+31.0k

Gained

49.5%

Growth

Python

Language

View on GitHub → ↑0.5% this week

🎭 Best For

Machine Learning and AI Engineer

⚖️ Compare With

vllm vs yolov5 vllm vs ray vllm vs DeepSpeed

🏷️ Topics & Ecosystem

amd blackwell cuda deepseek deepseek-v3 gpt gpt-oss inference kimi llama llm llm-serving model-serving moe openai pytorch qwen qwen3 tpu transformer

📊 Activity

Latest commit: 2026-10-14. Over the past 336 days, this repository gained 31.0k stars (+49.5% growth). Activity data is based on daily RepoPi snapshots of the GitHub repository.