vLLM

vLLM is a high-throughput and memory-efficient inference and serving engine for Large Language Models (LLMs). Deploy AI models faster with state-of-the-art performance. Easy, fast, and cost-efficient LLM serving for everyone.

关键词：vLLM,LLM,Large Language Model,inference,serving,AI,machine learning,deep learning,GPU,CUDA,transformer,performance,open source