Total 4 skills · from GitHub & curated sets, updated daily
NousResearch/hermes-agent/optional-skills/mlops/inference/serving-llms-vllm
vLLM: high-throughput LLM serving, OpenAI API, quantization.
NousResearch/hermes-agent/optional-skills/mlops/tensorrt-llm
High-throughput LLM inference on NVIDIA GPUs.
Orchestra-Research/AI-Research-SKILLs/12-inference-serving/tensorrt-llm
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
Orchestra-Research/AI-Research-SKILLs/12-inference-serving/vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.