The vLLM 0.27 release is out, and for anyone running multi-modal or CPU inference, it's not just a bump in version numbers. There's a new attention kernel, new token-limit options, and a big chunk of CPU and TPU performance work. If you're serving models with vLLM, here's what changed and what you should care about.

New CondensePyramid V1 attention kernel

The headline feature is the CondensePyramid V1 attention kernel. If you're serving long-context multi-modal models — think video or high-resolution image inputs attached to an LLM — this is a new implementation designed for CondensePyramid-style attention. It's a first version, so don't expect it to beat tuned kernels like FlashAttention on every shape yet. But if you're currently on a multi-modal model that relies on CondensePyramid attention, you now have a vLLM-native option instead of a custom patch. I'd try it on a small workload and compare against your current attention backend before rolling it out.

New token-tier limits on templates

You now get token-tier limits on templates. Concretely, you can set a template-level cap on input and output tokens per request, in addition to the model-level limits. This is handy for multi-tenant setups where you want to enforce per-task budgets without writing a custom post-processing hook. For example, a summarization endpoint could cap output at 1,024 tokens even if the model supports more. The tradeoff: a raw model that wasn't designed with these tiers might reject requests that a simple max-tokens override would have accepted, so test your prompts first.

CPU and other architecture performance

This is a big CPU release. ASR CPU preprocessing is now 2.5× faster via multi-threading (#44612). If you're transcribing audio on CPU — for example, Whisper-style ASR in front of an LLM — that's a measurable win. Also new: CPU W4A16 INT4 MoE support (#43409), which means you can run Mixture-of-Experts models with 4-bit weights and 16-bit activations on CPU. That's a memory win for CPU-only boxes.

There's also cgroup memory-limit-aware KV cache sizing (#45086). If you run vLLM inside a container with a cgroup memory limit, the KV cache now respects that limit — which relieves the old problem where vLLM sized the KV cache from the host's total memory and got OOM-killed inside the container. That alone is worth upgrading if you containerize vLLM.

Other CPU items: RISC-V oneDNN W8A8 INT8 (#44478) and RVV micro-GEMM for WNA16 (#44324), plus pinned memory for WSL2 (#41496). ZenCPU runtime logging (#42726) gives better diagnostics on AMD Zen CPUs.

TPU, online quantization, and Triton

TPU users get tpu-inference upgraded to v0.22.1 (#45793). In the online/mixed-precision world, there's online FP8 per-token-per-channel (PTPC) quantization (#44132), so you can quantize weights per token per channel at serving time without a pre-pass. modelopt_mixed support is extended to Ampere/SM80-86 (#45306) and Turing/SM75 (#45375).

For Triton users, the new VLLM_TRITON_FORCE_FIRST_CONFIG flag (#42425) lets you skip Triton autotuning, which can cut first-request latency. There's also Triton recompile detection (#45631) and fused multi-group block-table staged writes (#44944). These are small but add up if you run many small requests.

Migration notes

I didn't see breaking changes that would force a config rewrite, but for any minor-version bump, do the usual: read the full release notes, update your image, run a quick smoke test with your model, and check the new token-tier limits don't interfere with existing templates. The cgroup KV cache change might shift your memory footprint, so re-check your container memory limits.

What I'd test first

  1. If you run ASR preprocessing, benchmark before/after — 2.5× is worth verifying with your audio pipeline.
  2. If you containerize with cgroup limits, test that the KV cache now stays inside the limit.
  3. Try the CondensePyramid V1 kernel on a sample multi-modal model and compare latency/quality.

Sources: