NVIDIA GeForce RTX 5090

## Overview This PR adds **optional KV compression to Mooncake P/D transfer** through a versioned interface to [KVServe](https://github.com/hpdps-group/KVServe) (SIGCOMM 2026), implementing the fix

{"text":"Llama-3.1-8B-Instruct, Mooncake TCP, 25 Gb/s link, concurrency 8, max_tokens=32"}

## Overview Apply the loaded LM-head bias when computing Moondream3 and PhiMoE logits. ## Claims - Preserve the checkpoint's output projection, including its bias. - Reuse the logits processor's ex

{"text":"Moondream3とPhi-3.5-MoEモデルの推論検証、BF16 eager execution環境"}

## Purpose `CompressedTensorsW4A4Fp4.process_weights_after_loading` runs a fused NVFP4 linear (`qkv_proj`, `gate_up_proj`, Qwen3.5's `in_proj_qkvz` / `in_proj_ba`, ...) with the largest `weight_globa

{"text":"NVFP4量子化モデル (Qwen3.8-27B) を FlashInfer CUTLASS カーネルで実行し、fp32参照値とのコサイン類似度が0.999995以上であることを確認。"}

## Overview Preserve requested `token_ids` in chat derender responses, including plain and parsed streaming. Follow normal chat behavior when reasoning is hidden. ## Claims - The render → generate

{"text":"Qwen3-0.6BモデルをvLLMでサーブし、トークンIDの保持を確認"}

## Overview Allow Anthropic `/v1/messages/count_tokens` to count prompts containing tools without requiring an output tool parser. Keep parser validation on generation requests. ## Claims - Countin

{"text":"Qwen3-0.6BモデルでvLLMサーバーを起動し、ツール付きプロンプトのトークンカウントがHTTP 200で正常に動作することを確認。"}

## Overview Finish pooling requests cleanly when the engine aborts them, such as during `pause_generation(mode="abort")`. A tensor-less ABORT currently falls into the text-output path and kills the o

{"text":"Qwen3-Embedding-0.6Bモデルを使用し、64並列のエンベディングリクエストで検証。修正後はキャンセルされたリクエストを含む全リクエストが正常に完了した。"}

### Your current environment - vLLM `0.31.1rc1.dev159+g4274ae956` (main @ 4274ae956c), precompiled wheel. The code involved is unchanged on current main (3c062d391d); the links below point to 276fbcf

{"text":"Gemma-4-26B-A4B-NVFP4 モデル、CUDA graph capture、4096トークンバッファ、32フレームの動画入力"}

### Name and Version llama-server 0.6.0, llama-server 0.5.0 ### Operating systems Windows ### GGML backends CUDA ### Hardware Intel Core Ultra 9 285H RTX 5090 Mobile NVIDIA Studio Drivers 616.

{"text":"MTPモデルのロード時にBSOD発生"}

General perf improvements for qwen 3.x LLM. First round of a major series of accelerations for the qwen 3.x family of LLMs. Comfy kitchen PR: https://github.com/Comfy-Org/comfy-kitchen/pull/239 Ac

{"text":"qwen 3.8 w4a8, 96GB VRAM, creative writing with image input"}

Unify MTP head logic with model-prefetcher and remove the base level CUDAGraph generator. This fixes an issue where unconditional CUDAGraphs will graph dynamically migrated weights or an offload cast

{"text":"qwen 3.8 w4a8 MTP text generate, 96GB VRAM, --vram-headroom 17"}

## Overview This is a graph only change proposed in #28841. This patch speeds up token output speed by removing the state copy steps. This does not affect anything beside the get_rows copies, as we

{"text":"Various models tested with different quantizations on CUDA 13.4"}

**Proof of concept, not for merge as-is.** It touches the Qwen3.5 generation code that #16820 is changing. An agent loop resends its whole conversation every round, and `TextGenerate` re-prefills all

{"text":"Qwen3.8 27B, RTX 5090 32 GB, prefix cache enabled"}

https://github.com/Comfy-Org/ComfyUI/issues/16856 We have too long a legacy of custom nodes overriding the Linear API and there is an internal overrider of Linear.forward setting a precedent. Re

{"text":"Linux, RTX5090, 96GB, Wan 2.1 T2V 14B"}

## Overview Fixes two bugs in the Vulkan `TOP_K` shader (`topk_nary_search.comp`). **+inf / NaN inputs.** The shader finds the k-th largest value of each workgroup's block by mapping the floats

{"text":"MTPモデル、parallel: 1〜2、Vulkanバックエンド"}

### Name and Version Clean master `c479922ac` (2026-10-06). CUDA 13.3, `-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_NATIVE=ON`. ### Operating systems Linux ### GGML backends CUDA ### Ha

{"text":"CUDA 13.3, glm5-next, DFlash drafter, 2x RTX 5090"}

## Overview The vector FA kernel with q4_0 or q5_0 V cache gets a 128 to 336 byte stack frame on CUDA 12.8 for Blackwell (sm_120a, sm_100a, sm_101a). Older architectures and CUDA 12.9 and 13.0 are

{"text":"CUDA 12.8.1, q4_0 cache, 16k-32k context"}

### Name and Version ./llama-cli --version version: 0.5.0-dev (build 11379, commit 1537a0a8b) built with GNU 15.2.0 for Linux x86_64 ### Operating systems Linux ### GGML backends CUDA ### Hardwa

{"text":"GLM-5.3-Flash-Q5_K_Mモデルでコンテキスト307200、ubatch 8192設定時にOOMが発生"}

Extends the gate/up + GLU fusion, which today only covers small batches (MMVQ/MMVF), to prompt-processing batch sizes on the MMQ path. Alternative to #28702 implementing @am17an's suggestion there

{"text":"llama.cppでのprompt processing (Qwen3.8-27B, Llama-3.1-8B等) において、master比で2.0%〜10.7%の性能向上を確認。"}

Extends the gate/up + GLU fusion, which today only covers small batches (MMVQ/MMVF), to prompt-processing batch sizes on the MMQ path. Alternative to #28702 implementing @am17an's suggestion there

{"text":"Q4_K_M, Q5_K_M, Q6_K, Q8_0, IQ4_XS, tokens/sec improvement"}

## Overview This tries to use the 256x128 shape (coopmat2-only) with split_k when it appears profitable. It reuses the mul_mat split_k reduce shader. This should be stacked on #29920, but I gues

{"text":"Vulkanバックエンドでのconv3d split_k最適化の実装と、RTX 5090上での性能ベンチマーク結果の報告。"}