NVIDIA GeForce RTX 4090

## Overview With `kv_buffer_device="cpu"` on CUDA, NIXL P/D stages KV through a host buffer, and both host-buffer copies go through `copy_kv_blocks`. In a serving process these copies run at 1.2–2.

{"text":"4x RTX 4090環境でQwen3-0.6Bモデルを使用し、KVキャッシュのコピー速度向上とレイテンシ低減を確認"}

### Is there an existing issue for this problem? - [x] I have searched the existing issues ### Install method Manual ### Operating system Windows ### GPU vendor Nvidia (CUDA) ### GPU model RT

{"text":"FLUX.1 dev, denoising_start=0.6, add_noise=false, no mask, 1024²"}

### Your current environment - vLLM: `0.31.0` (`vllm 0.31.0+cu129`) - torch: `2.13.0+cu129`, Triton as shipped with the wheel, Python 3.12.15 - GPU: 1x NVIDIA GeForce RTX 4090 (SM89), driver 535.288.

{"text":"vLLM 0.31.0環境でQwen2.5-0.5B-Instructモデルを使用し、SamplingParamsにfloat値を渡した際にLLM.generate()がハングする"}

## Summary Bumps `diffusers` from 0.40.0 to 0.41.0. This is the dependency step for Qwen-Image-2.1 support: 0.41.0 adds `QwenImage21Pipeline`, `QwenImage21Transformer2DModel` and `AutoencoderKLQwenIm

{"text":"same-seed GPU smoke generations, diffusers 0.41.0"}

> **Stacked on #9709, which is stacked on #9708.** Until those merge, this PR also shows their commits. The change here is the last commit, `feat(ernie-image): load GGUF transformers`. ## Summary ER

{"text":"ERNIE-Image Turbo, 1024², 8 steps, Q4_K_M quantization"}

## Summary Ideogram 4 GGUF transformers did not install: the single-file config refused GGUF state dicts ("not supported yet"). They now install and generate through the same path as the safetensors

{"text":"RTX 4090, 1024², 12 steps, GGUF Q4_0 and Q5_K"}

## Overview <!-- Describe what this PR does and why. Be concise but complete --> Cont https://github.com/ggml-org/llama.cpp/pull/29887, adding multi-GPU support. Benchmarks on 2x 4090s ##

{"text":"2x RTX 4090, Qwen3.8-Flash-Next Q4_0, --moe-cache-mib 6544"}

## Summary ComfyUI-GGUF (molbal/ComfyUI-GGUF) writes a `Q8_CR` format: ComfyUI's `int8_tensorwise` + convrot weights inside a GGUF, stored as GGML type `I8` with an F32 `<weight>_scale` beside each

{"text":"13.8 GB Q8_CR Krea-2 finetune, 1024², 8 steps, ~1.4 s/it"}

### Is there an existing issue for this problem? - [x] I have searched the existing issues ### Install method Invoke's Launcher ### Operating system Windows ### GPU vendor Nvidia (CUDA) ### GP

{"text":"Kreation v3 checkpoint for Krea 2"}

The XIELU CUDA kernel template is already generic over the element type; only the F32/F16 type assertion and the else-if dispatch were missing. Add the nv_bfloat16 branch to the launcher, and drop the

{"text":"XIELUカーネルのBF16対応テストで、F32/F16/BF16の全ケースが成功（6/6）。"}

## Summary Follow-up to #9654, from its review. The commit was pushed after the PR had merged. - **Video decode tiling:** `wan_latents_to_video` used to tile whenever the untiled or streaming estima

{"text":"RTX 4090上でbf16、832x480x17の解像度でWan VAEのデコードテストが成功"}

## Summary Follow-up to #9654, from its review. The commit was pushed after the PR had merged. - **Video decode tiling:** `wan_latents_to_video` used to tile whenever the untiled or streaming estima

{"text":"bf16, 832x480x17, A14B VAE, TI2V VAE"}

### Name and Version llama.cpp b10361 (via the node-llama-cpp 3.20.0 Vulkan prebuilt). Linux 7.2, NVIDIA RTX 4090, driver 615.71.09 (open kernel module). ### Operating systems Linux ### GGML backe

{"text":"NVIDIA RTX 4090 (OC/undervolted) with Vulkan backend; sustained decoding caused Xid 109 error leading to infinite spin in ggml_vk_wait_for_fence."}

## Summary At high resolutions and for video, most of a diffusion step is attention, and Windows builds of PyTorch have no flash-attention kernel. This adds an opt-in `attention_backend: sage` settin

{"text":"RTX 4090, 2048², 15–35% faster for video models"}

## Summary At high resolutions and for video, most of a diffusion step is attention, and Windows builds of PyTorch have no flash-attention kernel. This adds an opt-in `attention_backend: sage` sett

{"text":"Windows環境下でSageAttention 2.2を使用し、PyTorchのSDPA比で2.0-2.9倍高速化、生成速度は3-35%向上"}

## Overview Release SimpleCPUOffload lookup-only CPU block references during cache reset. A request waiting for GPU allocation can otherwise prevent `reset_prefix_cache(reset_connector=True)` from

{"text":"Qwen2.5-0.5B-Instructモデル、FP16、TP=1、eager実行モードでKVキャッシュリセットの動作確認"}

## Overview Adds GGML_TYPE_BF16X (type 43), a fixed-rate near-lossless bfloat16 recompression at **11.5 bpw** (46 bytes per 32 weights, 1.39x smaller than BF16), plus CPU and GPU execution. Per 32-w

{"text":"RTX 4090上でMiniCPM5-1BモデルをBF16X量子化し、CUDAバックエンドで動作確認。PPLはF16とほぼ同等（21.0076 vs 21.0155）、生成速度は約306 tok/s。"}

## Summary **Bug: VAE decodes hanging on Windows/NVIDIA.** After a few generations across architectures, or just repeating FLUX.1, VAE decodes took between 67 s and more than 25 minutes at 100% GPU,

{"text":"Windows環境でSDXLからFLUX.1、Z-Image、Krea-2など複数アーキテクチャを連続生成し、VAEデコードのハングを解消して安定動作を確認。"}

## Summary On 16 GB cards, repeating a Wan video generation runs out of memory, or slows to a crawl while the GPU spills into shared memory. The model cache keeps exactly a node's working-memory est

{"text":"Wan video generation (TI2V) on 16GB cards; calibrated for RTX 4090 and RX 9060 XT to prevent OOM."}

The decision model `clef-flash` (Q8_0, 9.1B) always fails on the `/v1/systemone` endpoint. The same model works fine on `/v1/chat/completions`, and the larger `clef:27b` (Q4_K_M) works on `/v1/systemo

{"text":"Ollama 0.35.1環境でclef-flash (Q8_0, 9.1B)モデルを/v1/systemoneエンドポイントで実行した際、CUDAバックエンドでは'non-finite logit'、CPUバックエンドでは'cannot open model'エラーが発生し失敗した。"}