NVIDIA GeForce RTX 5070 Ti

Fixes #16551. With dynamic VRAM, `fp8_linear` never takes the FP8 matmul path for `weight_dtype = fp8_e4m3fn_fast` on a bf16 checkpoint. The weight is still in its file dtype (bf16, on CPU) when `f

{"text":"RTX 5070 Ti, fp8_e4m3fn_fast, 1024², 20 steps"}

## Overview I found the issue with the GPU **round(x)** code , when llama.cpp was compiled on windows with MSVC . With the MSVC that **round(x)** call compiled differently for GPU and for CPU and g

{"text":"f16 and bf16 tests passed after roundf(x) fix"}

## Overview Fixes #30000. Passes the quant block size directly as a specialization constant, restoring compile-time K-loop unrolling for unified CM2 quant shaders. <!-- Describe what this PR does and

{"text":"Vulkan on Windows, various quantization levels"}

### Problem `SCAIL2ColoredMask` is very slow on long videos. `_render_colored_masks` and `_first_appearance_cx_area` run entirely on `intermediate_device()`, which is the CPU unless `--gpu-only` is s

{"text":"PyTorch 2.14.0+cu130, 1280x720, 900 frames, 2 objects"}

## Summary `vllm/vllm-openai:0.24.0` fails to load `Vishva007/gemma-4-E4B-it-W4A16-AutoRound-GPTQ`: engine init dies in weight loading because vLLM looks for an **unquantized** `audio_tower...linear.

{"text":"vLLM 0.24.0でGemma 4 E4B AutoRound-GPTQモデルの読み込み失敗（audio_towerの量子化パラメータ処理エラー）"}

## Overview `ggml_vk_get_device` published the new device into `vk_instance.devices[idx]` before building it, with no lock around the slot. A second thread entering during the build sees a non-null

{"text":"Vulkan driver on Windows 11, 2 threads calling ggml_backend_vk_init(0)"}

### What is the issue? On Windows, Ollama suddenly stopped detecting my NVIDIA GPU and fell back to CPU. The failure was ultimately traced to a missing `ggml-cuda.dll` in the installed CUDA backend a

{"text":"NVIDIA GeForce RTX 5070 Ti, Ollama 0.35.0, ggml-cuda.dll restored after reinstall"}

### What is the issue? On Windows, Ollama suddenly stopped detecting my NVIDIA GPU and fell back to CPU. The failure was ultimately traced to a missing `ggml-cuda.dll` in the installed CUDA backend a

{"text":"Windows自動更新によりggml-cuda.dllが.tmpとして残存し、GPU検出失敗してCPUフォールバックが発生"}

### Name and Version Running Docker image: ghcr.io/ggml-org/llama.cpp:server-cuda13-b11151 ### Operating systems Linux ### GGML backends CUDA ### Hardware 1 x RTX 5070 TI (16 Gb) and 1 x RTX 50

{"text":"Gemma 4 31B QAT Q4_0モデルで、--split-mode tensorと不均等な--tensor-split (2.2,1)を使用するとGGML_ASSERTエラーが発生する。"}

### Custom Node Testing - [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-wi

{"text":"Flux devモデルで、Dynamic VRAM有効時はFP8計算が効かず1.07 it/s、無効化すると1.72 it/sになる"}

### Custom Node Testing - [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-wi

{"text":"FP8 weight_dtype, Flux dev model, 1024², 20 steps"}

### Name and Version llama.cpp `84e76d8a2` (2026-09-24); the kernel is unchanged on `master` at `8212c7802` (`ggml/src/ggml-cuda/unary.cu`, `op_round` returns `round(x)`). Also seen on `e613ef2c8`

{"text":"CUDA 13.4 / MSVC 19.51環境下で、f16のROUND演算がhalf-to-evenに丸められtest-backend-opsが失敗"}

### Name and Version llama.cpp `84e76d8a2` (2026-09-24); the kernel is unchanged on `master` at `8212c7802` (`ggml/src/ggml-cuda/unary.cu`, `op_round` returns `round(x)`). Also seen on `e613ef2c8`

{"text":"CUDA 13.4, f16 ROUND test cases failed"}

### Name and Version llama.cpp `84e76d8a2` (2026-09-24); the kernel is unchanged on `master` at `8212c7802` (`ggml/src/ggml-cuda/unary.cu`, `op_round` returns `round(x)`). Also seen on `e613ef2c8`

{"text":"CUDA backendでのtest-backend-ops実行時、f16 ROUNDケースがMSVCコンパイラ環境で失敗"}

## Overview Vulkan MMVQ row geometry affects small-batch performance on NVIDIA. This PR changes only the `MUL_MAT_ID` (`rm_id`) policy: NVIDIA devices not classified as `NVIDIA_PRE_TURING` use 4

{"text":"RTX 5070 Ti 16 GB, Qwen3-16B-A3B Q4_0/Q4_1/Q5_0, 31% expert offload"}

### Name and Version llama-server Version: 0.4.1-dev (build 10964, commit b29c606e2) built with MSVC 19.51.36256.0 for x64 ### Operating systems Windows ### GGML backends CUDA ### Hardware AMD

{"text":"Qwen3.8-27Bモデルを2枚のRTX 5070 Tiで実行中、/metricsエンドポイントがVictoriaMetricsによって1秒ごとにスクレイピングされると、プロンプト処理が20秒以上かかる場合にサーバーがサイレントに停止する。"}

### Name and Version llama-server Version: 0.4.1-dev (build 10964, commit b29c606e2) built with MSVC 19.51.36256.0 for x64 ### Operating systems Windows ### GGML backends CUDA ### Hardware AMD

{"text":"Qwen3.8-27Bモデルを2枚のRTX 5070 Tiで実行中、/metricsエンドポイントがVictoriaMetricsによって1秒ごとにスクレイピングされた際、プロンプト処理が20秒以上続くとサーバーがサイレントに停止しGPU負荷が0になる。"}

### What is the issue? ollama run qwen3-coder:30b terminates with exit status 0xc0000409 on an RTX 5070 Ti Laptop (Blackwell, sm_120). The error surfaces as CUDA error: shared object initialization f

{"text":"RTX 5070 Ti, Flash Attention enabled, crash during warmup"}

### Name and Version version: 0.4.0-dev (build 10825, commit 9e0e22059) built with GNU 13.3.0 for Linux x86_64 ### Operating systems Linux ### Which llama.cpp modules do you know to be affected?

{"text":"Qwen3.8-27B (Q5_K_M) を2枚のRTX 5070 Tiに分割配置しようとしたが、tensor splitの粒度問題によりOOMが発生"}

## Overview Adds a brief note to the -ctkd and -ctvd docs, pointing out that quantization of the MTP cache can unintuitively increase memory usage. Users of llama.cpp on low-VRAM systems are very f

{"text":"Qwen3.8-27B (Q3_K_XL) モデルを 16GB VRAM で動作させ、MTP KV キャッシュの量子化がメモリ使用量を増加させることを確認。"}