Repository navigation

ggml-cuda: Q8_0 weights take dequant-to-BF16 + cuBLAS by default; k-quants stay on MMQ - #39

Open

danielhanchen wants to merge 4 commits into

cuda-q8-bf16-cublas

Open

ggml-cuda: Q8_0 weights take dequant-to-BF16 + cuBLAS by default; k-quants stay on MMQ#39
danielhanchen wants to merge 4 commits into
masterfrom
cuda-q8-bf16-cublas

Starting job