GGUF Quantum model to be used in llama.cpp after:

https://github.com/ggerganov/llama.cpp/issues/6747

Downloads last month
25
GGUF
Model size
8.03B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support