metadata

license: mit
library_name: vllm
base_model:
  - deepseek-ai/DeepSeek-R1-0528
pipeline_tag: text-generation
tags:
  - deepseek
  - neuralmagic
  - redhat
  - llmcompressor
  - quantized
  - INT4
  - GPTQ

DeepSeek-R1-0528-quantized.w4a16

More evals coming soon

unquantized baseline on GSM8k

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9591|±  |0.0055|
|     |       |strict-match    |     5|exact_match|↑  |0.9568|±  |0.0056|

this INT4 quantized model on GSM8k

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9560|±  |0.0056|
|     |       |strict-match    |     5|exact_match|↑  |0.9553|±  |0.0057|