OPEA
/

Mistral-Small-3.1-24B-Instruct-2503-int4-AutoRound-gptq-sym

4-bit precision

Model card Files Files and versions Community

cicdatopea commited on Mar 19

Commit

158d8b8

·

verified ·

1 Parent(s): 21b90db

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -105,12 +105,12 @@ print(decoded)
 ## Generate the model
 Here is the sample command to reproduce the model.
 ```bash
-pip install auto-round
 auto-round-mllm \
 --model mistralai/Mistral-Small-3.1-24B-Instruct-2503 \
 --device 0 \
 --bits 4 \
---format 'auto_gptq' \
 --output_dir "./tmp_autoround"
 ```

 ## Generate the model
 Here is the sample command to reproduce the model.
 ```bash
+pip install git+https://github.com/intel/auto-round.git@main
 auto-round-mllm \
 --model mistralai/Mistral-Small-3.1-24B-Instruct-2503 \
 --device 0 \
 --bits 4 \
+--format 'auto_awq,auto_gptq' \
 --output_dir "./tmp_autoround"
 ```