README.md

---
license: mit
datasets:
- observerw/ChiseLLM-Completion
- observerw/ChiseLLM-Decompile
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
---

# ChiseLLM Models

<img src="https://raw.githubusercontent.com/observerw/ChiseLLM/refs/heads/main/assets/logo.svg" alt="ChiseLLM" style="width:30%">

[GitHub](https://github.com/observerw/ChiseLLM)

ChiseLLM is a series of **large reasoning models specifically trained for the [Chisel Hardware Construction language](https://www.chisel-lang.org)**, aimed at revolutionizing HCL-Baed Agile Hardware Development Methodology (AHDM).

Built on [Qwen/Qwen2.5-Coder-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) with domain-adaptive fine-tuning, the model combines high-quality reasoning datasets and specific thinking patterns to significantly enhance performance in hardware design tasks.

ChiseLLM Models can:

- **Transform natural language specifications into high-quality Chisel code** (Spec-to-Chisel)
- **Intelligently translate Verilog code into enhanced Chisel implementations** (Decompile-to-Chisel)
- **Generate hardware designs with superior variability and extensibility**, surpassing traditional design approaches

### Use Cases

ChiseLLM Models is particularly suited for the following applications:

- **Rapid Hardware Design Prototyping**: Dramatically shortens the design cycle from specification to implementation
- **Verilog Code Modernization**: Intelligently converts legacy Verilog code into extensible Chisel implementations
- **Hardware Architecture Exploration**: Generates multiple design variants for the same functional requirements
- **Design Refactoring and Optimization**: Leverages Chisel's advanced features to improve existing hardware designs
- **Agile Hardware Development Education**: Serves as an assistive tool for learning Chisel and modern hardware design methods

### Training results

Spec-to-Chisel task on VerilogEval:

| Models                         | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------------------------ | --------- | --------- | --------- | --------- |
| Llama3.1-8B-Instruct           | 4.33      | 9.90      | 13.21     | 9.02      |
| Qwen2.5-Coder-7B-Instruct      | 21.94     | 31.87     | 36.73     | 37.08     |
| \*Deepseek-R1-Distill-Llama-8B | 9.31      | 15.44     | 17.72     | 16.01     |
| \*ChiseLLM-7B                  | **29.41** | **47.08** | **54.04** | **58.82** |

| Models                          | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------------------------- | --------- | --------- | --------- | --------- |
| Qwen2.5-Coder-32B-Instruct      | 41.02     | 53.85     | 58.79     | 73.47     |
| Qwen2.5-72B-Instruct            | 39.74     | 49.30     | 52.90     | 61.31     |
| Llama-3.3-70B-Instruct          | 38.14     | 44.90     | 48.02     | 65.97     |
| \*Deepseek-R1-Distill-Qwen-32B  | 38.50     | 54.58     | 61.16     | 52.19     |
| \*Deepseek-R1-Distill-Llama-70B | 36.62     | 52.28     | 59.90     | 51.72     |
| \*ChiseLLM-32B                  | **51.43** | **68.29** | **72.78** | **76.45** |

| Models        | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------- | --------- | --------- | --------- | --------- |
| Deepseek-V3   | 50.16     | 63.44     | 67.32     | 76.37     |
| GPT-4o        | 42.04     | 60.16     | 65.17     | 69.76     |
| \*Deepseek-R1 | **62.74** | **76.05** | **80.16** | **82.85** |

Decompile-to-Chisel task on VerilogEval:

| Models                         | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------------------------ | --------- | --------- | --------- | --------- |
| Llama3.1-8B-Instruct           | 5.43      | 12.29     | 16.08     | 11.15     |
| Qwen2.5-Coder-7B-Instruct      | 27.60     | 34.58     | 37.19     | 43.23     |
| \*Deepseek-R1-Distill-Llama-8B | 10.05     | 16.15     | 18.13     | 12.03     |
| \* ChiseLLM-7B                 | **50.47** | **70.99** | **78.08** | **59.19** |

| Models                          | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------------------------- | --------- | --------- | --------- | --------- |
| Qwen2.5-Coder-32B-Instruct      | 41.19     | 48.96     | 51.59     | 53.93     |
| Qwen2.5-72B-Instruct            | 40.54     | 47.32     | 49.83     | 59.30     |
| Llama-3.3-70B-Instruct          | 38.38     | 46.96     | 51.36     | 48.00     |
| \*Deepseek-R1-Distill-Qwen-32B  | 45.03     | 63.02     | 70.18     | 53.17     |
| \*Deepseek-R1-Distill-Llama-70B | 37.50     | 55.05     | 63.84     | 45.59     |
| \*ChiseLLM-32B                  | **56.41** | **72.00** | **77.67** | **64.71** |

| Models        | pass@1    | pass@3    | pass@5    | syntax(%) |
| ------------- | --------- | --------- | --------- | --------- |
| Deepseek-V3   | **54.57** | 63.19     | 66.71     | **66.19** |
| GPT-4o        | 42.39     | 65.75     | 71.83     | 53.77     |
| \*Deepseek-R1 | 53.45     | **71.50** | **77.91** | 59.13     |

### Framework versions

- Transformers 4.51.0
- Pytorch 2.6.0a0+df5bbc09d1.nv24.12
- Datasets 3.4.1
- Tokenizers 0.21.0

### Training hyperparameters

The following hyperparameters were used during training:

- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 8
- total_train_batch_size: 128
- total_eval_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 3.0

## Citation

If you are interested in our work, please consider citing this, it would be greatly appreciated!

```bibtex
@misc{wang2025chisellmunleashingpowerreasoning,
      title={ChiseLLM: Unleashing the Power of Reasoning LLMs for Chisel Agile Hardware Development}, 
      author={Bowei Wang and Jiaran Gao and Yelai Feng and Renzhi Chen and Shanshan Li and Lei Wang},
      year={2025},
      eprint={2504.19144},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2504.19144}, 
}
```