Create README.md

README.md

@@ -0,0 +1,109 @@
+---
+license: mit
+datasets:
+- observerw/ChiseLLM-Completion
+- observerw/ChiseLLM-Decompile
+base_model:
+- Qwen/Qwen2.5-Coder-7B-Instruct
+---
+
+# ChiseLLM Models
+
+<img src="https://raw.githubusercontent.com/observerw/ChiseLLM/refs/heads/main/assets/logo.svg" alt="ChiseLLM" style="width:30%">
+
+![GitHub Repo stars](https://img.shields.io/github/stars/observerw/ChiseLLM)
+
+ChiseLLM is a series of **large reasoning models specifically trained for the [Chisel Hardware Construction language](https://www.chisel-lang.org)**, aimed at revolutionizing HCL-Baed Agile Hardware Development Methodology (AHDM).
+
+Built on [Qwen/Qwen2.5-Coder-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) with domain-adaptive fine-tuning, the model combines high-quality reasoning datasets and specific thinking patterns to significantly enhance performance in hardware design tasks.
+
+ChiseLLM Models can:
+
+- **Transform natural language specifications into high-quality Chisel code** (Spec-to-Chisel)
+- **Intelligently translate Verilog code into enhanced Chisel implementations** (Decompile-to-Chisel)
+- **Generate hardware designs with superior variability and extensibility**, surpassing traditional design approaches
+
+### Use Cases
+
+ChiseLLM Models is particularly suited for the following applications:
+
+- **Rapid Hardware Design Prototyping**: Dramatically shortens the design cycle from specification to implementation
+- **Verilog Code Modernization**: Intelligently converts legacy Verilog code into extensible Chisel implementations
+- **Hardware Architecture Exploration**: Generates multiple design variants for the same functional requirements
+- **Design Refactoring and Optimization**: Leverages Chisel's advanced features to improve existing hardware designs
+- **Agile Hardware Development Education**: Serves as an assistive tool for learning Chisel and modern hardware design methods
+
+### Training results
+
+Spec-to-Chisel task on VerilogEval:
+
+| Models                         | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------------------------ | --------- | --------- | --------- | --------- |
+| Llama3.1-8B-Instruct           | 4.33      | 9.90      | 13.21     | 9.02      |
+| Qwen2.5-Coder-7B-Instruct      | 21.94     | 31.87     | 36.73     | 37.08     |
+| \*Deepseek-R1-Distill-Llama-8B | 9.31      | 15.44     | 17.72     | 16.01     |
+| \*ChiseLLM-7B                  | **29.41** | **47.08** | **54.04** | **58.82** |
+
+| Models                          | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------------------------- | --------- | --------- | --------- | --------- |
+| Qwen2.5-Coder-32B-Instruct      | 41.02     | 53.85     | 58.79     | 73.47     |
+| Qwen2.5-72B-Instruct            | 39.74     | 49.30     | 52.90     | 61.31     |
+| Llama-3.3-70B-Instruct          | 38.14     | 44.90     | 48.02     | 65.97     |
+| \*Deepseek-R1-Distill-Qwen-32B  | 38.50     | 54.58     | 61.16     | 52.19     |
+| \*Deepseek-R1-Distill-Llama-70B | 36.62     | 52.28     | 59.90     | 51.72     |
+| \*ChiseLLM-32B                  | **51.43** | **68.29** | **72.78** | **76.45** |
+
+| Models        | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------- | --------- | --------- | --------- | --------- |
+| Deepseek-V3   | 50.16     | 63.44     | 67.32     | 76.37     |
+| GPT-4o        | 42.04     | 60.16     | 65.17     | 69.76     |
+| \*Deepseek-R1 | **62.74** | **76.05** | **80.16** | **82.85** |
+
+Decompile-to-Chisel task on VerilogEval:
+
+| Models                         | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------------------------ | --------- | --------- | --------- | --------- |
+| Llama3.1-8B-Instruct           | 5.43      | 12.29     | 16.08     | 11.15     |
+| Qwen2.5-Coder-7B-Instruct      | 27.60     | 34.58     | 37.19     | 43.23     |
+| \*Deepseek-R1-Distill-Llama-8B | 10.05     | 16.15     | 18.13     | 12.03     |
+| \* ChiseLLM-7B                 | **50.47** | **70.99** | **78.08** | **59.19** |
+
+| Models                          | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------------------------- | --------- | --------- | --------- | --------- |
+| Qwen2.5-Coder-32B-Instruct      | 41.19     | 48.96     | 51.59     | 53.93     |
+| Qwen2.5-72B-Instruct            | 40.54     | 47.32     | 49.83     | 59.30     |
+| Llama-3.3-70B-Instruct          | 38.38     | 46.96     | 51.36     | 48.00     |
+| \*Deepseek-R1-Distill-Qwen-32B  | 45.03     | 63.02     | 70.18     | 53.17     |
+| \*Deepseek-R1-Distill-Llama-70B | 37.50     | 55.05     | 63.84     | 45.59     |
+| \*ChiseLLM-32B                  | **56.41** | **72.00** | **77.67** | **64.71** |
+
+| Models        | pass@1    | pass@3    | pass@5    | syntax(%) |
+| ------------- | --------- | --------- | --------- | --------- |
+| Deepseek-V3   | **54.57** | 63.19     | 66.71     | **66.19** |
+| GPT-4o        | 42.39     | 65.75     | 71.83     | 53.77     |
+| \*Deepseek-R1 | 53.45     | **71.50** | **77.91** | 59.13     |
+
+### Framework versions
+
+- Transformers 4.51.0
+- Pytorch 2.6.0a0+df5bbc09d1.nv24.12
+- Datasets 3.4.1
+- Tokenizers 0.21.0
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+
+- learning_rate: 1e-05
+- train_batch_size: 2
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 8
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 128
+- total_eval_batch_size: 64
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.05
+- num_epochs: 3.0