Hugging Face's logo

tokyotech-llm
/
Llama-3.1-Swallow-8B-Instruct-v0.1

Text Generation
Transformers
Safetensors
English
Japanese
llama
conversational
text-generation-inference
Model card Files Community
Taishi-N324 commited on
Commit
ca04291
·
verified ·
1 Parent(s): 64274ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -83,13 +83,13 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/) pro
83
  | Model | coding | extraction | humanities | math | reasoning | roleplay | stem | writing | JMTAvg |
84
  |-------|--------|------------|------------|------|-----------|----------|------|---------|--------|
85
  | RakutenAI-7B-chat | 0.2475 | 0.3522 | 0.4692 | 0.2140 | 0.3926 | 0.4427 | 0.3977 | 0.4434 | 0.3699 |
86
- | Qwen2-7B-Instruct | 0.4635 | 0.6909 | 0.6857 | 0.5970 | 0.5042 | 0.6667 | 0.5353 | 0.6808 | 0.6030 |
87
- | Qwen2.5-7B-Instruct | **0.5111** | 0.7489 | 0.6913 | 0.5742 | 0.4851 | 0.6810 | 0.5350 | 0.6810 | **0.6134** |
88
  | Tanuki-8B-dpo-v1.0 | 0.3019 | 0.4772 | 0.5658 | 0.4129 | 0.3590 | 0.5120 | 0.4770 | 0.6159 | 0.4652 |
89
  | Llama 3 8B Instruct | 0.3744 | 0.6876 | 0.6225 | 0.2070 | 0.5032 | 0.5248 | 0.5326 | 0.4884 | 0.4926 |
90
  | Llama 3.1 8B Instruct | 0.3234 | 0.7362 | 0.4973 | 0.4787 | 0.3210 | 0.4670 | 0.4656 | 0.4314 | 0.4651 |
91
- | Llama 3 Youko 8B Instruct | 0.2950 | 0.7332 | **0.7125** | 0.2533 | 0.4987 | 0.6514 | 0.5438 | **0.7091** | 0.5496 |
92
- | Llama-3-ELYZA-JP-8B | 0.2908 | 0.6421 | 0.6406 | 0.3088 | **0.5500** | **0.6740** | 0.5251 | 0.6744 | 0.5382 |
93
  | Llama 3 heron brain 8B v0.3 | 0.2929 | 0.5635 | 0.6241 | 0.2135 | 0.4582 | 0.5354 | 0.5273 | 0.5099 | 0.4656 |
94
  | Llama 3 Swallow 8B Instruct | 0.3547 | 0.6508 | 0.5371 | 0.2718 | 0.4007 | 0.5493 | 0.4752 | 0.5730 | 0.4766 |
95
  | Llama 3.1 Swallow 8B Instruct | 0.3132 | **0.7734** | 0.6645 | 0.3880 | 0.5230 | 0.5711 | 0.4953 | 0.5330 | 0.5327 |
 
83
  | Model | coding | extraction | humanities | math | reasoning | roleplay | stem | writing | JMTAvg |
84
  |-------|--------|------------|------------|------|-----------|----------|------|---------|--------|
85
  | RakutenAI-7B-chat | 0.2475 | 0.3522 | 0.4692 | 0.2140 | 0.3926 | 0.4427 | 0.3977 | 0.4434 | 0.3699 |
86
+ | Qwen2-7B-Instruct | 0.4635 | 0.6909 | 0.6857 | **0.5970** | 0.5042 | 0.6667 | 0.5353 | 0.6808 | 0.6030 |
87
+ | Qwen2.5-7B-Instruct | **0.5111** | 0.7489 | 0.6913 | 0.5742 | 0.4851 | **0.6810** | 0.5350 | 0.6810 | **0.6134** |
88
  | Tanuki-8B-dpo-v1.0 | 0.3019 | 0.4772 | 0.5658 | 0.4129 | 0.3590 | 0.5120 | 0.4770 | 0.6159 | 0.4652 |
89
  | Llama 3 8B Instruct | 0.3744 | 0.6876 | 0.6225 | 0.2070 | 0.5032 | 0.5248 | 0.5326 | 0.4884 | 0.4926 |
90
  | Llama 3.1 8B Instruct | 0.3234 | 0.7362 | 0.4973 | 0.4787 | 0.3210 | 0.4670 | 0.4656 | 0.4314 | 0.4651 |
91
+ | Llama 3 Youko 8B Instruct | 0.2950 | 0.7332 | **0.7125** | 0.2533 | 0.4987 | 0.6514 | **0.5438** | **0.7091** | 0.5496 |
92
+ | Llama-3-ELYZA-JP-8B | 0.2908 | 0.6421 | 0.6406 | 0.3088 | **0.5500** | 0.6740 | 0.5251 | 0.6744 | 0.5382 |
93
  | Llama 3 heron brain 8B v0.3 | 0.2929 | 0.5635 | 0.6241 | 0.2135 | 0.4582 | 0.5354 | 0.5273 | 0.5099 | 0.4656 |
94
  | Llama 3 Swallow 8B Instruct | 0.3547 | 0.6508 | 0.5371 | 0.2718 | 0.4007 | 0.5493 | 0.4752 | 0.5730 | 0.4766 |
95
  | Llama 3.1 Swallow 8B Instruct | 0.3132 | **0.7734** | 0.6645 | 0.3880 | 0.5230 | 0.5711 | 0.4953 | 0.5330 | 0.5327 |