uploaded readme
Browse files
README.md
ADDED
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
Quantization made by Richard Erkhov.
|
2 |
+
|
3 |
+
[Github](https://github.com/RichardErkhov)
|
4 |
+
|
5 |
+
[Discord](https://discord.gg/pvy7H8DZMG)
|
6 |
+
|
7 |
+
[Request more models](https://github.com/RichardErkhov/quant_request)
|
8 |
+
|
9 |
+
|
10 |
+
NorLlama-3B - bnb 4bits
|
11 |
+
- Model creator: https://huggingface.co/NorGLM/
|
12 |
+
- Original model: https://huggingface.co/NorGLM/NorLlama-3B/
|
13 |
+
|
14 |
+
|
15 |
+
|
16 |
+
|
17 |
+
Original model description:
|
18 |
+
---
|
19 |
+
license: cc-by-nc-sa-4.0
|
20 |
+
language:
|
21 |
+
- 'no'
|
22 |
+
---
|
23 |
+
|
24 |
+
Gnerative Pretrained Tranformer with 3 Billion parameters for Norwegian. NorLlama-3B is based on Llama architechture, and pretrained on [Tencent Pre-training Framework](https://github.com/Tencent/TencentPretrain)
|
25 |
+
|
26 |
+
It belongs to NorGLM, a suite of pretrained Norwegian Generative Language Models. NorGLM can be used for non-commercial purposes.
|
27 |
+
|
28 |
+
## Datasets
|
29 |
+
All models in NorGLM are trained on 200G datasets, nearly 25B tokens, including Norwegian, Denish, Swedish, Germany and English.
|
30 |
+
|
31 |
+
## Run the Model
|
32 |
+
|
33 |
+
```python
|
34 |
+
import torch
|
35 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
36 |
+
|
37 |
+
model_id = "NorGLM/NorLlama-3B"
|
38 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
39 |
+
model = AutoModelForCausalLM.from_pretrained(
|
40 |
+
model_id,
|
41 |
+
device_map='auto',
|
42 |
+
torch_dtype=torch.bfloat16
|
43 |
+
)
|
44 |
+
|
45 |
+
text = "Tom ønsket å gå på barene med venner"
|
46 |
+
inputs = tokenizer(text, return_tensors="pt")
|
47 |
+
outputs = model.generate(**inputs, max_new_tokens=20)
|
48 |
+
```
|
49 |
+
|
50 |
+
## Citation Information
|
51 |
+
If you feel our work is helpful, please cite our paper:
|
52 |
+
|
53 |
+
```
|
54 |
+
@article{liu2023nlebench+,
|
55 |
+
title={NLEBench+ NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian},
|
56 |
+
author={Liu, Peng and Zhang, Lemei and Farup, Terje Nissen and Lauvrak, Even W and Ingvaldsen, Jon Espen and Eide, Simen and Gulla, Jon Atle and Yang, Zhirong},
|
57 |
+
journal={arXiv preprint arXiv:2312.01314},
|
58 |
+
year={2023}
|
59 |
+
}
|
60 |
+
```
|
61 |
+
|