Commit
ยท
231cbd0
1
Parent(s):
5ee5911
Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,29 @@
|
|
1 |
# Fauno - Italian LLM
|
2 |
|
3 |

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
## Model initialization
|
5 |
```python
|
6 |
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
|
|
|
1 |
# Fauno - Italian LLM
|
2 |
|
3 |

|
4 |
+
|
5 |
+
Get ready to meet Fauno - the Italian language model crafted by the [RSTLess Research Group](https://rstless-lab.netlify.app/) from the Sapienza University of Rome.
|
6 |
+
|
7 |
+
The talented research team behind Fauno includes [Andrea Bacciu](https://andreabac3.github.io/), [Dr. Giovanni Trappolini](https://sites.google.com/view/giovannitrappolini), [Andrea Santilli](https://www.santilli.xyz/), and [Professor Fabrizio Silvestri](https://sites.google.com/diag.uniroma1.it/fabriziosilvestri/home).
|
8 |
+
|
9 |
+
Fauno represents a cutting-edge development in open-source Italian Large Language Modeling. It's trained on extensive Italian synthetic datasets, encompassing a wide range of fields such as medical data ๐ฉบ, technical content from Stack Overflow ๐ป, Quora discussions ๐ฌ, and Alpaca data ๐ฆ translated into Italian.
|
10 |
+
|
11 |
+
Hence, our model is able to answer to your questions in Italian ๐, fix your buggy code ๐ and understand a minimum of medical literature ๐.
|
12 |
+
|
13 |
+
## The ๐ฎ๐น open-source version of chatGPT!
|
14 |
+
Discover the capabilities of Fauno and experience the evolution of Italian language models for yourself.
|
15 |
+

|
16 |
+
|
17 |
+
### Why Fauno?
|
18 |
+
We started with a model called Baize, named after a legendary creature from Chinese literature. Continuing along this thematic line, we developed our Italian model based on Baize and named it Fauno, inspired by an iconic figure from Roman mythology. This choice underlines the link between the two models, while maintaining a distinctive identity rooted in Italian culture.
|
19 |
+
|
20 |
+
## ๐ Model's details
|
21 |
+
Fauno is a fine-tuned version of the LoRa weights of [Baize](https://github.com/project-baize/baize-chatbot), that is an improved version of [LLama](https://github.com/facebookresearch/llama).
|
22 |
+
|
23 |
+
We translated and cleaned the data of Baize, and then we fine-tuned the 7b, 13b model using a single RTX A6000 (48GB of VRAM) with 19 hours for one epoch.
|
24 |
+
|
25 |
+
Fauno 13B and 30B are coming soon!
|
26 |
+
|
27 |
## Model initialization
|
28 |
```python
|
29 |
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
|