Update README.md

README.md

@@ -39,6 +39,28 @@ language:
 - sr
 - sv
 - uk
+datasets:
+- oscar-corpus/colossal-oscar-1.0
+- HuggingFaceFW/fineweb-edu
+- joelniklaus/eurlex_resources
+- joelito/legal-mc4
+- projecte-aina/CATalog
+- UFRGS/brwac
+- community-datasets/hrwac
+- danish-foundation-models/danish-gigaword
+- HiTZ/euscrawl
+- PleIAs/French-PD-Newspapers
+- PleIAs/French-PD-Books
+- AI-team-UoA/greek_legal_code
+- HiTZ/latxa-corpus-v1.1
+- allenai/peS2o
+- pile-of-law/pile-of-law
+- PORTULAN/parlamento-pt
+- hoskinson-center/proof-pile
+- togethercomputer/RedPajama-Data-1T
+- bigcode/starcoderdata
+- bjoernp/tagesschau-2018-2023
+- EleutherAI/the_pile_deduplicated
 ---
 
 ![](./images/salamandra_header.png)
@@ -282,13 +304,13 @@ The pre-training corpus comprises data from 35 European languages and 92 program
 The initial three training epochs used 2.4 trillion tokens, obtained by manually adjusting data proportion to balance the representation 
 and give more importance to Spain’s co-official (Spanish, Catalan, Galician, and Basque). This way, we downsampled code and English data to half, 
 Spanish co-official languages were oversampled by 2x, and the remaining languages were kept in their original proportions.
-Following, we trained two additional epochs during which the Colossal OSCAR dataset was replaced with the FineWebEdu dataset. 
+During the following epochs, the Colossal OSCAR dataset was replaced with the FineWeb-Edu dataset. 
 This adjustment resulted in a total of 2.68 trillion tokens, distributed as outlined below:
 
-![lang distrib](./images/corpus_languages.png)
+![lang distrib](./images/corpus_languages_1.1.png)
 
 The pretraining corpus is predominantly composed of data from Colossal OSCAR, which contributes a significant 53,05% of the total tokens. 
-Following this, Starcoder provides 13,67%, and FineWebEdu (350B tokens subset) adds 10,24%. The next largest sources are HPLT at 4,21% and French-PD at 3,59%. 
+Following this, Starcoder provides 13,67%, and FineWeb-Edu (350BT subset) adds 10,24%. The next largest sources are HPLT at 4,21% and French-PD at 3,59%. 
 Other notable contributions include MaCoCu, Legal-ES, and EurLex, each contributing around 1.72% to 1.41%. 
 These major sources collectively form the bulk of the corpus, ensuring a rich and diverse dataset for training the language model. 
 The remaining 10% comes from smaller sources in various languages.
@@ -430,7 +452,7 @@ To consult the data summary document with the respective licences, please send a
 </details>
 
 The model was trained on 3 pre-training epochs with 2.4T tokens per epoch, 2 additional pre-training epochs in which the English part 
-of the Colossal OSCAR dataset was replaced with FineWebEdu (350T subset), resulting in 2.68T tokens per epoch; 
+of the Colossal OSCAR dataset was replaced with FineWeb-Edu (350BT subset), resulting in 2.68T tokens per epoch; 
 and 1 final epoch of 0.315T higher quality tokens, meaning that the total number of tokens seen during pre-training is approximately 12.875 trillion tokens.
 
 We provide an extense Datasheet section following the best practices defined by [(Gebru et al., 2021)](https://arxiv.org/pdf/1803.09010).
@@ -463,7 +485,7 @@ and public institutions, which can be found in detail in the acknowledgements.
 
 **Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number.**
 
-This work/research has been promoted and financed by the Government of Catalonia through the [Aina project](https://projecteaina.cat/).
+This work has been promoted and financed by the Government of Catalonia through the [Aina Project](https://projecteaina.cat/).
 
 This work is funded by the _Ministerio para la Transformación Digital y de la Función Pública_ - Funded by EU – NextGenerationEU 
 within the framework of [ILENIA Project](https://proyectoilenia.es/) with reference 2022/TL22/00215337.
@@ -1111,4 +1133,4 @@ Technical report coming soon.
 |:---:|:---:|:---:|
 |2B| [Link](https://huggingface.co/BSC-LT/salamandra-2b) | [Link](https://huggingface.co/BSC-LT/salamandra-2b-instruct) |
 |7B| [Link](https://huggingface.co/BSC-LT/salamandra-7b) | [Link](https://huggingface.co/BSC-LT/salamandra-7b-instruct) |
-|40B| [Link](https://huggingface.co/BSC-LT/ALIA-40b) | WiP |
+|40B| [Link](https://huggingface.co/BSC-LT/ALIA-40b) | WiP |