Update README.md

Add more information on post-training

README.md

@@ -18,7 +18,7 @@ base_model:
 # Llama-Krikri-8B-Instruct: An Instruction-tuned Large Language Model for the Greek language
 
 Following the release of [Meltemi-7B](https://huggingface.co/ilsp/Meltemi-7B-v1) on the 26th March 2024, we are happy to welcome Krikri to the family of ILSP open Greek LLMs.
-Krikri is built on top of [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B), extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present **Llama-Krikri-8B-Instruct**, along with the base model, [Llama-Krikri-8B-Base](https://huggingface.co/ilsp/Llama-Krikri-8B-Base).
+Krikri is built on top of [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B), extending its capabilities for Greek through continual pretraining on a large corpus of high-quality and locally relevant Greek texts. We present **Llama-Krikri-8B-Instruct**, along with the base model, [Llama-Krikri-8B-Base](https://huggingface.co/ilsp/Llama-Krikri-8B-Base)
 
 ![image/png](llama-krikri-image.jpg)
 
@@ -59,7 +59,25 @@ Llama-Krikri-8B-Instruct is the result of post-training Llama-Kriki-8B-Base and
 - Conversion or structured extraction (e.g., XML, JSON) in data-to-text & text-to-data settings.
 - Analytical thinking and Chain-of-Thought (CoT) reasoning for problem-solving.
 
-🚨 **More information on the post-training corpus and methdology coming soon.** 🚨
+We used a multi-stage process in order to build Llama-Krikri-8B-Instruct which includes:
+- 2-stage Supervised Fine-Tuning with a combination of Greek & English instruction-response pairs
+  - **Stage 1**: **856,946** instruction-response pairs (371,379 Greek + 485,567 English)
+  - **Stage 2**: **638,408** instruction-response pairs (279,948 Greek + 358,460 English)
+- Alignment with a combination of Greek & English preference triplets
+  - **Length Normalized DPO**: **92,394** preference triplets (47,132 Greek + 45,262 English)
+ 
+To build the SFT & DPO data, we utilized various methodologies including:
+- Collecting existing high-quality datasets such as [Tulu 3](https://huggingface.co/datasets/allenai/tulu-3-sft-mixture), [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk), [MAGPIE Ultra](https://huggingface.co/datasets/argilla/magpie-ultra-v1.0), [Orca Agent Instruct](https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1), [IFEval Like Data](https://huggingface.co/datasets/argilla/ifeval-like-data), [UltraFeedback](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized), [NVIDIA HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2), [Intel Orca](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs), [UltraMedical](https://huggingface.co/datasets/TsinghuaC3I/UltraMedical-Preference), and other datasets focused on safety, truthfulness, and instruction-following.
+- Translating various data into Greek using an in-house translation tool.
+- Distilling (with the MAGPIE methodology) models which exhibit strong performance in Greek, such as [Gemma 2 27B IT](https://huggingface.co/google/gemma-2-27b-it).
+- Scoring data with the [Skywork Reward Gemma 2 27B v0.2](https://huggingface.co/Skywork/Skywork-Reward-Gemma-2-27B-v0.2) Reward Model and filtering using rule-based filters.
+- Creating data for sentence and document translation using high-quality parallel corpora mainly from [ELRC-SHARE](https://elrc-share.eu/).
+- Synthetically extracting question-answer pairs (RAG) and multi-turn dialogues from diverse sources such as Wikipedia, EUR-LEX, Greek School Books, and Kallipos.
+
+# Evaluation
+
+
+🚨 **More information on post-training, methdology, and evaluation coming soon.** 🚨
 
 
 # How to use