llm-course-hw3-lora / README.md
mcnckc's picture
Update README.md
3ae0f9c verified
---
library_name: transformers
datasets:
- cardiffnlp/tweet_eval
language:
- en
metrics:
- f1
base_model:
- OuteAI/Lite-Oute-1-300M-Instruct
pipeline_tag: text-classification
---
Модель `OuteAI/Lite-Oute-1-300M-Instruct` дообученная на датасете `cardiffnlp/tweet_eval`, задача классификации сентимента твита, вывести одно из трех слов -
`negative`, `neutral`, `positive`.
## Дообучение
Модель дообучалась при помощи LoRA.
- Ранг LoRA = `8`
- `alpha=16`
- LoRA применялась только к весам Key, Value в attention
- `BATCH_SIZE = 16`
- `LEARNING_RATE = 2e-4`
- `NUM_EPOCHS = 2`
- `AdamW`
## Метрика на валидации
F1=0.53
![image/png](https://cdn-uploads.huggingface.co/production/uploads/67b331dfe2883deef7c92e6f/tRdw-OAVMZfZg-mHywylC.png)
## Примеры генерации
**Tweet:** "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin" \
**Label:** positive \
**Output:** \
positive \
positive \
positive
**Tweet:** "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ" \
**Label:** neutral \
**Output:** \
neutral \
neutral \
neutral \
neut
**Tweet:** Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night. \
**Label:** neutral \
**Output:** \
neutral \
positive \
positive \
pos
**Tweet:** Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays \
**Label:** neutral \
**Output:** \
neutral \
neutral \
neutral \
neut
**Tweet:** @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017" \
**Label:** positive \
**Output:** \
neutral \
neutral \
neutral \
neut