Hugging Face's logo

mcnckc
/
llm-course-hw3-lora

Text Classification
Transformers
Safetensors
English
mistral
text-generation
text-generation-inference
Model card Files Community
llm-course-hw3-lora / README.md
mcnckc's picture
mcnckc
Update README.md
3ae0f9c verified
preview code
|
raw
history blame contribute delete
1.95 kB
metadata
library_name: transformers
datasets:
  - cardiffnlp/tweet_eval
language:
  - en
metrics:
  - f1
base_model:
  - OuteAI/Lite-Oute-1-300M-Instruct
pipeline_tag: text-classification

Модель OuteAI/Lite-Oute-1-300M-Instruct дообученная на датасете cardiffnlp/tweet_eval, задача классификации сентимента твита, вывести одно из трех слов - negative, neutral, positive.

Дообучение

Модель дообучалась при помощи LoRA.

  • Ранг LoRA = 8
  • alpha=16
  • LoRA применялась только к весам Key, Value в attention
  • BATCH_SIZE = 16
  • LEARNING_RATE = 2e-4
  • NUM_EPOCHS = 2
  • AdamW

Метрика на валидации

F1=0.53

image/png

Примеры генерации

Tweet: "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin"
Label: positive
Output:
positive
positive
positive

Tweet: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"
Label: neutral
Output:
neutral
neutral
neutral
neut

Tweet: Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.
Label: neutral
Output:
neutral
positive
positive
pos

Tweet: Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays
Label: neutral
Output:
neutral
neutral
neutral
neut

Tweet: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"
Label: positive
Output:
neutral
neutral
neutral
neut