license: mit language:
- en base_model:
- MatteoKhan/Mistral-LLaMA-Fusion library_name: transformers tags:
- fine-tuned
- cosmetic-domain
- lora
- mistral
- llama
- rtx4060-optimized π CosmeticAdvisor: Expert Model for Beauty & Cosmetic Queries π Overview Mistral-LLaMA-Fusion-Cosmetic is a domain-specialized language model, fine-tuned on a dataset focused on cosmetic-related queries. Built from the powerful Mistral-LLaMA-Fusion, this version benefits from LoRA-based fine-tuning and GPU optimization on a RTX 4060.
π Created by: Matteo Khan π Affiliation: Apprentice at TW3 Partners (Generative AI Research) π License: MIT
π Connect on LinkedIn(https://www.linkedin.com/in/matteo-khan-a10309263/) π Base Model
π§ Model Details Architecture: Mistral + LLaMA fusion
Technique: Fine-tuned with LoRA (Low-Rank Adaptation)
Base Model: MatteoKhan/Mistral-LLaMA-Fusion
Training Dataset: Proprietary dataset (Parquet) of user queries in the cosmetic and beauty domain
Training Hardware: RTX 4060 (8GB VRAM), 3 epochs
π― Intended Use This model is optimized for:
β Responding to beauty & cosmetic product questions
β Assisting in cosmetic product recommendation
β Enhancing chatbots in beauty domains
β Cosmetic-focused creative content generation
π οΈ Technical Details Fine-tuning Method: LoRA (r=8, Ξ±=16, dropout=0.05)
Quantization: 4-bit NF4 via bitsandbytes
Training Strategy: Gradient checkpointing + mixed precision (fp16)
Sequence Length: 256 tokens
Batch Strategy: Batch size 1 + gradient accumulation 16
π§ͺ Training Configuration (LoRA) python Copier Modifier peft_config = LoraConfig( task_type=TaskType.CAUSAL_LM, inference_mode=False, r=8, lora_alpha=16, lora_dropout=0.05, target_modules=["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], bias="none", ) π How to Use python Copier Modifier from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "MatteoKhan/CosmeticAdvisor" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)
prompt = "What skincare products are best for oily skin?" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_length=256) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) β οΈ Limitations May hallucinate or provide incorrect information
Knowledge is limited to cosmetic domain-specific data
Should not replace professional dermatological advice
π§Ύ Citation If you use this model in your research, please cite:
bibtex Copier Modifier @misc{mistralllama2025cosmetic, title={Mistral-LLaMA-Fusion-Cosmetic}, author={Matteo Khan}, year={2025}, note={Fine-tuned for cosmetic domain}, url={https://huggingface.co/MatteoKhan/CosmeticAdvisor} }