mervinpraison commited on
Commit
2c67432
·
verified ·
1 Parent(s): 37465a5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -4
README.md CHANGED
@@ -5,7 +5,6 @@ tags:
5
  - transformers
6
  - unsloth
7
  - mllama
8
- license: apache-2.0
9
  language:
10
  - en
11
  ---
@@ -13,9 +12,58 @@ language:
13
  # Uploaded finetuned model
14
 
15
  - **Developed by:** mervinpraison
16
- - **License:** apache-2.0
17
  - **Finetuned from model :** unsloth/llama-3.2-11b-vision-instruct-unsloth-bnb-4bit
18
 
19
- This mllama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
5
  - transformers
6
  - unsloth
7
  - mllama
 
8
  language:
9
  - en
10
  ---
 
12
  # Uploaded finetuned model
13
 
14
  - **Developed by:** mervinpraison
 
15
  - **Finetuned from model :** unsloth/llama-3.2-11b-vision-instruct-unsloth-bnb-4bit
16
 
17
+ ```yaml
18
+ dataset:
19
+ - name: mervinpraison/accelerometer-fall-axis-plots-alpaca
20
+ dataset_num_proc: 2
21
+ dataset_text_field: text
22
+ gradient_accumulation_steps: 4
23
+ hf_model_name: mervinpraison/llama-3.2-11B-vision-harupfall-axis
24
+ huggingface_save: 'true'
25
+ learning_rate: 0.0002
26
+ load_in_4bit: true
27
+ loftq_config: null
28
+ logging_steps: 1
29
+ lora_alpha: 16
30
+ lora_bias: none
31
+ lora_dropout: 0
32
+ lora_r: 16
33
+ lora_target_modules:
34
+ - q_proj
35
+ - k_proj
36
+ - v_proj
37
+ - o_proj
38
+ - gate_proj
39
+ - up_proj
40
+ - down_proj
41
+ lr_scheduler_type: linear
42
+ max_seq_length: 2048
43
+ max_steps: 4194
44
+ model_name: unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit
45
+ model_parameters: 11b
46
+ num_train_epochs: 2
47
+ ollama_model: mervinpraison/llama-3.2-11b-vision-harupfall-axis
48
+ ollama_save: 'true'
49
+ optim: adamw_8bit
50
+ output_dir: outputs
51
+ packing: false
52
+ per_device_train_batch_size: 1
53
+ quantization_method:
54
+ - q4_k_m
55
+ random_state: 3407
56
+ seed: 3407
57
+ train: 'true'
58
+ use_gradient_checkpointing: unsloth
59
+ use_rslora: false
60
+ warmup_steps: 5
61
+ weight_decay: 0.01
62
 
63
+ # Vision-specific parameters
64
+ finetune_vision_layers: false
65
+ finetune_language_layers: true
66
+ finetune_attention_modules: true
67
+ finetune_mlp_modules: true
68
+ vision_instruction: "Determine whether the following accelerometer data indicates a fall or not."
69
+ ```