takyan commited on
Commit
7f139d5
·
verified ·
1 Parent(s): da71d75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +63 -0
README.md CHANGED
@@ -20,3 +20,66 @@ language:
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
23
+
24
+
25
+ # How to Use
26
+ 以下はelyza-tasks-100-TVに対する回答出力用のコードです。
27
+ ```
28
+ from unsloth import FastLanguageModel
29
+ import torch
30
+ import json
31
+
32
+ model_name = "takyan/llm-jp-3-13b-finetune-2"
33
+
34
+ max_seq_length = 2048
35
+ dtype = None
36
+ load_in_4bit = True
37
+
38
+ model, tokenizer = FastLanguageModel.from_pretrained(
39
+ model_name = model_name,
40
+ max_seq_length = max_seq_length,
41
+ dtype = dtype,
42
+ load_in_4bit = load_in_4bit,
43
+ token = "your hf token",
44
+ )
45
+ FastLanguageModel.for_inference(model)
46
+
47
+ datasets = []
48
+ with open("./elyza-tasks-100-TV_0.jsonl", "r") as f:
49
+ item = ""
50
+ for line in f:
51
+ line = line.strip()
52
+ item += line
53
+ if item.endswith("}"):
54
+ datasets.append(json.loads(item))
55
+ item = ""
56
+
57
+ from tqdm import tqdm
58
+
59
+ results = []
60
+ for data in tqdm(datasets):
61
+
62
+ input = data["input"]
63
+
64
+ prompt = f"""### 指示
65
+ {input}
66
+ ### 回答:
67
+ """
68
+
69
+ tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
70
+ with torch.no_grad():
71
+ outputs = model.generate(
72
+ tokenized_input,
73
+ max_new_tokens=100,
74
+ do_sample=False,
75
+ repetition_penalty=1.2
76
+ )[0]
77
+ output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)
78
+
79
+ results.append({"task_id": data["task_id"], "input": input, "output": output})
80
+
81
+ with open(f"./{model_name}_output.jsonl", 'w', encoding='utf-8') as f:
82
+ for result in results:
83
+ json.dump(result, f, ensure_ascii=False)
84
+ f.write('\n')
85
+ ```