Update README.md

README.md

@@ -274,10 +274,10 @@ This model is a fine-tuned version of [google/flan-t5-large](https://huggingface
 
 - **NLP Paper's Abstract + Introduction --> {Question} [SEP] {Answer}**
 
-## How to Use ( Code Snippets )
+## (1) How to use: Inference on CPU ( Code Snippets )
 
 
-### # Load model directly 
+### Load model directly 
 ```(python)
 from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
 
@@ -305,14 +305,46 @@ summaries = model.generate(input_ids =inputs["input_ids"], max_new_tokens=100, d
 summaries = model.generate(input_ids =inputs["input_ids"], max_new_tokens=100, do_sample = True, top_p = 0.95)
 ```
 
-
-
 ```
 decoded_summaries = [tokenizer.decode(s, skip_special_tokens=False, clean_up_tokenization_spaces=True) for s in summaries]
 decoded_summaries = [d.replace("<n>", " ").replace(tokenizer.pad_token, "").replace(tokenizer.eos_token, "") for d in decoded_summaries]
 
 ```
 
+## (2) Faster Inference on GPU 
+- about 60x faster than (1) [CPU --> COLAB T4 GPU]
+
+### Additional Installation 
+```(python)
+!pip install accelerate -q
+!pip install bitsandbytes -q
+!pip install optimum -q
+```
+
+### Load model directly 
+```(python)
+import torch
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM,BitsAndBytesConfig
+from optimum.bettertransformer import BetterTransformer
+
+# load model in 4-bit
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.bfloat16
+)
+
+tokenizer = AutoTokenizer.from_pretrained("UNIST-Eunchan/FLAN-T5-NLP-Paper-to-Question-Generation")
+model = AutoModelForSeq2SeqLM.from_pretrained("UNIST-Eunchan/FLAN-T5-NLP-Paper-to-Question-Generation", quantization_config=quantization_config)
+model = BetterTransformer.transform(model)
+```
+
+
+### For Multiple Question Generation  (👍)
+```(python)
+# use to(device) 
+summaries = model.generate(input_ids =inputs["input_ids"].to(device), max_new_tokens=100, do_sample = True, top_p = 0.95, num_return_sequences = 4)
+```
+
 
 ### Training results