Safetensors
qwen2
linqq9 commited on
Commit
30720f1
·
verified ·
1 Parent(s): 4875e47

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -43,9 +43,9 @@ The code of Hammer 2.1 models have been in the latest Hugging face transformers
43
  ## How to Use
44
  Hammer models offer flexibility in deployment and usage, fully supporting both **vLLM** deployment and **Hugging Face Transformers** tool calling. Below are the specifics on how to make use of these features:
45
 
46
- #### Using vLLM
47
 
48
- ##### Option 1: Using Hammer client
49
  vLLM offers efficient serving with lower latency. To serve the model with vLLM:
50
  ```
51
  vllm serve MadeAgents/Hammer2.1-7b --host 0.0.0.0 --port 8000 --tensor-parallel-size 1
@@ -96,7 +96,7 @@ print(response)
96
  ~~~
97
 
98
 
99
- ##### Option 2: Using vLLM’s built-in tool calling
100
  Hammer2.1 supports vllm’s built-in tool calling. This functionality requires vllm>=0.6. If you want to enable this functionality, please start vllm’s OpenAI-compatible service with:
101
  ~~~
102
  vllm serve MadeAgents/Hammer2.1-7b --enable-auto-tool-choice --tool-call-parser hermes
@@ -180,7 +180,7 @@ print(chat_response.choices[0].message.content)
180
  ~~~
181
 
182
 
183
- #### Using Hugging Face Transformers
184
  Hammer2.1’s chat template also includes a tool calling template, meaning that you can use Hugging Face transformers’ tool calling support. This is a simple example of how to use our model using Transformers.
185
  ~~~
186
  import torch
 
43
  ## How to Use
44
  Hammer models offer flexibility in deployment and usage, fully supporting both **vLLM** deployment and **Hugging Face Transformers** tool calling. Below are the specifics on how to make use of these features:
45
 
46
+ ### Using vLLM
47
 
48
+ #### Option 1: Using Hammer client
49
  vLLM offers efficient serving with lower latency. To serve the model with vLLM:
50
  ```
51
  vllm serve MadeAgents/Hammer2.1-7b --host 0.0.0.0 --port 8000 --tensor-parallel-size 1
 
96
  ~~~
97
 
98
 
99
+ #### Option 2: Using vLLM’s built-in tool calling
100
  Hammer2.1 supports vllm’s built-in tool calling. This functionality requires vllm>=0.6. If you want to enable this functionality, please start vllm’s OpenAI-compatible service with:
101
  ~~~
102
  vllm serve MadeAgents/Hammer2.1-7b --enable-auto-tool-choice --tool-call-parser hermes
 
180
  ~~~
181
 
182
 
183
+ ### Using Hugging Face Transformers
184
  Hammer2.1’s chat template also includes a tool calling template, meaning that you can use Hugging Face transformers’ tool calling support. This is a simple example of how to use our model using Transformers.
185
  ~~~
186
  import torch