Update README.md
Browse files
README.md
CHANGED
@@ -43,9 +43,9 @@ The code of Hammer 2.1 models have been in the latest Hugging face transformers
|
|
43 |
## How to Use
|
44 |
Hammer models offer flexibility in deployment and usage, fully supporting both **vLLM** deployment and **Hugging Face Transformers** tool calling. Below are the specifics on how to make use of these features:
|
45 |
|
46 |
-
|
47 |
|
48 |
-
|
49 |
vLLM offers efficient serving with lower latency. To serve the model with vLLM:
|
50 |
```
|
51 |
vllm serve MadeAgents/Hammer2.1-7b --host 0.0.0.0 --port 8000 --tensor-parallel-size 1
|
@@ -96,7 +96,7 @@ print(response)
|
|
96 |
~~~
|
97 |
|
98 |
|
99 |
-
|
100 |
Hammer2.1 supports vllm’s built-in tool calling. This functionality requires vllm>=0.6. If you want to enable this functionality, please start vllm’s OpenAI-compatible service with:
|
101 |
~~~
|
102 |
vllm serve MadeAgents/Hammer2.1-7b --enable-auto-tool-choice --tool-call-parser hermes
|
@@ -180,7 +180,7 @@ print(chat_response.choices[0].message.content)
|
|
180 |
~~~
|
181 |
|
182 |
|
183 |
-
|
184 |
Hammer2.1’s chat template also includes a tool calling template, meaning that you can use Hugging Face transformers’ tool calling support. This is a simple example of how to use our model using Transformers.
|
185 |
~~~
|
186 |
import torch
|
|
|
43 |
## How to Use
|
44 |
Hammer models offer flexibility in deployment and usage, fully supporting both **vLLM** deployment and **Hugging Face Transformers** tool calling. Below are the specifics on how to make use of these features:
|
45 |
|
46 |
+
### Using vLLM
|
47 |
|
48 |
+
#### Option 1: Using Hammer client
|
49 |
vLLM offers efficient serving with lower latency. To serve the model with vLLM:
|
50 |
```
|
51 |
vllm serve MadeAgents/Hammer2.1-7b --host 0.0.0.0 --port 8000 --tensor-parallel-size 1
|
|
|
96 |
~~~
|
97 |
|
98 |
|
99 |
+
#### Option 2: Using vLLM’s built-in tool calling
|
100 |
Hammer2.1 supports vllm’s built-in tool calling. This functionality requires vllm>=0.6. If you want to enable this functionality, please start vllm’s OpenAI-compatible service with:
|
101 |
~~~
|
102 |
vllm serve MadeAgents/Hammer2.1-7b --enable-auto-tool-choice --tool-call-parser hermes
|
|
|
180 |
~~~
|
181 |
|
182 |
|
183 |
+
### Using Hugging Face Transformers
|
184 |
Hammer2.1’s chat template also includes a tool calling template, meaning that you can use Hugging Face transformers’ tool calling support. This is a simple example of how to use our model using Transformers.
|
185 |
~~~
|
186 |
import torch
|