Update README.md
Browse files
README.md
CHANGED
|
@@ -7,6 +7,7 @@ inference: false
|
|
| 7 |
|
| 8 |
`MegaBeam-Mistral-7B-512k` is a Large-Context LLM that supports 524,288 tokens in its context. `MegaBeam-Mistral-7B-512k` was trained on [Mistral-7B Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), and can be deployed using various serving frameworks like [vLLM](https://github.com/vllm-project/vllm) and Amazon SageMaker's [DJL](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-models-frameworks-djl-serving.html) endpoint. Please refer to our [GitRepo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/tree/main/megabeam-mistral-7b) for deployment and inference examples.
|
| 9 |
|
|
|
|
| 10 |
|
| 11 |
## Evaluations
|
| 12 |
We evaluated `MegaBeam-Mistral-7B-512k` on three long-context benchmarks. For each benchmark, we deployed the `MegaBeam-Mistral-7B-512k` model with [vLLM (v0.5.1)](https://github.com/vllm-project/vllm/releases/tag/v0.5.1) on an EC2 instance and obtained LLM responses through the OpenAI API provided by vLLM.
|
|
|
|
| 7 |
|
| 8 |
`MegaBeam-Mistral-7B-512k` is a Large-Context LLM that supports 524,288 tokens in its context. `MegaBeam-Mistral-7B-512k` was trained on [Mistral-7B Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), and can be deployed using various serving frameworks like [vLLM](https://github.com/vllm-project/vllm) and Amazon SageMaker's [DJL](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-models-frameworks-djl-serving.html) endpoint. Please refer to our [GitRepo](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/tree/main/megabeam-mistral-7b) for deployment and inference examples.
|
| 9 |
|
| 10 |
+
**New update!** - Watch our [talk on MegaBeam](https://neurips.cc/Expo/Conferences/2024/talk%20panel/100673) at NeurIPS 2024
|
| 11 |
|
| 12 |
## Evaluations
|
| 13 |
We evaluated `MegaBeam-Mistral-7B-512k` on three long-context benchmarks. For each benchmark, we deployed the `MegaBeam-Mistral-7B-512k` model with [vLLM (v0.5.1)](https://github.com/vllm-project/vllm/releases/tag/v0.5.1) on an EC2 instance and obtained LLM responses through the OpenAI API provided by vLLM.
|