What transformers version can this be deployed with?
I tried deploying this model on AWS SageMaker however it seems like the transformers library doesn't have an update to date version to yet to handle gemma 3.
How can this be deployed?
To use Gemma-3 models, you need the latest development version of the Transformers library (4.50.0.dev0). You can install it directly from the GitHub branch using: pip install git+https://github.com/huggingface/[email protected] as mentioned here.
Ok thanks. The issue that I am having though is that in order to deploy on sagemaker, I have to put which transformers version and there hasn't been a new release with Gemma 3 yet. I also tried extending a Deep learning container by installing the (4.50.0.dev0) but ran into some compatibility issues.
What would be the easiest way for me to deploy this on sagemaker?
A new stable version of Transformers is now available which is compatible to Gemma3. Please update it using pip install -U transformers and try again. Let us know if this helps! Thank you
I had similar issue and even the newer version didnt help. It gave me the below error.
Traceback (most recent call last): File "/usr/local/bin/dockerd-entrypoint.py", line 21, in from sagemaker_huggingface_inference_toolkit import serving File "/opt/conda/lib/python3.10/site-packages/sagemaker_huggingface_inference_toolkit/serving.py", line 18, in from sagemaker_huggingface_inference_toolkit import handler_service, mms_model_server File "/opt/conda/lib/python3.10/site-packages/sagemaker_huggingface_inference_toolkit/handler_service.py", line 28, in from sagemaker_huggingface_inference_toolkit.transformers_utils import ( File "/opt/conda/lib/python3.10/site-packages/sagemaker_huggingface_inference_toolkit/transformers_utils.py", line 24, in from transformers.pipelines import Conversation, Pipeline
2025-04-04T19:40:20.455Z
ImportError: cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.10/site-packages/transformers/pipelines/init.py)