NVIDIA Triton Inference Server Compatibility?

#8
by RohanAdwankar - opened

When using NVIDIA Triton Inference Server, it says it is not compatible with IR version 10, does anyone have advice from downgrading the version to 9 or other ways to run the model on Triton? Thanks!

| DeepSeek-R1-Distill-Qwen-1.5B | 1 | UNAVAILABLE: Internal: onnx runtime error 1: Load model from /models/DeepSeek-R1-Distill-Qwen-1.5B/1/model.onnx failed:/workspace/onnxruntime/onnxruntime/core/graph/model.cc:149 onnxruntime::Model::Model(onnx::ModelProto&&, const PathString&, const IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment