NVIDIA Triton Inference Server Compatibility?
#8
by
RohanAdwankar
- opened
When using NVIDIA Triton Inference Server, it says it is not compatible with IR version 10, does anyone have advice from downgrading the version to 9 or other ways to run the model on Triton? Thanks!
| DeepSeek-R1-Distill-Qwen-1.5B | 1 | UNAVAILABLE: Internal: onnx runtime error 1: Load model from /models/DeepSeek-R1-Distill-Qwen-1.5B/1/model.onnx failed:/workspace/onnxruntime/onnxruntime/core/graph/model.cc:149 onnxruntime::Model::Model(onnx::ModelProto&&, const PathString&, const IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9