I’m new to HuggingFace/Optimum, in fact I started working on AI/ML stuff recently. I tried to optimum models with DML EP (on my windows PC), for example take optimum/vit-base-patch16-224 · Hugging Face model = ORTModelForImageClassification.from_pretrained(model_name, provider=“DmlExecutionProvid…

This is automatic behavior in onnxruntime: Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf. There was an issue in onnxruntime repo that was closed w…

No, it is different. As I mentioned in the other post, pipeline is forcing everything to CPU EP and I don’t see the issue with the latest packages

Thanks @IlyasMoutawwakil for your response. Not just with shape related operators. simple ops like add, sub, equal are also getting placed on CPU EP with provider=“DmlExecutionProvider” option with optimum/gpt2 · Hugging Face 2024-06-06 08:58:16.7010989 [V:onnxruntime:, session_state.cc:1152 onn…

Not able run all nodes on DML with optimum

🤗Optimum

regisss June 6, 2024, 8:41am 2

Looks like a duplicate of Not able to run on DML with pipeline

Topic		Replies	Views
Not able to run on DML with pipeline 🤗Optimum	2	460	June 6, 2024
Some nodes were not assigned to the preferred execution providers 🤗Optimum	1	3676	January 10, 2024
Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf Models	0	1059	November 10, 2023
What is the execution provider of optimum[onnxruntime]? 🤗Optimum	0	311	March 11, 2024
ORT CLI vs. Programmatic 🤗Optimum	12	1416	August 17, 2023

Not able run all nodes on DML with optimum

Related topics