# DeBERTa V2 ONNX with pipeline does not work **URL:** \ **Category:** Beginners\ **Created:** [April 6, 2023, 9:42am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748 "2023-04-06T09:42:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![rcshubhadeep](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@rcshubhadeep](https://discuss.huggingface.co/u/rcshubhadeep)\ **Post date:** [April 6, 2023, 9:42am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/1 "2023-04-06T09:42:14Z")
I have noticed that [this PR](https://github.com/huggingface/transformers/pull/17617) already adds the support for DeBERTa kind of a model to be exported as ONNX. I read through the PR and checked everything possible. However I can’t make the following code work. ```auto from transformers import AutoTokenizer, AutoConfig, DebertaTokenizerFast, pipeline, DebertaV2Tokenizer, __version__ from optimum.onnxruntime import ORTModelForTokenClassification, ORTModelForQuestionAnswering tokenizer = AutoTokenizer.from_pretrained("{custom_fine_tuned_NER_DeBERTaV2}") model = ORTModelForTokenClassification.from_pretrained("{custom_fine_tuned_NER_DeBERTaV2}", export=True, use_auth_token=True) pipe = pipeline("ner", model=model, tokenizer=tokenizer) pipe("MY TEXT GOES HERE") ``` It fails with the following stack trace - ```auto --------------------------------------------------------------------------- InvalidArgument Traceback (most recent call last) Cell In[25], line 1 ----> 1 pipe("I am a skilled engineer. I have worked in JS, CPP, Java, J2ME, and Python. I know Oracle and MySQL") File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/token_classification.py:214, in TokenClassificationPipeline. __call__ (self, inputs, **kwargs) 211 if offset_mapping: 212 kwargs["offset_mapping"] = offset_mapping --> 214 return super(). __call__ (inputs, **kwargs) File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1109, in Pipeline. __call__ (self, inputs, num_workers, batch_size, *args, **kwargs) 1101 return next( 1102 iter( 1103 self.get_iterator( (...) 1106 ) 1107 ) 1108 else: -> 1109 return self.run_single(inputs, preprocess_params, forward_params, postprocess_params) File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1116, in Pipeline.run_single(self, inputs, preprocess_params, forward_params, postprocess_params) 1114 def run_single(self, inputs, preprocess_params, forward_params, postprocess_params): 1115 model_inputs = self.preprocess(inputs, **preprocess_params) -> 1116 model_outputs = self.forward(model_inputs, **forward_params) 1117 outputs = self.postprocess(model_outputs, **postprocess_params) 1118 return outputs File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1015, in Pipeline.forward(self, model_inputs, **forward_params) 1013 with inference_context(): 1014 model_inputs = self._ensure_tensor_on_device(model_inputs, device=self.device) -> 1015 model_outputs = self._forward(model_inputs, **forward_params) 1016 model_outputs = self._ensure_tensor_on_device(model_outputs, device=torch.device("cpu")) 1017 else: File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/token_classification.py:240, in TokenClassificationPipeline._forward(self, model_inputs) 238 logits = self.model(model_inputs.data)[0] 239 else: --> 240 output = self.model(**model_inputs) 241 logits = output["logits"] if isinstance(output, dict) else output[0] 243 return { 244 "logits": logits, 245 "special_tokens_mask": special_tokens_mask, (...) 248 **model_inputs, 249 } File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/optimum/modeling_base.py:85, in OptimizedModel. __call__ (self, *args, **kwargs) 84 def __call__ (self, *args, **kwargs): ---> 85 return self.forward(*args, **kwargs) File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/optimum/onnxruntime/modeling_ort.py:1363, in ORTModelForTokenClassification.forward(self, input_ids, attention_mask, token_type_ids, **kwargs) 1360 onnx_inputs["token_type_ids"] = token_type_ids 1362 # run inference -> 1363 outputs = self.model.run(None, onnx_inputs) 1364 logits = outputs[self.output_names["logits"]] 1366 if use_torch: File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:200, in Session.run(self, output_names, input_feed, run_options) 198 output_names = [output.name for output in self._outputs_meta] 199 try: --> 200 return self._sess.run(output_names, input_feed, run_options) 201 except C.EPFail as err: 202 if self._enable_fallback: InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids ``` I am not sure what I am doing wrong? The same code works for other models (such as another model which BERT based and then fine tuned by me) I am using tokenzier version 0.13.3 transformers version 4.27.4 and optimum version 1.7.3 I am on a AMD based machine in EC2 (AWS) Please help as this is blocking me from optimizing the model. I can’t find any docs on it. ---
**Author:** ![Gopalkrishna](https://avatars.discourse-cdn.com/v4/letter/g/898d66/32.png) [@Gopalkrishna](https://discuss.huggingface.co/u/Gopalkrishna)\ **Post date:** [February 27, 2024, 6:56am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/2 "2024-02-27T06:56:16Z")
@rcshubhadeep please let me know if u found a fix for this. i am facing this same issue right now. ---
**Author:** ![rcshubhadeep](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@rcshubhadeep](https://discuss.huggingface.co/u/rcshubhadeep)\ **Post date:** [February 27, 2024, 7:18am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/3 "2024-02-27T07:18:10Z")
Please [see my question and the accepted answer](https://stackoverflow.com/questions/75948679/deberta-onnx-export-does-not-work-for-token-type-ids) in Stackoverflow. I had implemented a custom pipeline initially closely mimicking the one from the original code but the accepted answer shows a simpler (and perhaps more idiomatic way) ---
**Author:** ![Gopalkrishna](https://avatars.discourse-cdn.com/v4/letter/g/898d66/32.png) [@Gopalkrishna](https://discuss.huggingface.co/u/Gopalkrishna)\ **Post date:** [February 27, 2024, 7:48am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/4 "2024-02-27T07:48:05Z")
@rcshubhadeep I am unable to locate in the code where the tokenizer.model\_inputs has been initialized. Can u help in any way? ---
**Author:** ![rcshubhadeep](https://avatars.discourse-cdn.com/v4/letter/r/c57346/32.png) [@rcshubhadeep](https://discuss.huggingface.co/u/rcshubhadeep)\ **Post date:** [February 27, 2024, 8:11am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/5 "2024-02-27T08:11:49Z")
You must have created a tokenizer object. It is a property of that object. ---
**Author:** ![Gopalkrishna](https://avatars.discourse-cdn.com/v4/letter/g/898d66/32.png) [@Gopalkrishna](https://discuss.huggingface.co/u/Gopalkrishna)\ **Post date:** [February 27, 2024, 8:59am UTC](https://discuss.huggingface.co/t/deberta-v2-onnx-with-pipeline-does-not-work/35748/6 "2024-02-27T08:59:51Z")
got it thank you.