# Run\_backward: expected dtype Float but got dtype Long **URL:** \ **Category:** Intermediate\ **Created:** [December 30, 2023, 6:57pm UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497 "2023-12-30T18:57:56Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![hkamineni](https://avatars.discourse-cdn.com/v4/letter/h/35a633/32.png) [@hkamineni](https://discuss.huggingface.co/u/hkamineni)\ **Post date:** [December 30, 2023, 6:57pm UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497/1 "2023-12-30T18:57:56Z")
Hi , I am getting Expected float error on one of my models using transformers. This is coming at the line 199 Variable.\_execution\_engine.run\_backward( in torch/autograd package. Below is the complete stack trace of the error. File /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2745, in Trainer.training\_step(self, model, inputs) 2743 else: 2744 logger.info(f"`loss.dtype={loss.dtype} , loss={loss}`.") → 2745 self.accelerator.backward(loss) 2747 return loss.detach() / self.args.gradient\_accumulation\_steps File /opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py:1910, in Accelerator.backward(self, loss, \*\*kwargs) 1908 print(f"`acc: loss.dtype={loss.dtype} , loss={loss}`.“) 1909 logger.info(f"`acc: loss.dtype={loss.dtype} , loss={loss}`.”) → 1910 loss.backward(\*\*kwargs) File /opt/conda/lib/python3.10/site-packages/torch/\_tensor.py:489, in Tensor.backward(self, gradient, retain\_graph, create\_graph, inputs) 479 return handle\_torch\_function( 480 Tensor.backward, 481 (self,), (…) 486 inputs=inputs, 487 ) 488 print(f"`dtype={self.dtype} self={self} gradient={gradient}` .") → 489 torch.autograd.backward( 490 self, gradient, retain\_graph, create\_graph, inputs=inputs 491 ) File /opt/conda/lib/python3.10/site-packages/torch/autograd/ **init**.py:199, in backward(tensors, grad\_tensors, retain\_graph, create\_graph, grad\_variables, inputs) 197 print(f"`tensors={tensors} , 0_dtype={tensors[0].dtype}` .“) 198 print(f"`grad_tensors_={grad_tensors_} , 0_dtype={grad_tensors_[0].dtype}` .”) → 199 Variable._execution\_engine.run\_backward( # Calls into the C++ engine to run the backward pass 200 tensors, grad\_tensors_, retain\_graph, create\_graph, inputs, 201 allow\_unreachable=True, accumulate\_grad=True) I did some debugging and logged the values of tensor at each of the stack trace call. loss.dtype=torch.float32 , loss=0.006293008103966713`. ( in trainer.py ) `acc: loss.dtype=torch.float32 , loss=0.006293008103966713`. (in accelerator.py) `dtype=torch.float32 self=0.006293008103966713 gradient=None` . ( in torch/\_tensor.py) The 2 lines below at last stack trace in torch/autograd/ **init**.py `tensors=(tensor(0.0063, device='cuda:0', grad_fn=),) , 0_dtype=torch.float32` . `grad_tensors_=(tensor(1., device='cuda:0'),) , 0_dtype=torch.float32` . The line numbers above may be slightly off as I added print or logging statements before the function call. As you can see the tensor value is always float but I am getting type error. I am puzzled by the error. What variable is it expecting to be float that is long? Thanks for the help in advance. ---
**Author:** ![suhaaspk](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@suhaaspk](https://discuss.huggingface.co/u/suhaaspk)\ **Post date:** [February 4, 2024, 7:28pm UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497/2 "2024-02-04T19:28:13Z")
Hi, I am getting the same error. Did you find out what the issue is? ---
**Author:** ![hkamineni](https://avatars.discourse-cdn.com/v4/letter/h/35a633/32.png) [@hkamineni](https://discuss.huggingface.co/u/hkamineni)\ **Post date:** [February 5, 2024, 4:17am UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497/3 "2024-02-05T04:17:57Z")
Yes, the error was that we only had one class in the labels, and the model could not be trained to distinguish between multiple classes when we only inputted one class in the data. Thus, it instead threw this error. ---
**Author:** ![InderV94](https://avatars.discourse-cdn.com/v4/letter/i/9de053/32.png) [@InderV94](https://discuss.huggingface.co/u/InderV94)\ **Post date:** [July 2, 2024, 6:43am UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497/4 "2024-07-02T06:43:07Z")
I am kind of ending up with same error , can you help me to understand what exactly caused your error? ---
**Author:** ![hkamineni](https://avatars.discourse-cdn.com/v4/letter/h/35a633/32.png) [@hkamineni](https://discuss.huggingface.co/u/hkamineni)\ **Post date:** [July 3, 2024, 12:36am UTC](https://discuss.huggingface.co/t/run-backward-expected-dtype-float-but-got-dtype-long/67497/5 "2024-07-03T00:36:13Z")
My model was a classifier model, but the data had only one label so the model could not be trained to distinguish between multiple categories as there was only one label.