# When i use 'transformers.TrainingArguments', i got loss errors

**URL:** <https://discuss.huggingface.co/t/when-i-use-transformers-trainingarguments-i-got-loss-errors/37765>\
**Category:** 🤗AutoTrain\
**Created:** [April 26, 2023, 8:55am UTC](https://discuss.huggingface.co/t/when-i-use-transformers-trainingarguments-i-got-loss-errors/37765 "2023-04-26T08:55:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hexixi](https://avatars.discourse-cdn.com/v4/letter/h/47e85d/32.png) [@Hexixi](https://discuss.huggingface.co/u/Hexixi)\
**Post date:** [April 26, 2023, 8:55am UTC](https://discuss.huggingface.co/t/when-i-use-transformers-trainingarguments-i-got-loss-errors/37765/1 "2023-04-26T08:55:27Z")

</div>

when i use ‘transformers.TrainingArguments’ and set (evaluation\_strategy=“steps”,save\_strategy=“steps”, eval\_steps=200,) , i got loss errors.  
my code:

```
    trainer = ModifiedTrainer(
    model=model,
    train_dataset=train_data,
    eval_dataset =val_data, 

    args=transformers.TrainingArguments(
        per_device_train_batch_size=4,
        gradient_accumulation_steps=1,
        # warmup_steps=100,
        num_train_epochs=4,
        learning_rate=2e-6,
        fp16=True,
        logging_steps=10,
        optim="adamw_torch",
        # evaluation_strategy="steps",
        save_strategy="steps",
        # eval_steps=200,
        save_steps=200,
        # load_best_model_at_end=True,
        output_dir='./fenlei',
        save_total_limit=2,
        # load_best_model_at_end=True,
        # remove_unused_columns=True,
    ),
    callbacks=[TensorBoardCallback(writer)],
    data_collator=data_collator,)

```

it could be:  
│ ❱ 230 │ trainer.train() │  
│ 231 │ │  
│ 232 │ writer.close() │  
│ 233 │ # save model │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:1662 in │  
│ train │  
│ │  
│ 1659 │ │ inner\_training\_loop = find\_executable\_batch\_size( │  
│ 1660 │ │ │ self.\_inner\_training\_loop, self.\_train\_batch\_size, args.auto\_find\_batch\_size │  
│ 1661 │ │ ) │  
│ ❱ 1662 │ │ return inner\_training\_loop( │  
│ 1663 │ │ │ args=args, │  
│ 1664 │ │ │ resume\_from\_checkpoint=resume\_from\_checkpoint, │  
│ 1665 │ │ │ trial=trial, │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:2006 in │  
│ \_inner\_training\_loop │  
│ │  
│ 2003 │ │ │ │ │ self.state.epoch = epoch + (step + 1 + steps\_skipped) / steps\_in\_epo │  
│ 2004 │ │ │ │ │ self.control = self.callback\_handler.on\_step\_end(args, self.state, s │  
│ 2005 │ │ │ │ │ │  
│ ❱ 2006 │ │ │ │ │ self.\_maybe\_log\_save\_evaluate(tr\_loss, model, trial, epoch, ignore\_k │  
│ 2007 │ │ │ │ else: │  
│ 2008 │ │ │ │ │ self.control = self.callback\_handler.on\_substep\_end(args, self.state │  
│ 2009 │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:2287 in │  
│ \_maybe\_log\_save\_evaluate │  
│ │  
│ 2284 │ │ │ │ │ ) │  
│ 2285 │ │ │ │ │ metrics.update(dataset\_metrics) │  
│ 2286 │ │ │ else: │  
│ ❱ 2287 │ │ │ │ metrics = self.evaluate(ignore\_keys=ignore\_keys\_for\_eval) │  
│ 2288 │ │ │ self.\_report\_to\_hp\_search(trial, self.state.global\_step, metrics) │  
│ 2289 │ │ │  
│ 2290 │ │ if self.control.should\_save: │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:2995 in │  
│ evaluate │  
│ │  
│ 2992 │ │ start\_time = time.time() │  
│ 2993 │ │ │  
│ 2994 │ │ eval\_loop = self.prediction\_loop if self.args.use\_legacy\_prediction\_loop else se │  
│ ❱ 2995 │ │ output = eval\_loop( │  
│ 2996 │ │ │ eval\_dataloader, │  
│ 2997 │ │ │ description=“Evaluation”, │  
│ 2998 │ │ │ # No point gathering the predictions if there are no metrics, otherwise we d │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:3176 in │  
│ evaluation\_loop │  
│ │  
│ 3173 │ │ │ │ │ batch\_size = observed\_batch\_size │  
│ 3174 │ │ │ │  
│ 3175 │ │ │ # Prediction step │  
│ ❱ 3176 │ │ │ loss, logits, labels = self.prediction\_step(model, inputs, prediction\_loss\_o │  
│ 3177 │ │ │ inputs\_decode = self.\_prepare\_input(inputs[“input\_ids”]) if args.include\_inp │  
│ 3178 │ │ │ │  
│ 3179 │ │ │ if is\_torch\_tpu\_available(): │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/transformers/trainer.py:3431 in │  
│ prediction\_step │  
│ │  
│ 3428 │ │ │ else: │  
│ 3429 │ │ │ │ if has\_labels or loss\_without\_labels: │  
│ 3430 │ │ │ │ │ with self.compute\_loss\_context\_manager(): │  
│ ❱ 3431 │ │ │ │ │ │ loss, outputs = self.compute\_loss(model, inputs, return\_outputs= │  
│ 3432 │ │ │ │ │ │ # loss= self.compute\_loss(model, inputs, return\_outputs=True) │  
│ 3433 │ │ │ │ │ │  
│ 3434 │ │ │ │ │ loss = loss.mean().detach() │  
│ │  
│ /home/hexinyu/miniconda3/envs/nlp/lib/python3.10/site-packages/torch/\_tensor.py:930 in **iter** │  
│ │  
│ 927 │ │ # NB: We have intentionally skipped **torch\_function** dispatch here. │  
│ 928 │ │ # See gh-54457 │  
│ 929 │ │ if self.dim() == 0: │  
│ ❱ 930 │ │ │ raise TypeError(“iteration over a 0-d tensor”) │  
│ 931 │ │ if torch.\_C.\_get\_tracing\_state(): │  
│ 932 │ │ │ warnings.warn( │  
│ 933 │ │ │ │ "Iterating over a tensor might cause the trace to be incorrect. " │  
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯  
TypeError: iteration over a 0-d tensor

---

<div class="post-metadata">

**Author:** ![zhangsongbo365](https://avatars.discourse-cdn.com/v4/letter/z/ba9def/32.png) [@zhangsongbo365](https://discuss.huggingface.co/u/zhangsongbo365)\
**Post date:** [April 28, 2023, 5:45am UTC](https://discuss.huggingface.co/t/when-i-use-transformers-trainingarguments-i-got-loss-errors/37765/2 "2023-04-28T05:45:48Z")

</div>

I have the same problem. I have tried 4.27.1 and 4.28.1, also the same error.