# Custom trainer evaluation function

**URL:** <https://discuss.huggingface.co/t/custom-trainer-evaluation-function/19394>\
**Category:** Intermediate\
**Created:** [June 20, 2022, 7:21pm UTC](https://discuss.huggingface.co/t/custom-trainer-evaluation-function/19394 "2022-06-20T19:21:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![wz1232](https://avatars.discourse-cdn.com/v4/letter/w/ebca7d/32.png) [@wz1232](https://discuss.huggingface.co/u/wz1232)\
**Post date:** [June 20, 2022, 7:21pm UTC](https://discuss.huggingface.co/t/custom-trainer-evaluation-function/19394/1 "2022-06-20T19:21:14Z")

</div>

Hi, so I am trying to override the Trainer evaluate() function with my own method. It runs, but the problem is that when I use multi-GPUs (8 in my case), it seems to split the eval dataset across the 8 GPUs but only report the metric calculated for the eval subset on 1 GPU.

My boilerplate code is shown in here: [scratch/trainer\_eval.py at master · kevinghst/scratch · GitHub](https://github.com/kevinghst/scratch/blob/master/trainer_eval.py)

Can someone please help take a look and identify the underlying reason for this behavior?

Thanks,  
Kevin