# Multi-GPU Distributed Training using Accelerate on Windows

**URL:** <https://discuss.huggingface.co/t/multi-gpu-distributed-training-using-accelerate-on-windows/50071>\
**Category:** 🤗Accelerate\
**Created:** [August 9, 2023, 4:38am UTC](https://discuss.huggingface.co/t/multi-gpu-distributed-training-using-accelerate-on-windows/50071 "2023-08-09T04:38:47Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rtb1271](https://avatars.discourse-cdn.com/v4/letter/r/76d3ee/32.png) [@rtb1271](https://discuss.huggingface.co/u/rtb1271)\
**Post date:** [August 9, 2023, 4:38am UTC](https://discuss.huggingface.co/t/multi-gpu-distributed-training-using-accelerate-on-windows/50071/1 "2023-08-09T04:38:47Z")

</div>

I am trying to use multi-gpu distributed training on a model using the Accelerate library. I have already setup my congifs using accelerate config and am using accelerate launch train.py but I keep getting the following errors:

```auto
raise RuntimeError("Distributed package doesn't have NCCL " "built in")
RuntimeError: Distributed package doesn't have NCCL built in
ERROR:torch.distributed.elastic.multiprocessing.api:failed

raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError

```

I am running this on a Windows system and I understand that NCCL is not available on Windows. Would appreciate if anyone could provide a workaround for Windows 😃