# How to Create one Process But Using Multi GPU?

**URL:** <https://discuss.huggingface.co/t/how-to-create-one-process-but-using-multi-gpu/39938>\
**Category:** DeepSpeed\
**Created:** [May 15, 2023, 12:10pm UTC](https://discuss.huggingface.co/t/how-to-create-one-process-but-using-multi-gpu/39938 "2023-05-15T12:10:30Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![fahadh4ilyas](https://avatars.discourse-cdn.com/v4/letter/f/bc79bd/32.png) [@fahadh4ilyas](https://discuss.huggingface.co/u/fahadh4ilyas)\
**Post date:** [May 15, 2023, 12:10pm UTC](https://discuss.huggingface.co/t/how-to-create-one-process-but-using-multi-gpu/39938/1 "2023-05-15T12:10:30Z")

</div>

So, after I check [Memory Requirement](https://huggingface.co/docs/transformers/main_classes/deepspeed#memory-requirements) and applied deepspeed stage 3 to my script. I tried to run my script to a pc with 2 GPU. But, somehow deepspeed creating one process per GPU and making the usage of RAM double the amount of memory requirement. I suspect because each process load it’s own model so each offloading its params and optimizer to CPU making the usage of RAM doubled.

Is there a way to force deepspeed only create one process but using 2 GPU instead of making one process for each of GPUS?