# Deepspeed integration with Trainer in Colab crashing: TypeError: object.\_\_init\_\_() takes exactly one argument (the instance to initialize) **URL:** \ **Category:** Intermediate\ **Created:** [December 21, 2022, 9:37pm UTC](https://discuss.huggingface.co/t/deepspeed-integration-with-trainer-in-colab-crashing-typeerror-object-init-takes-exactly-one-argument-the-instance-to-initialize/28255 "2022-12-21T21:37:05Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![seamusl](https://avatars.discourse-cdn.com/v4/letter/s/f19dbf/32.png) [@seamusl](https://discuss.huggingface.co/u/seamusl)\ **Post date:** [December 21, 2022, 9:37pm UTC](https://discuss.huggingface.co/t/deepspeed-integration-with-trainer-in-colab-crashing-typeerror-object-init-takes-exactly-one-argument-the-instance-to-initialize/28255/1 "2022-12-21T21:37:05Z")
Using hugging face trainer, I am training models in Colab notebook with no problems. I now need to use deepspeed since I’m running out of memory. Deepspeed was installed without any problems using pip install deepspeed (Torch 1.13) already installed. When I run !ds\_report in the notebook, all looks good. However, when I add deepspeed=ds\_config\_dict to the end of my TrainingArguments, it crashes with the following: — START crash details — /usr/local/lib/python3.8/dist-packages/transformers/deepspeed.py in **init** (self, config\_file\_or\_dict) 65 dep\_version\_check(“accelerate”) 66 dep\_version\_check(“deepspeed”) —\> 67 super(). **init** (config\_file\_or\_dict) 68 TypeError: object. **init** () takes exactly one argument (the instance to initialize) — END crash details — I’ve tried lots of combinations i.e. different config dicts and config stored in a json file on disk. I also looked for solutions online but I haven’t come across the problem. Any help appreciated. Thanks. The following is the cell from my notebook. ds\_config\_dict = { “zero\_optimization”: { “stage”: 2, “offload\_optimizer”: { “device”: “cpu”, “pin\_memory”: True }, “allgather\_partitions”: True, “allgather\_bucket\_size”: 2e8, “reduce\_scatter”: True, “reduce\_bucket\_size”: 2e8, “overlap\_comm”: True, “contiguous\_gradients”: True } } BS = 10 GRAD\_ACC = 2 LR = 5e-5 WD = 0.01 WARMUP = 0.1 N\_EPOCHS = 5 model\_name = model\_checkpoint.split(“/”)[-1] !echo $ds\_config\_dict args = TrainingArguments( f"{model\_name}-finetuned-{source\_lang}-to-{target\_lang}", evaluation\_strategy = “epoch”, logging\_strategy = “epoch”, save\_strategy = “epoch”, learning\_rate=LR, per\_device\_train\_batch\_size=BS, per\_device\_eval\_batch\_size=BS, num\_train\_epochs=N\_EPOCHS, weight\_decay=WD, # report\_to=‘wandb’, ``` gradient_accumulation_steps=GRAD_ACC, warmup_ratio=WARMUP, fp16 = True, deepspeed=ds_config_dict ``` ) ---
**Author:** ![seamusl](https://avatars.discourse-cdn.com/v4/letter/s/f19dbf/32.png) [@seamusl](https://discuss.huggingface.co/u/seamusl)\ **Post date:** [December 23, 2022, 11:30pm UTC](https://discuss.huggingface.co/t/deepspeed-integration-with-trainer-in-colab-crashing-typeerror-object-init-takes-exactly-one-argument-the-instance-to-initialize/28255/2 "2022-12-23T23:30:23Z")
Needed the following and it’s working now: # DeepSpeed requires a distributed environment even when only one process is used. # This emulates a launcher in the notebook import os os.environ[“MASTER\_ADDR”] = “localhost” os.environ[“MASTER\_PORT”] = “9994” # modify if RuntimeError: Address already in use os.environ[“RANK”] = “0” os.environ[“LOCAL\_RANK”] = “0” os.environ[“WORLD\_SIZE”] = “1” ---
**Author:** ![Aillian](https://avatars.discourse-cdn.com/v4/letter/a/22d042/32.png) [@Aillian](https://discuss.huggingface.co/u/Aillian)\ **Post date:** [October 1, 2023, 10:56pm UTC](https://discuss.huggingface.co/t/deepspeed-integration-with-trainer-in-colab-crashing-typeerror-object-init-takes-exactly-one-argument-the-instance-to-initialize/28255/3 "2023-10-01T22:56:30Z")
How to setup deepspeed in a multi gpu system? I use runpod instance with 8 A6000 GPUs and i get the same error