# Load iterable dataset from disk

**URL:** <https://discuss.huggingface.co/t/load-iterable-dataset-from-disk/20161>\
**Category:** Beginners\
**Created:** [July 8, 2022, 7:47pm UTC](https://discuss.huggingface.co/t/load-iterable-dataset-from-disk/20161 "2022-07-08T19:47:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jrf](https://avatars.discourse-cdn.com/v4/letter/j/3be4f8/32.png) [@jrf](https://discuss.huggingface.co/u/jrf)\
**Post date:** [July 8, 2022, 7:47pm UTC](https://discuss.huggingface.co/t/load-iterable-dataset-from-disk/20161/1 "2022-07-08T19:47:52Z")

</div>

Hello, I’ve made a custom dataset and saved it to disk using datasets.save\_to\_disk. I would like to shuffle it together with common\_voice, but don’t currently have the diskspace for common\_voice.

I can load common\_voice with streaming=True and it becomes an iterable, which works fine.

So, my question is, can I somehow load my saved dataset as an iterable too so I can combine them using interleave\_datasets? datasets.load\_from\_disk doesn’t seem to have that option.

thanks!  
Jonathan

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [July 11, 2022, 1:33pm UTC](https://discuss.huggingface.co/t/load-iterable-dataset-from-disk/20161/2 "2022-07-11T13:33:01Z")

</div>

Hi! `load_from_disk` doesn’t support streaming at the moment, but you can save the dataset to JSON for instance and stream it from there with:

```python
custom_ds.to_json("path/to/data/file")
custom_ds_iter = load_dataset("json", data_files="path/to/data/file", split="train", streaming=True, features=custom_ds.features) 

```

---

<div class="post-metadata">

**Author:** ![jrf](https://avatars.discourse-cdn.com/v4/letter/j/3be4f8/32.png) [@jrf](https://discuss.huggingface.co/u/jrf)\
**Post date:** [July 21, 2022, 9:21pm UTC](https://discuss.huggingface.co/t/load-iterable-dataset-from-disk/20161/3 "2022-07-21T21:21:37Z")

</div>

Thank you – I finally got this to work on my low resource machine.
