# Calculate custom dataset size

**URL:** <https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113>\
**Category:** 🤗Datasets\
**Created:** [June 13, 2022, 5:15pm UTC](https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113 "2022-06-13T17:15:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](https://discuss.huggingface.co/u/aclifton314)\
**Post date:** [June 13, 2022, 5:15pm UTC](https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113/1 "2022-06-13T17:15:21Z")

</div>

Suppose I have a custom dataset and have converted it to a HF Datasets object. Is there a way to calculate the dataset size in GB from this object?

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [June 13, 2022, 5:52pm UTC](https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113/2 "2022-06-13T17:52:14Z")

</div>

Hi! You can use the following formula to get the size in GB from a HF dataset: `hf_dataset.data.nbytes / 1e9`

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [June 14, 2022, 4:51pm UTC](https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113/3 "2022-06-14T16:51:07Z")

</div>

Note that for vision and speech datasets:

`hf_dataset.data` may only contain the paths to local files. If you want to get the size in bytes of all the image/audio files, you might need to iterate over the image/audio files by yourself and check their sizes.

---

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](https://discuss.huggingface.co/u/aclifton314)\
**Post date:** [June 15, 2022, 9:39pm UTC](https://discuss.huggingface.co/t/calculate-custom-dataset-size/19113/4 "2022-06-15T21:39:25Z")

</div>

Fantastic. Many thanks @mariosasko and @lhoestq!