# Image dataset with custom script best practices

**URL:** <https://discuss.huggingface.co/t/image-dataset-with-custom-script-best-practices/76517>\
**Category:** 🤗Datasets\
**Created:** [March 8, 2024, 10:59am UTC](https://discuss.huggingface.co/t/image-dataset-with-custom-script-best-practices/76517 "2024-03-08T10:59:51Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![adamnarozniak](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adamnarozniak/32/24231_2.png) [@adamnarozniak](https://discuss.huggingface.co/u/adamnarozniak)\
**Post date:** [March 8, 2024, 10:59am UTC](https://discuss.huggingface.co/t/image-dataset-with-custom-script-best-practices/76517/1 "2024-03-08T10:59:51Z")

</div>

Hi,  
I want to add an image dataset that requires a custom script.  
What is the recommended way of handling images in \_generate\_example in a custom script?  
Based on the docs ([link](https://huggingface.co/docs/datasets/v2.18.0/en/package_reference/main_classes#datasets.Features)) a few approaches are allowed: absolute path (alone), np.ndarray, PIL.Image.Image, dict with relate path and bytes `{"path": ..., "bytes": ...}`.

I saw a few different approaches applied to different datasets in the Hub. [MNIST](https://huggingface.co/datasets/mnist/blob/main/mnist.py) reads all the objects (images and labels) and then yields np.ndarray (I guess the aspect of reading all files prior to using yield won’t scale) (same approach in fashion\_mnist). [imagenette](https://huggingface.co/datasets/frgfm/imagenette/blob/main/imagenette.py) yields the dict with path and bytes. [alkzar90/NIH-Chest-X-ray-dataset](https://huggingface.co/datasets/alkzar90/NIH-Chest-X-ray-dataset/blob/main/NIH-Chest-X-ray-dataset.py) returns paths.

Which of these approaches gives the best performance, or are there any inclinations to choose one over another? Drawbacks? Pros? Should I pay special attention to anything? (I can maintain all of them etc. I’m interested in the best performance)