# How to return PIL.Image from dataset generator?

**URL:** <https://discuss.huggingface.co/t/how-to-return-pil-image-from-dataset-generator/31160>\
**Category:** 🤗Datasets\
**Created:** [February 5, 2023, 7:56pm UTC](https://discuss.huggingface.co/t/how-to-return-pil-image-from-dataset-generator/31160 "2023-02-05T19:56:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](https://discuss.huggingface.co/u/jbmaxwell)\
**Post date:** [February 5, 2023, 7:56pm UTC](https://discuss.huggingface.co/t/how-to-return-pil-image-from-dataset-generator/31160/1 "2023-02-05T19:56:45Z")

</div>

I have a custom `GeneratorBasedBuilder` subclass that I’m trying to get running. It seems like I’m _almost_ there, but my trainer is failing with a type error when trying to access a generated sample—to be more specific, I’m working with the `train_text_to_image.py` script and it’s failing at the `preprocess_train(examples)` function…

From what I understand, this is because the actual data is a `PIL.Image`, but my generator’s `features` dict lists the features as `"image": datasets.Value("string")`, which is obviously incorrect. However, it doesn’t seem like I can indicate an Image as a type here, so how am I supposed to do this?..

In my `_generate_examples` function I’m creating two images and a string as the types for my three columns.

```auto
features=datasets.Features(
    {
        "image": [Image.Image],
        "text": datasets.Value(("string")),
        "conditioning_image": [Image.Image]
    }
)

```

In addition to the above, I’ve tried the original (template) `datasets.Value("string")`, `Image` (which fails with `cannot pickle module`), and `Image.Image`…

I’m doing some preprocessing in my `_generate_examples` function—basically adding a random mask—which I’d rather not do at the caller, if possible…

My `_generate_examples` function returns:

```auto
yield idx, { "image": target_img, "text": prompts, "conditioning_image": cond_img }

```

in case that helps… “target\_img” and “cond\_img” are both type `PIL.Image`

Why is this difficult… ???

---

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](https://discuss.huggingface.co/u/jbmaxwell)\
**Post date:** [February 5, 2023, 8:37pm UTC](https://discuss.huggingface.co/t/how-to-return-pil-image-from-dataset-generator/31160/2 "2023-02-05T20:37:12Z")

</div>

Digging around in the `ImageFolder` class (well, it’s superclass) I found `datasets.Image`, but it fails with no useful error:

```auto
Exception has occurred: DatasetGenerationError
An error occurred while generating the dataset

During handling of the above exception, another exception occurred:

The above exception was the direct cause of the following exception:

  File "/home/james/src/diffusers/examples/text_to_image/train_text_to_image.py", line 547, in main
    dataset = load_dataset(
  File "/home/james/src/diffusers/examples/text_to_image/train_text_to_image.py", line 831, in <module>
    main()

```

---

<div class="post-metadata">

**Author:** ![jbmaxwell](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@jbmaxwell](https://discuss.huggingface.co/u/jbmaxwell)\
**Post date:** [February 5, 2023, 8:48pm UTC](https://discuss.huggingface.co/t/how-to-return-pil-image-from-dataset-generator/31160/3 "2023-02-05T20:48:38Z")

</div>

okay, apologies for all the latent (or not so latent) rage… it was `datasets.Image()`… heh…