# Modifying ViT to include 4th channel

**URL:** <https://discuss.huggingface.co/t/modifying-vit-to-include-4th-channel/54086>\
**Category:** 🤗Transformers\
**Created:** [September 8, 2023, 10:34am UTC](https://discuss.huggingface.co/t/modifying-vit-to-include-4th-channel/54086 "2023-09-08T10:34:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pmoct](https://avatars.discourse-cdn.com/v4/letter/p/58956e/32.png) [@pmoct](https://discuss.huggingface.co/u/pmoct)\
**Post date:** [September 8, 2023, 10:34am UTC](https://discuss.huggingface.co/t/modifying-vit-to-include-4th-channel/54086/1 "2023-09-08T10:34:04Z")

</div>

Hello!

I have a series of image data with RGB channels, but I have also added another channel containing a segmentation output that could potentially help the model with the classification task I am working on. I am utilizing the code from the image classification github ([link](https://github.com/huggingface/transformers/blob/main/examples/pytorch/image-classification/run_image_classification.py)) to get started, but I notice that it takes 3-channel images. How would I begin to modify the code to take in a 4-channel image?

I appreciate any advice or help!

---

<div class="post-metadata">

**Author:** ![da2r-20](https://avatars.discourse-cdn.com/v4/letter/d/c68b51/32.png) [@da2r-20](https://discuss.huggingface.co/u/da2r-20)\
**Post date:** [October 15, 2024, 8:47am UTC](https://discuss.huggingface.co/t/modifying-vit-to-include-4th-channel/54086/2 "2024-10-15T08:47:11Z")

</div>

Hi,  
did you find a solution?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 15, 2024, 9:28am UTC](https://discuss.huggingface.co/t/modifying-vit-to-include-4th-channel/54086/3 "2024-10-15T09:28:04Z")

</div>

I think you can do just the loading part of the image with a single character change, but I don’t know if the transformers library can handle this correctly…

```py
def pil_loader(path: str):
    with open(path, "rb") as f:
        im = Image.open(f)
        return im.convert("RGBA") # 4 channels
        #return im.convert("RGB") # 3 channels

```