# ValueError: Image features and image tokens do not match

**URL:** <https://discuss.huggingface.co/t/valueerror-image-features-and-image-tokens-do-not-match/150302>\
**Category:** 🤗Transformers\
**Created:** [April 14, 2025, 7:47am UTC](https://discuss.huggingface.co/t/valueerror-image-features-and-image-tokens-do-not-match/150302 "2025-04-14T07:47:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brianzhengca](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/brianzhengca/32/45344_2.png) [@Brianzhengca](https://discuss.huggingface.co/u/Brianzhengca)\
**Post date:** [April 14, 2025, 7:47am UTC](https://discuss.huggingface.co/t/valueerror-image-features-and-image-tokens-do-not-match/150302/1 "2025-04-14T07:47:57Z")

</div>

I am trying to use assistant\_model for Llava 7B, but it seems like nothing is working. Transformers version = 4.51.2  
Reproducible code example:

```auto
from transformers import LlavaOnevisionForConditionalGeneration, LlavaOnevisionProcessor
from PIL import Image

import torch
import requests

img_urls =["https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png",
           "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"]
images = [Image.open(requests.get(img_urls[0], stream=True).raw),
          Image.open(requests.get(img_urls[1], stream=True).raw)]

target_processor = LlavaOnevisionProcessor.from_pretrained("llava-hf/llava-onevision-qwen2-7b-ov-hf")
target_processor.tokenizer.padding_side = "left"
draft_processor = LlavaOnevisionProcessor.from_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf")
draft_processor.tokenizer.padding_side = "left"
target = LlavaOnevisionForConditionalGeneration.from_pretrained("llava-hf/llava-onevision-qwen2-7b-ov-hf").to('cuda')
draft = LlavaOnevisionForConditionalGeneration.from_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf").to('cuda')

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Describe this image in 500 words."}
        ]
    }
]

prompt = target_processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = target_processor(text=prompt, images=[images[0]], return_tensors="pt").to("cuda")

with torch.no_grad():
    generated_ids = target.generate(**inputs, max_new_tokens=1000, assistant_model=draft, tokenizer=target_processor.tokenizer, assistant_tokenizer=draft_processor.tokenizer)
generated_texts = target_processor.batch_decode(generated_ids, skip_special_tokens=True)

print(generated_texts)

```

I kept getting this error: ` raise ValueError( ValueError: Image features and image tokens do not match: tokens: 0, features 2709`

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 14, 2025, 8:05am UTC](https://discuss.huggingface.co/t/valueerror-image-features-and-image-tokens-do-not-match/150302/2 "2025-04-14T08:05:14Z")

</div>

Similar issue?

> <https://github.com/huggingface/transformers/issues/36002>
>
> \*\*Model: llava-hf/llava-1.5-7b-hf\*\*
> \*\*Issue Description\*\*
> When I try to generate… a response using the fine-tuned model, I encounter the following error:
> ValueError: Image features and image tokens do not match: tokens: 575, features: 576
> 
> This error occurs during the generate() call, indicating a mismatch between the number of image tokens and image features.
> 
> \*\*Steps I’ve Taken:\*\*
> 
> \- Image Preprocessing:
> \- I resized the input image to dimensions that are multiples of the patch\_size (14 for LLaVA models).
> \- The image is resized to 518x336, which is a multiple of 14.
> 
> \*\*Processor Configuration:\*\*
> 
> \- I manually set the patch\_size and vision\_feature\_select\_strategy in the processor to match the model's configuration.
> \- I verified that the processor's configuration is correct.
> 
> \*\*Debugging Inputs:\*\*
> 
> \- I printed the inputs (input\_ids, pixel\_values, etc.) to ensure they are correctly formatted.
> \- The inputs are moved to the GPU for processing.
> 
> \*\*Model Loading:\*\*
> The model and processor are loaded from the fine-tuned directory, and the model is moved to the GPU.
> 
> Code Snippet
> Here’s the relevant part of my code:
> 
> \`
> \`\`\`
> \#Load the processor and model
> processor1 = LlavaProcessor.from\_pretrained(model\_path)
> new\_model\_v1 = LlavaForConditionalGeneration.from\_pretrained(model\_path).to("cuda:0")
> 
> \#Resize the image
> patch\_size = new\_model\_v1.config.vision\_config.patch\_size
> shortest\_edge = processor1.image\_processor.size.get("shortest\_edge", 336)
> original\_width, original\_height = raw\_image.size
> scale\_factor = shortest\_edge / min(original\_width, original\_height)
> new\_width = int(original\_width \* scale\_factor)
> new\_height = int(original\_height \* scale\_factor)
> 
> new\_width = (new\_width // patch\_size) \* patch\_size
> new\_height = (new\_height // patch\_size) \* patch\_size
> raw\_image = raw\_image.resize((new\_width, new\_height)) # Resized to multiples of patch\_size
> 
> \#Process inputs
> inputs = processor1(images=raw\_image, text=prompt, return\_tensors='pt')
> inputs = {k: v.to("cuda:0") for k, v in inputs.items()}
> 
> \#Generate response
> output = new\_model\_v1.generate(
> input\_ids=inputs\["input\_ids"\],
> pixel\_values=inputs\["pixel\_values"\],
> max\_new\_tokens=200,
> do\_sample=False
> )
> 
> \`\`\`
> \*\*Questions:\*\*
> 
> \- Why is there a mismatch between the number of image tokens (575) and image features (576)?
> \- Is there a mistake in my image preprocessing or model configuration that could cause this issue?
> \- How can I ensure that the number of image tokens matches the number of image features?
> \- Are there any additional steps I need to take to align the image tokens and features correctly?
> 
> \*\*Additional Information\*\*
> 
> \- I have tried the Hugging Face transformers library both version 4.48.2 and 4.48.2.
> \- The fine-tuned model is saved in Google Drive, and I’m loading it in a new Colab session.
> \- The base model is llava-hf/llava-1.5-7b-hf.
> 
> \<img width="885" alt="Image" src="https://github.com/user-attachments/assets/444c2286-629d-4385-85df-6c485a27f076" /\>
> \<img width="418" alt="Image" src="https://github.com/user-attachments/assets/3a24ce8b-09ae-4a36-b54f-3de0946619b8" /\>

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [April 14, 2025, 8:18am UTC](https://discuss.huggingface.co/t/valueerror-image-features-and-image-tokens-do-not-match/150302/3 "2025-04-14T08:18:36Z")

</div>

It seems to be an error that occurs easily, and it is difficult to find the cause…

> <https://github.com/huggingface/transformers/issues/35775>
>
> \### System Info
> 
> \- \`transformers\` version: 4.48.0
> \- Platform: Linux-5.15.0-1067-…nvidia-x86\_64-with-glibc2.35
> \- Python version: 3.11.11
> \- Huggingface\_hub version: 0.27.1
> \- Safetensors version: 0.5.2
> \- Accelerate version: 1.2.1
> \- Accelerate config: - compute\_environment: LOCAL\_MACHINE
> - distributed\_type: FSDP
> - mixed\_precision: bf16
> - use\_cpu: False
> - debug: False
> - num\_processes: 1
> - machine\_rank: 0
> - num\_machines: 1
> - rdzv\_backend: static
> - same\_network: True
> - main\_training\_function: main
> - enable\_cpu\_affinity: False
> - fsdp\_config: {'fsdp\_activation\_checkpointing': True, 'fsdp\_auto\_wrap\_policy': 'TRANSFORMER\_BASED\_WRAP', 'fsdp\_backward\_prefetch': 'BACKWARD\_PRE', 'fsdp\_cpu\_ram\_efficient\_loading': True, 'fsdp\_forward\_prefetch': False, 'fsdp\_offload\_params': False, 'fsdp\_sharding\_strategy': 'FULL\_SHARD', 'fsdp\_state\_dict\_type': 'SHARDED\_STATE\_DICT', 'fsdp\_sync\_module\_states': True, 'fsdp\_transformer\_layer\_cls\_to\_wrap': '', 'fsdp\_use\_orig\_params': True}
> - downcast\_bf16: no
> - tpu\_use\_cluster: False
> - tpu\_use\_sudo: False
> - tpu\_env: \[\]
> \- PyTorch version (GPU?): 2.5.1 (True)
> \- Tensorflow version (GPU?): not installed (NA)
> \- Flax version (CPU?/GPU?/TPU?): not installed (NA)
> \- Jax version: not installed
> \- JaxLib version: not installed
> \- Using distributed or parallel set-up in script?: False
> \- Using GPU in script?: True
> \- GPU type: NVIDIA H100 80GB HBM3
> 
> \### Who can help?
> 
> @amyeroberts @qubvel @zucchini-nlp 
> 
> \### Information
> 
> \- \[\] The official example scripts
> \- \[x\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[x\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> \`\`\`python
> from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
> from datasets import load\_dataset
> import torch
> processor = AutoProcessor.from\_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf")
> model = LlavaOnevisionForConditionalGeneration.from\_pretrained("llava-hf/llava-onevision-qwen2-0.5b-ov-hf", torch\_dtype=torch.float16, device\_map="auto")
> dataset = load\_dataset("lmms-lab/docvqa", 'DocVQA')
> 
> d = dataset\['test'\]\[2482\]
> question = d\['question'\]
> image = d\['image'\]
> conversation = \[
> {
> "role": "user",
> "content": \[
> {"type": "image"},
> {"type": "text", "text": question},
> \],
> },
> \]
> prompt = processor.apply\_chat\_template(conversation, add\_generation\_prompt=True)
> inputs = processor(images=image, text=prompt, return\_tensors="pt").to("cuda")
> with torch.no\_grad():
> outputs = model(\*\*inputs)
> \`\`\`
> 
> Traceback as follows:
> \`\`\`
> Traceback (most recent call last):
> File "\<stdin\>", line 2, in \<module\>
> File "/mnt/home/miniforge3/envs/vek/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in \_wrapped\_call\_impl
> return self.\_call\_impl(\*args, \*\*kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/mnt/home/miniforge3/envs/vek/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in \_call\_impl
> return forward\_call(\*args, \*\*kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/mnt/home/miniforge3/envs/vek/lib/python3.11/site-packages/transformers/models/llava\_onevision/modeling\_llava\_onevision.py", line 688, in forward
> raise ValueError(
> ValueError: Image features and image tokens do not match: tokens: 7332, features 7261
> \`\`\`
> 
> \### Expected behavior
> 
> Expected: Output correctly without errors.
> 
> This is a follow-up issue of https://github.com/huggingface/transformers/issues/34625, where the behavior is the same but for different reasons. The reproduction example is a slight modification of the one provided by @chchch0109.

> <https://github.com/huggingface/transformers/issues/34284>
>
> \### System Info
> 
> transformers version: 4.46.0.dev0
> 
> \### Who can help?
> 
> @amyerobe…rts, @qubvel
> 
> \### Information
> 
> \- \[\] The official example scripts
> \- \[\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> Before #33608 (testing on commit d00f1ca860f19f4c0962882e56044bb6ef7b5626) the code below would run without error:
> 
> \`\`\` python
> import torch
> from transformers import LlavaForConditionalGeneration, LlavaProcessor
> 
> model = LlavaForConditionalGeneration.from\_pretrained("llava-hf/llava-1.5-7b-hf")
> processor = LlavaProcessor.from\_pretrained("llava-hf/llava-1.5-7b-hf")
> processor.patch\_size = 14
> processor.vision\_feature\_select\_strategy = "default"
> 
> device = torch.device("cuda")
> model = model.eval()
> model = model.to(device)
> inputs = processor(
> text=\["Sentence with two images 1. \<image\> 2. \<image\>", "Sentence with one image \<image\>"\],
> images= torch.rand((3, 3, 336, 336), dtype=torch.float),
> return\_tensors="pt",
> truncation=True,
> padding=True,
> )
> inputs = inputs.to(device)
> with torch.no\_grad():
> model(\*\*inputs)
> \`\`\`
> 
> However, after #33608 (testing on commit 0f49deacbff3e57cde45222842c0db6375e4fa43), it fails with the error
> \`\`\`
> \---------------------------------------------------------------------------
> ValueError Traceback (most recent call last)
> Cell In\[1\], line 21
> 19 inputs = inputs.to(device)
> 20 with torch.no\_grad():
> \---\> 21 model(\*\*inputs)
> 
> File ~/miniconda3/envs/hf/lib/python3.10/site-packages/torch/nn/modules/module.py:1736, in Module.\_wrapped\_call\_impl(self, \*args, \*\*kwargs)
> 1734 return self.\_compiled\_call\_impl(\*args, \*\*kwargs) # type: ignore\[misc\]
> 1735 else:
> \-\> 1736 return self.\_call\_impl(\*args, \*\*kwargs)
> 
> File ~/miniconda3/envs/hf/lib/python3.10/site-packages/torch/nn/modules/module.py:1747, in Module.\_call\_impl(self, \*args, \*\*kwargs)
> 1742 # If we don't have any hooks, we want to skip the rest of the logic in
> 1743 # this function, and just call forward.
> 1744 if not (self.\_backward\_hooks or self.\_backward\_pre\_hooks or self.\_forward\_hooks or self.\_forward\_pre\_hooks
> 1745 or \_global\_backward\_pre\_hooks or \_global\_backward\_hooks
> 1746 or \_global\_forward\_hooks or \_global\_forward\_pre\_hooks):
> \-\> 1747 return forward\_call(\*args, \*\*kwargs)
> 1749 result = None
> 1750 called\_always\_called\_hooks = set()
> 
> File ~/miniconda3/envs/hf/lib/python3.10/site-packages/transformers/models/llava/modeling\_llava.py:524, in LlavaForConditionalGeneration.forward(self, input\_ids, pixel\_values, attention\_mask, position\_ids, past\_key\_values, inputs\_embeds, vision\_feature\_layer, vision\_feature\_select\_strategy, labels, use\_cache, output\_attentions, output\_hidden\_states, return\_dict, cache\_position, num\_logits\_to\_keep)
> 522 n\_image\_features = image\_features.shape\[1\]
> 523 if n\_image\_tokens != n\_image\_features:
> \--\> 524 raise ValueError(
> 525 f"Image features and image tokens do not match: tokens: {n\_image\_tokens}, features {n\_image\_features}"
> 526 )
> 527 special\_image\_mask = (
> 528 (input\_ids == self.config.image\_token\_index)
> 529 .unsqueeze(-1)
> 530 .expand\_as(inputs\_embeds)
> 531 .to(inputs\_embeds.device)
> 532 )
> 533 image\_features = image\_features.to(inputs\_embeds.device, inputs\_embeds.dtype)
> 
> ValueError: Image features and image tokens do not match: tokens: 1152, features 576
> \`\`\`
> 
> \### Expected behavior
> 
> Before #33608 multi-image input and variable-image input to LLaVa worked as expected. The added check on image features and image tokens in #33608 doesn't seem to take into account 1. input sequences with multiple images 2. batches with variable number of images in each input sequence.

> **[llava-hf/llava-1.5-7b-hf · HELP](https://huggingface.co/llava-hf/llava-1.5-7b-hf/discussions/37)**
>
> I'm trying to use llava-1.5-7b-hf and i'm new and clueless in LMMs but i have an error when i try to use the simple example raise ValueError( ValueError: The input provided to the model are wrong. ...