# Phi3 Mini 4k Instruct Flash Attention not found

**URL:** <https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613>\
**Category:** 🤗Transformers\
**Created:** [May 7, 2024, 10:31am UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613 "2024-05-07T10:31:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![swtb](https://avatars.discourse-cdn.com/v4/letter/s/8c91f0/32.png) [@swtb](https://discuss.huggingface.co/u/swtb)\
**Post date:** [May 7, 2024, 10:31am UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613/1 "2024-05-07T10:31:08Z")

</div>

When loading “microsoft/Phi-3-mini-4k-instruct” using transformers.pipeline with “load\_in\_4bit=True” I get a warning stating:

```auto
WARNING:transformers_modules.microsoft.Phi-3-mini-4k-instruct.920b6cf52a79ecff578cc33f61922b23cbc88115.modeling_phi3:`flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.

WARNING:transformers_modules.microsoft.Phi-3-mini-4k-instruct.920b6cf52a79ecff578cc33f61922b23cbc88115.modeling_phi3:Current `flash-attenton` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

```

Perhaps I have loaded it incorrectly? I have accelerate in my environment

---

<div class="post-metadata">

**Author:** ![HamzaYslmn](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hamzayslmn/32/26837_2.png) [@HamzaYslmn](https://discuss.huggingface.co/u/HamzaYslmn)\
**Post date:** [May 10, 2024, 6:04pm UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613/2 "2024-05-10T18:04:10Z")

</div>

+1 ı have same problem.

`flash-attention` package not found, consider installing for better performance: No module named ‘flash\_attn’.  
Current `flash-attenton` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

microsoft/Phi-3-mini-128k-instruct"

---

<div class="post-metadata">

**Author:** ![swtb](https://avatars.discourse-cdn.com/v4/letter/s/8c91f0/32.png) [@swtb](https://discuss.huggingface.co/u/swtb)\
**Post date:** [May 10, 2024, 9:02pm UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613/3 "2024-05-10T21:02:20Z")

</div>

Solution for me was to install the missing package.

I thought flash attention was handled in accelerate.

But I think phi3 implements their own version using a specific package.

You can just do pip install flash-attn

---

<div class="post-metadata">

**Author:** ![HamzaYslmn](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hamzayslmn/32/26837_2.png) [@HamzaYslmn](https://discuss.huggingface.co/u/HamzaYslmn)\
**Post date:** [May 10, 2024, 10:16pm UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613/4 "2024-05-10T22:16:57Z")

</div>

Collecting flash-attn  
Using cached flash\_attn-2.5.8.tar.gz (2.5 MB)  
Installing build dependencies … done  
Getting requirements to build wheel … error  
error: subprocess-exited-with-error

note: This error originates from a subprocess, and is likely not a problem with pip.  
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.  
│ exit code: 1  
╰─\> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [May 11, 2024, 10:17am UTC](https://discuss.huggingface.co/t/phi3-mini-4k-instruct-flash-attention-not-found/85613/5 "2024-05-11T10:17:42Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.