# Adding Audio-MAE to Transformers

**URL:** <https://discuss.huggingface.co/t/adding-audio-mae-to-transformers/136670>\
**Category:** 🤗Transformers\
**Created:** [January 21, 2025, 10:31am UTC](https://discuss.huggingface.co/t/adding-audio-mae-to-transformers/136670 "2025-01-21T10:31:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmed-Telili](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Ahmed-Telili](https://discuss.huggingface.co/u/Ahmed-Telili)\
**Post date:** [January 21, 2025, 10:31am UTC](https://discuss.huggingface.co/t/adding-audio-mae-to-transformers/136670/1 "2025-01-21T10:31:27Z")

</div>

Hello Hugging Face Community,

I have trained the Audio-MAE model on 30M underwater sound samples from the Orcasound dataset using 8 H100 GPUs for over a month. While exploring the Hugging Face Transformers library, I noticed that there is an implementation for Vision Transformer (ViT) MAE, but no implementation for Audio-MAE.

Given the growing interest in underwater audio-based models, I wanted to ask if it would be possible to add the Audio-MAE model to the Transformers library, pretrained on 30 million samples. I believe it could be valuable for various audio-related applications, especially in underwater acoustics.

Thank you in advance for your feedback!

Best regards,

---

<div class="post-metadata">

**Author:** ![Alanturner2](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alanturner2/32/37542_2.png) [@Alanturner2](https://discuss.huggingface.co/u/Alanturner2)\
**Post date:** [January 21, 2025, 1:20pm UTC](https://discuss.huggingface.co/t/adding-audio-mae-to-transformers/136670/2 "2025-01-21T13:20:43Z")

</div>

Hello @Ahmed-Telili !

Thank you for sharing your incredible work and for bringing this to the community’s attention! Training an Audio-MAE model on 30 million underwater sound samples is an impressive achievement, and your suggestion to add the model to the Hugging Face Transformers library is both valuable and exciting.

Here’s how you could move forward:

1. **Open a Feature Request** :

2. **Prepare a Pretrained Model for Sharing** :

3. **Contribute an Implementation** :

This initiative could significantly benefit the audio research community, especially in niche domains like underwater acoustics. Kudos to you for leading the way, and I’m excited to see where this goes!

Best regards,  
Alan.