# Facebook BART Fine-tuning - Transformers CUDA error: CUBLAS\_STATUS\_NOT\_INITIALIZE

**URL:** <https://discuss.huggingface.co/t/facebook-bart-fine-tuning-transformers-cuda-error-cublas-status-not-initialize/37680>\
**Category:** 🤗Transformers\
**Created:** [April 25, 2023, 12:08pm UTC](https://discuss.huggingface.co/t/facebook-bart-fine-tuning-transformers-cuda-error-cublas-status-not-initialize/37680 "2023-04-25T12:08:03Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![cog](https://avatars.discourse-cdn.com/v4/letter/c/df788c/32.png) [@cog](https://discuss.huggingface.co/u/cog)\
**Post date:** [May 2, 2023, 6:00am UTC](https://discuss.huggingface.co/t/facebook-bart-fine-tuning-transformers-cuda-error-cublas-status-not-initialize/37680/2 "2023-05-02T06:00:13Z")

</div>

hi @LidorPrototype ,

First, check your own dataset will match with `facebook/bart-base` 's vocab size.  
Second, check word that in your own dataset is in model’s vocabulary.  
Third, check model, dataloader, tokenizer config is correct. There many mistake are config’s wrong path or config setting is wrong.

As you see this [article](https://discuss.huggingface.co/t/masked-language-model-for-bart-not-bert/19945/4), if your own dataset contain some data that dose not support at model’s support vocab, you can face that error.

If you want to check detail, please debug some easy code like docs example code.  
Follow line with your own dataset, you can see trackback message more detail.  
(Some data not contain vocab, mismatch config, etc…)

[bart docs page](https://huggingface.co/docs/transformers/v4.28.1/en/model_doc/bart#bart)

```auto
from transformers import AutoTokenizer, BartModel
import torch

tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base")
model = BartModel.from_pretrained("facebook/bart-base")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state

```

---

_[View the full topic](https://discuss.huggingface.co/t/facebook-bart-fine-tuning-transformers-cuda-error-cublas-status-not-initialize/37680)._
