# Issue with Transformer notebook's Getting Started Tokenizers

**URL:** <https://discuss.huggingface.co/t/issue-with-transformer-notebooks-getting-started-tokenizers/3444>\
**Category:** Beginners\
**Created:** [January 29, 2021, 11:11pm UTC](https://discuss.huggingface.co/t/issue-with-transformer-notebooks-getting-started-tokenizers/3444 "2021-01-29T23:11:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![krwin](https://avatars.discourse-cdn.com/v4/letter/k/e495f1/32.png) [@krwin](https://discuss.huggingface.co/u/krwin)\
**Post date:** [January 29, 2021, 11:11pm UTC](https://discuss.huggingface.co/t/issue-with-transformer-notebooks-getting-started-tokenizers/3444/1 "2021-01-29T23:11:56Z")

</div>

## In Documentation, I went to Transformer Notebooks and clicked on the collab for Getting Started Tokenizer. I executed each cell and when I got to the cell where:

from tokenizers.trainers import BpeTrainer  
trainer = BpeTrainer(vocab\_size=25000, show\_progress=True, initial\_alphabet=ByteLevel.alphabet())  
tokenizer.train(trainer, [“big.txt”])

## print(“Trained vocab size: {}”.format(tokenizer.get\_vocab\_size()))

I ran the cell and got this:

## TypeError: Can’t convert \<tokenizers.trainers.BpeTrainer object at 0x7f8641325570\> to Sequence

I am assuming these cells should work so something changed with the software but not updated in the notebook. I am trying to learn transformers on my own so where can I go to learn if Hugging Face Doc is not up to date? Any help will be appreciated.

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [January 30, 2021, 11:43am UTC](https://discuss.huggingface.co/t/issue-with-transformer-notebooks-getting-started-tokenizers/3444/2 "2021-01-30T11:43:48Z")

</div>

Hi @krwin, this indeed seems to be a bug in the [notebook](https://colab.research.google.com/github/huggingface/transformers/blob/master/notebooks/01-training-tokenizers.ipynb), where the order of the arguments for `tokenizer.train()` in this cell

```auto
from tokenizers.trainers import BpeTrainer

# We initialize our trainer, giving him the details about the vocabulary we want to generate
trainer = BpeTrainer(vocab_size=25000, show_progress=True, initial_alphabet=ByteLevel.alphabet())
tokenizer.train(trainer, ["big.txt"])

print("Trained vocab size: {}".format(tokenizer.get_vocab_size()))

```

is back-to-front (see the [docs](https://huggingface.co/docs/tokenizers/python/latest/api/reference.html#tokenizers.Tokenizer.train)). To fix the problem you can just specify the arguments explicitly:

```auto
tokenizer.train(trainer=trainer, files=["big.txt"])

```

cc: @anthony

---

<div class="post-metadata">

**Author:** ![krwin](https://avatars.discourse-cdn.com/v4/letter/k/e495f1/32.png) [@krwin](https://discuss.huggingface.co/u/krwin)\
**Post date:** [January 30, 2021, 9:58pm UTC](https://discuss.huggingface.co/t/issue-with-transformer-notebooks-getting-started-tokenizers/3444/3 "2021-01-30T21:58:44Z")

</div>

Thank you very much for helping me and being so prompt.  
Hope you have a great day!