# Finetuned I-Bert for question answering task

**URL:** <https://discuss.huggingface.co/t/finetuned-i-bert-for-question-answering-task/12147>\
**Category:** 🤗Transformers\
**Created:** [November 22, 2021, 7:41am UTC](https://discuss.huggingface.co/t/finetuned-i-bert-for-question-answering-task/12147 "2021-11-22T07:41:23Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mamoon](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@Mamoon](https://discuss.huggingface.co/u/Mamoon)\
**Post date:** [November 22, 2021, 7:41am UTC](https://discuss.huggingface.co/t/finetuned-i-bert-for-question-answering-task/12147/1 "2021-11-22T07:41:23Z")

</div>

How can we fine-tune an Integer Bert for question answering task?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [November 22, 2021, 10:28am UTC](https://discuss.huggingface.co/t/finetuned-i-bert-for-question-answering-task/12147/2 "2021-11-22T10:28:46Z")

</div>

Hi,

As explained in [IBERT’s model card](https://huggingface.co/kssteven/ibert-roberta-base): fine-tuning the model consists of 3 stages:

1. full precision fine-tuning
2. quantization
3. quantization-aware training.

So for the first step, you can fine-tune IBERT just as any BERT model. You can take a look at the [example scripts](https://github.com/huggingface/transformers/tree/master/examples/pytorch/question-answering) as well as the [official QA notebook](https://github.com/huggingface/notebooks/blob/master/examples/question_answering.ipynb).

Step 2 is simply setting the `quantize` attribute of the model’s configuration to `True`.

Step 3 is the same as step 1, but now with your quantized model.