Hello, I like to change the number of labels that a trained model has. I am loading a model that was trained on 17 classes and I like adapt this model to my own task. Now if I simply change the number of labels like this: model_checkpoint ="vblagoje/bert-english-uncased-finetuned-pos" model = Aut…

model.classifier = nn.Linear(786,1) # you have 1 class? I think you should change it to 2 model.num_labels = 2 # while here you specify 2 classes so its a bit confusing Unless you are aiming for a sigmoid function for your last layer is thats why your adding 1 class then i think you need to change…

Check this. Its neatly implemented here. And a pretty neat architecture as well. model —> downstream_task . Well defined. legacyai/tf-transformers/blob/main/src/tf_transformers/notebooks/tutorials/bert_cola_text_classification.ipynb { "cells": [ { "cell_type": "code", "execution_…

Thanks for your help! It was way too late when I wrote this :see_no_evil: It turned out that the easiest way to solve my initial question is this: config = AutoConfig.from_pretrained(model_checkpoint) config.num_labels = 2 model = AutoModelForTokenClassification.from_config(config)

It works, but how this change affects the model architecture, and the results? It would be great if anyone can explain the intuition behind this.

I don’t think this solved your problem. Initialising model with ‘from_config’ only changes model configuration and it does not load model weight. Does anyone know how to solve this problem?

Changing the classification head scenarios suggested above does not work for my case. Instead, I suggest, which works for me, that you can change the body instead of head as follows old_model= BertForSequenceClassification.from_pretrained("model-x") new_model=BertForSequenceClassification.from_pre…

This is now possible (thanks to @sgugger ) by passing in an additional argument called ignore_mismatched_sizes, which you can set to True. If you have an already fine-tuned model with, let’s say 17 labels, and you want to replace the head with one that has 10 outputs, you can do it as follows: from…

Hi @oliverguhr Which solution worked for you for binary classification?

Thank you! This fixed my problem too! It was weird that this didn’t work: AutoModelForSequenceClassification.from_pretrained(“huggingface/CodeBERTa-language-id”, num_labels=15) but this did: config = AutoConfig.from_pretrained(“huggingface/CodeBERTa-language-id”) config.num_labels = 15 model =…

How do I change the classification head of a model?

🤗Transformers

Aliseyfi December 21, 2021, 7:18pm 20

I guess you’re right. I’m still looking for a solution HERE

Topic		Replies	Views
Re-Training with new number of classes 🤗Transformers	2	1142	January 3, 2022
How to manually change the head of a fine-tuned model that doesn't work with AutoModelFor*? 🤗Transformers	0	359	January 9, 2022
Replacing last layer of a fine-tuned model to use different set of labels Beginners	6	6732	December 23, 2021
Does changing the classification head of models always require fine-tuning? Beginners	1	682	May 5, 2023
How do i take only "BERT" weights from BertForSequenceClassification model? 🤗Transformers	0	1494	February 16, 2022

How do I change the classification head of a model?

Related topics