# Pipeline Error: PeftModel... is not supported for text-classification

**URL:** <https://discuss.huggingface.co/t/pipeline-error-peftmodel-is-not-supported-for-text-classification/88841>\
**Category:** 🤗Transformers\
**Created:** [May 27, 2024, 8:35pm UTC](https://discuss.huggingface.co/t/pipeline-error-peftmodel-is-not-supported-for-text-classification/88841 "2024-05-27T20:35:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vinkaga](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vinkaga/32/27473_2.png) [@vinkaga](https://discuss.huggingface.co/u/vinkaga)\
**Post date:** [May 27, 2024, 8:35pm UTC](https://discuss.huggingface.co/t/pipeline-error-peftmodel-is-not-supported-for-text-classification/88841/1 "2024-05-27T20:35:05Z")

</div>

How do I create a text-classification pipeline with a PEFT/LoRA trained model? My (partial) code is

```auto
model_checkpoint = 'roberta-base'
model = AutoModelForSequenceClassification.from_pretrained(
    model_checkpoint, num_labels=2, id2label=id2label, label2id=label2id)
...
peft_config = LoraConfig(task_type="SEQ_CLS",
                         r=4,
                         lora_alpha=32,
                         lora_dropout=0.01,
                         target_modules=['query'],
                         )
model = get_peft_model(model, peft_config)
training_args = TrainingArguments(
    output_dir=model_checkpoint + "-lora-text-classification",
    learning_rate=lr,
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=batch_size,
    num_train_epochs=num_epochs,
    weight_decay=0.01,
    evaluation_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    report_to=None,
)
# creater trainer object
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["validation"],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)
trainer.train()

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)

```

But the last step generates this error

> The model ‘PeftModelForSequenceClassification’ is not supported for text-classification. Supported models are […].

What is the correct way to make pipeline with PEFT/LoRA trained model?

---

<div class="post-metadata">

**Author:** ![vinkaga](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vinkaga/32/27473_2.png) [@vinkaga](https://discuss.huggingface.co/u/vinkaga)\
**Post date:** [May 29, 2024, 4:29pm UTC](https://discuss.huggingface.co/t/pipeline-error-peftmodel-is-not-supported-for-text-classification/88841/2 "2024-05-29T16:29:41Z")

</div>

> The model ‘PeftModelForSequenceClassification’ is not supported for text-classification. Supported models are […].

Is actually a warning not error. Even with the warning, the inference is the same as if done manually like shown here [Text classification](https://huggingface.co/docs/transformers/en/tasks/sequence_classification#inference).