# GPTQ quantization on Custom dataset

**URL:** <https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145>\
**Category:** 🤗Transformers\
**Created:** [December 27, 2023, 9:17am UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145 "2023-12-27T09:17:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Saugatkafley](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saugatkafley/32/26738_2.png) [@Saugatkafley](https://discuss.huggingface.co/u/Saugatkafley)\
**Post date:** [December 27, 2023, 9:17am UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145/1 "2023-12-27T09:17:21Z")

</div>

Following the tutorial from `optimum`, it says a list of string can be passed as a dataset, but my question is how to pass dataset having multiple columns ?

```python
dataset = ["auto-gptq is an easy-to-use model quantization library with user-friendly apis, based on GPTQ algorithm."]
quantization = GPTQConfig(bits=4, dataset = dataset, tokenizer=tokenizer)

```

---

<div class="post-metadata">

**Author:** ![VikrantKadam](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@VikrantKadam](https://discuss.huggingface.co/u/VikrantKadam)\
**Post date:** [January 15, 2024, 8:39am UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145/2 "2024-01-15T08:39:11Z")

</div>

Any further progress on this?? I am facing same issue

---

<div class="post-metadata">

**Author:** ![Saugatkafley](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saugatkafley/32/26738_2.png) [@Saugatkafley](https://discuss.huggingface.co/u/Saugatkafley)\
**Post date:** [January 17, 2024, 9:33am UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145/3 "2024-01-17T09:33:57Z")

</div>

I haven’t looked much on this , @VikrantKadam will look onto it later.

---

<div class="post-metadata">

**Author:** ![Jamesshaw](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jamesshaw/32/22591_2.png) [@Jamesshaw](https://discuss.huggingface.co/u/Jamesshaw)\
**Post date:** [January 17, 2024, 9:54am UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145/4 "2024-01-17T09:54:48Z")

</div>

If you want to pass a dataset with multiple columns, you need to structure your dataset appropriately. A list of strings, as in your example, is suitable for a single-column dataset. However, for a dataset with multiple columns, each entry in the list can be a dictionary where the keys represent column names and the values are the corresponding data for each column.

Here’s an example:

pythonCopy code

```auto
dataset = [
    {"text": "auto-gptq is an easy-to-use model quantization library", "label": "positive"},
    {"text": "another example of a sentence", "label": "negative"},
    # Add more entries as needed
]

quantization = GPTQConfig(bits=4, dataset=dataset, tokenizer=tokenizer)

```

In this example, each entry in the `dataset` list is a dictionary with keys `"text"` and `"label"`. You can have additional columns if needed. Adjust the column names and values based on the structure of your dataset.

Make sure that the column names you use in the dataset match the column names expected by the `GPTQConfig` or any other part of your code that processes the dataset. If you are using a custom class or function for dataset processing, you might need to modify it accordingly to handle the dictionary structure.

---

<div class="post-metadata">

**Author:** ![DebasishDhal99](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/debasishdhal99/32/19893_2.png) [@DebasishDhal99](https://discuss.huggingface.co/u/DebasishDhal99)\
**Post date:** [January 24, 2025, 3:58pm UTC](https://discuss.huggingface.co/t/gptq-quantization-on-custom-dataset/67145/5 "2025-01-24T15:58:43Z")

</div>

Tried this, apparently formatting the custom data rows into dict format isn’t enough, the code is also getting stuck because it needs “input\_ids” and “attention\_mask” for every such row. Might be wrong.