# Configuring Group Size for BitsAndBytesConfig

**URL:** <https://discuss.huggingface.co/t/configuring-group-size-for-bitsandbytesconfig/93622>\
**Category:** 🤗Transformers\
**Created:** [June 23, 2024, 2:47pm UTC](https://discuss.huggingface.co/t/configuring-group-size-for-bitsandbytesconfig/93622 "2024-06-23T14:47:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![melhoushi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/melhoushi/32/28550_2.png) [@melhoushi](https://discuss.huggingface.co/u/melhoushi)\
**Post date:** [June 23, 2024, 2:47pm UTC](https://discuss.huggingface.co/t/configuring-group-size-for-bitsandbytesconfig/93622/1 "2024-06-23T14:47:29Z")

</div>

I am trying to find a way to configure the group size (a.k.a. block size) of NF4 quantization.  
According to my understanding, NF4 quantization can be done by `BitsAndBytesConfig`.

Looking at this documentation page:

> **[Quantization](https://huggingface.co/docs/transformers/en/main_classes/quantization)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Some configs like `AwqConfig` and `GPTQConfig` do have `group_size` arg but `BitsAndBytesConfig| doesn’t.

How can I know what is the group\_size of `BitsAndBytesConfig` and how can I modify it?