# Training Arguments to do pure bf16 training?

**URL:** <https://discuss.huggingface.co/t/training-arguments-to-do-pure-bf16-training/66483>\
**Category:** 🤗Transformers\
**Created:** [December 20, 2023, 3:42pm UTC](https://discuss.huggingface.co/t/training-arguments-to-do-pure-bf16-training/66483 "2023-12-20T15:42:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![chiragjn](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/chiragjn/32/22144_2.png) [@chiragjn](https://discuss.huggingface.co/u/chiragjn)\
**Post date:** [December 20, 2023, 3:42pm UTC](https://discuss.huggingface.co/t/training-arguments-to-do-pure-bf16-training/66483/1 "2023-12-20T15:42:53Z")

</div>

I am trying to finetune some large language models and I was wondering if there is a configuration to do pure bf16 training i.e. parameters, optimizer, gradients all in bf16?

I already know about `--bf16 True` which uses torch AMP, but I don’t want to use mixed precision at all if possible.  
I know there are certain risks involved with stability but getting rid of mixed precision will help reduce memory footprint and everything in bf16 will keep things fast.

I am currently exploring how Lightning does this with their `bf16-true` mode to replicate it with trainer.