# Float16 on CPU torch support

**URL:** <https://discuss.huggingface.co/t/float16-on-cpu-torch-support/69427>\
**Category:** Beginners\
**Created:** [January 16, 2024, 9:04pm UTC](https://discuss.huggingface.co/t/float16-on-cpu-torch-support/69427 "2024-01-16T21:04:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rmiller3](https://avatars.discourse-cdn.com/v4/letter/r/c5a1d2/32.png) [@rmiller3](https://discuss.huggingface.co/u/rmiller3)\
**Post date:** [January 16, 2024, 9:04pm UTC](https://discuss.huggingface.co/t/float16-on-cpu-torch-support/69427/1 "2024-01-16T21:04:56Z")

</div>

I’ve read about torch not supporting float16 on CPU when trying to test microsoft/phi2 model.  
Example threads:

- [microsoft/phi-2 · RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'](https://huggingface.co/microsoft/phi-2/discussions/14)
- [Better error message when trying to run fp16 weights on CPU · Issue #96292 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/issues/96292)

How come the following code works on a GPT2 model where I’m seemingly setting it to run in that config? Is torch\_dtype somehow different than the topic in those comments?

```auto
        pipe = transformers.pipeline('text-generation', model=model, tokenizer=tokenizer, device="cpu", torch_dtype=torch.float16)

        res = pipe("Here is a recipe for vegan banana bread:\n",
                           max_new_tokens=max_new_tokens,
                           do_sample=False,
                           use_cache=True)

```