# Converting logits to string without .generate()

**URL:** <https://discuss.huggingface.co/t/converting-logits-to-string-without-generate/31634>\
**Category:** Beginners\
**Created:** [February 13, 2023, 4:16pm UTC](https://discuss.huggingface.co/t/converting-logits-to-string-without-generate/31634 "2023-02-13T16:16:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![g8i96](https://avatars.discourse-cdn.com/v4/letter/g/7993a0/32.png) [@g8i96](https://discuss.huggingface.co/u/g8i96)\
**Post date:** [February 13, 2023, 4:16pm UTC](https://discuss.huggingface.co/t/converting-logits-to-string-without-generate/31634/1 "2023-02-13T16:16:36Z")

</div>

Hello, I am currently experimenting with the CodeGen model, but I don’t know how to use the inputs\_embeds parameter with the .generate function.

Currently, my code is the following:

```auto
    tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-2B-mono")
    model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen-2B-mono").to(device)
    
    inputs = tokenizer(text, return_tensors="pt").to(device)

    inputs = {
        'inputs_embeds': extract_custom_embeddings(inputs['input_ids']),
        'attention_mask': inputs['attention_mask']
    }
    
    output = model.generate(**inputs)

```

However, if I run it, I get the following error:

```auto
ValueError: If inputs_embeds is passed as model-specific keyword input, then model has to be an encoder-decoder and not a CodeGenForCausalLM.

```

I want to do this because I am currently trying to improve on the existing embedding extraction method.  
Is there maybe somewhere some tutorial on how to do this?