# Using model.generate() in parrellel / faster?

**URL:** <https://discuss.huggingface.co/t/using-model-generate-in-parrellel-faster/58116>\
**Category:** Beginners\
**Created:** [October 11, 2023, 3:45am UTC](https://discuss.huggingface.co/t/using-model-generate-in-parrellel-faster/58116 "2023-10-11T03:45:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jared312](https://avatars.discourse-cdn.com/v4/letter/j/db5fbb/32.png) [@jared312](https://discuss.huggingface.co/u/jared312)\
**Post date:** [October 11, 2023, 3:45am UTC](https://discuss.huggingface.co/t/using-model-generate-in-parrellel-faster/58116/1 "2023-10-11T03:45:53Z")

</div>

Hey all,  
I have fine tuned a llama model and have loaded it into a model object with AutoPeftModelForCausalLM.

I now want to used this model for inference on a lot of queries. Currently I am looping (yes for looping) through each query and calling mymodel.generate, then tokenizer.decode to get each response.

Is there a smarter way I can do this in huggingface? Can you generate in parrellel or batches? I have 4 GPUs avaliable to me.

Thanks  
Jared