# Deploying Sentence Transformer as sagemaker endpoint

**URL:** <https://discuss.huggingface.co/t/deploying-sentence-transformer-as-sagemaker-endpoint/24455>\
**Category:** Amazon SageMaker\
**Created:** [October 14, 2022, 5:53pm UTC](https://discuss.huggingface.co/t/deploying-sentence-transformer-as-sagemaker-endpoint/24455 "2022-10-14T17:53:42Z")\
**Posts on this page:** 1\
**Showing post:** 10

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [May 16, 2023, 4:15pm UTC](https://discuss.huggingface.co/t/deploying-sentence-transformer-as-sagemaker-endpoint/24455/10 "2023-05-16T16:15:35Z")

</div>

> [@Is Transformers using GPU by default?](https://discuss.huggingface.co/t/is-transformers-using-gpu-by-default/8500):
>
> I’m instantiating a model with this tokenizer = AutoTokenizer.from\_pretrained("nlptown/bert-base-multilingual-uncased-sentiment") model = AutoModelForSequenceClassification.from\_pretrained("nlptown/bert-base-multilingual-uncased-sentiment") Then running a for loop to get prediction over 10k sentences on a G4 instance (T4 GPU). GPU usage (averaged by minute) is a flat 0.0%. What is wrong? How to use GPU with Transformers?

---

_[View the full topic](https://discuss.huggingface.co/t/deploying-sentence-transformer-as-sagemaker-endpoint/24455)._