# Using S3 as model cache for Huggingface LLM inference DLC on Sagemaker

**URL:** <https://discuss.huggingface.co/t/using-s3-as-model-cache-for-huggingface-llm-inference-dlc-on-sagemaker/43270>\
**Category:** Amazon SageMaker\
**Created:** [June 14, 2023, 7:12pm UTC](https://discuss.huggingface.co/t/using-s3-as-model-cache-for-huggingface-llm-inference-dlc-on-sagemaker/43270 "2023-06-14T19:12:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nth-attempt](https://avatars.discourse-cdn.com/v4/letter/n/5f8ce5/32.png) [@nth-attempt](https://discuss.huggingface.co/u/nth-attempt)\
**Post date:** [June 14, 2023, 7:12pm UTC](https://discuss.huggingface.co/t/using-s3-as-model-cache-for-huggingface-llm-inference-dlc-on-sagemaker/43270/1 "2023-06-14T19:12:51Z")

</div>

Hi,

Is it possible to use the Huggingface LLM inference container for Sagemaker ([Introducing the Hugging Face LLM Inference Container for Amazon SageMaker](https://huggingface.co/blog/sagemaker-huggingface-llm)) in a way that I can specify path to a S3 bucket where I have the models downloaded ready for use instead of downloading the models from internet. Essentially using the S3 path as a HF\_HUB cache or using the S3 path to download the models on to the local container.

This is useful in the cases

- where we can’t connect to internet
- when we have fine-tuned models stored on S3

Thank you!

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [June 21, 2023, 3:36pm UTC](https://discuss.huggingface.co/t/using-s3-as-model-cache-for-huggingface-llm-inference-dlc-on-sagemaker/43270/2 "2023-06-21T15:36:57Z")

</div>

We release a blog post on how to do this: [Securely deploy LLMs inside VPCs with Hugging Face and Amazon SageMaker](https://www.philschmid.de/sagemaker-llm-vpc)