# Feature Extraction pipeline for a single token

**URL:** <https://discuss.huggingface.co/t/feature-extraction-pipeline-for-a-single-token/24514>\
**Category:** Beginners\
**Created:** [October 16, 2022, 11:27am UTC](https://discuss.huggingface.co/t/feature-extraction-pipeline-for-a-single-token/24514 "2022-10-16T11:27:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![matheusvolpon](https://avatars.discourse-cdn.com/v4/letter/m/e495f1/32.png) [@matheusvolpon](https://discuss.huggingface.co/u/matheusvolpon)\
**Post date:** [October 16, 2022, 11:27am UTC](https://discuss.huggingface.co/t/feature-extraction-pipeline-for-a-single-token/24514/1 "2022-10-16T11:27:14Z")

</div>

Hi all.  
I would like to extract the word embeddings for some input tokens (single words, not sentences) from DistilBertForMaskedLM models that I trained with the Hugging Face library. Does the Feature Extraction pipeline compute only contextualized embeddings? Or, when the pipeline receives a single token, it computes a decontextualized embedding? (using the mean of the word’s embeddings from all it contexts).