# TimeSeriesTransformerModel dimensionality issue

**URL:** <https://discuss.huggingface.co/t/timeseriestransformermodel-dimensionality-issue/102802>\
**Category:** 🤗Transformers\
**Created:** [August 18, 2024, 12:39am UTC](https://discuss.huggingface.co/t/timeseriestransformermodel-dimensionality-issue/102802 "2024-08-18T00:39:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ArgyleR](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@ArgyleR](https://discuss.huggingface.co/u/ArgyleR)\
**Post date:** [August 18, 2024, 12:39am UTC](https://discuss.huggingface.co/t/timeseriestransformermodel-dimensionality-issue/102802/1 "2024-08-18T00:39:24Z")

</div>

Hi, fairly new to TimeSeriesTransformerModel here but I can’t figure out what’s going on with the model I’m trying to initialise:  
It seems like I have to force feature\_size to be 73 to match mat1 (understand the math reasoning but I can’t find what feature\_size is in the documentation)  
Error Code:

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/2/a/2a5ce2b16a66011bf0c8153e0185cf51324355e1.png)

However, if I force the feature size to be 73, the output looks empty…  
tensor(, size=(1, 0, 64), grad\_fn=)

Any help would be appreciated!

Edit:  
Does this happen because the prediction len is 0? I’ve noticed the encoder\_last\_hidden\_states is populated… This is all I actually need as I want to take the encoded representation and fit it to a separate fc head… Could someone confirm my thinking that the encoder\_last\_hidden\_states should be the embeddings for the time\_series data prior to any prediction?

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/64626_2.png) [@mahmutc](https://discuss.huggingface.co/u/mahmutc)\
**Post date:** [August 19, 2024, 10:13am UTC](https://discuss.huggingface.co/t/timeseriestransformermodel-dimensionality-issue/102802/2 "2024-08-19T10:13:58Z")

</div>

hi @ArgyleR  
If you haven’t yet, please see the example [Time Series Transformer](https://huggingface.co/docs/transformers/main/en/model_doc/time_series_transformer#transformers.TimeSeriesTransformerModel.forward.example).

I’m checking your code.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 19, 2024, 11:48am UTC](https://discuss.huggingface.co/t/timeseriestransformermodel-dimensionality-issue/102802/3 "2024-08-19T11:48:18Z")

</div>

cc @kashif