# \[urgent\]Can you reconstruct datasets using the cache file (.arrow file)?

**URL:** <https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027>\
**Category:** 🤗Datasets\
**Created:** [January 2, 2021, 11:01pm UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027 "2021-01-02T23:01:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![zeyuyun1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zeyuyun1/32/524_2.png) [@zeyuyun1](https://discuss.huggingface.co/u/zeyuyun1)\
**Post date:** [January 2, 2021, 11:01pm UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/1 "2021-01-02T23:01:05Z")

</div>

Hi,

Basically I construct a dataset through the mapping function in jupyter notebook. I can find the cache file (.arrow). Is there a way I can reconstruct the dataset similar to “datasets.load\_from\_disk()”

 ![Screen Shot 2021-01-02 at 2.59.22 PM](https://us1.discourse-cdn.com/hellohellohello/original/2X/e/ec9b77dd6687907537c7b63392ae1b214ede4e3e.png)

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 3, 2021, 2:16pm UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/2 "2021-01-03T14:16:02Z")

</div>

Sure ! If you only have the .arrow file you can do

```python
from datasets import Dataset 

dataset = Dataset.from_file(path_to_arrow_file)

```

---

<div class="post-metadata">

**Author:** ![zeyuyun1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/zeyuyun1/32/524_2.png) [@zeyuyun1](https://discuss.huggingface.co/u/zeyuyun1)\
**Post date:** [January 4, 2021, 2:09am UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/3 "2021-01-04T02:09:22Z")

</div>

> [@lhoestq](#):
>
> `Dataset`

It works! thank you so much!

---

<div class="post-metadata">

**Author:** ![ezio98](https://avatars.discourse-cdn.com/v4/letter/e/b9e5f3/32.png) [@ezio98](https://discuss.huggingface.co/u/ezio98)\
**Post date:** [August 24, 2021, 12:58pm UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/4 "2021-08-24T12:58:08Z")

</div>

I think your answer can easily load if the arrow\_file is single. But when I process the data using multiple processes, the datasets are stored into several arrow files. Can I simply load each of them and concatenate them?

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [August 24, 2021, 1:09pm UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/5 "2021-08-24T13:09:38Z")

</div>

Yes you can load them and concatenate them 🙂  
That’s exactly what `map` does under the hood

---

<div class="post-metadata">

**Author:** ![ezio98](https://avatars.discourse-cdn.com/v4/letter/e/b9e5f3/32.png) [@ezio98](https://discuss.huggingface.co/u/ezio98)\
**Post date:** [August 27, 2021, 7:51am UTC](https://discuss.huggingface.co/t/urgent-can-you-reconstruct-datasets-using-the-cache-file-arrow-file/3027/6 "2021-08-27T07:51:46Z")

</div>

Thanks, I get it.