# Load Dataset Fail for Custom Json Format

**URL:** <https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350>\
**Category:** Beginners\
**Created:** [January 24, 2023, 8:43pm UTC](https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350 "2023-01-24T20:43:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nanami](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nanami/32/18907_2.png) [@nanami](https://discuss.huggingface.co/u/nanami)\
**Post date:** [January 24, 2023, 8:43pm UTC](https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350/1 "2023-01-24T20:43:15Z")

</div>

I have a json file that has the following format (used :  
[{ “A” : string, “B”: list of string, “C”: list of list of bool }, sample2, sample3, …]

When I used load\_dataset(“json”, data\_files={“train”:data\_path + “Data/train.json”), I got the following error:

> datasets.builder.DatasetGenerationError: An error occurred while generating the dataset  
> pyarrow.lib.ArrowTypeError: Expected bytes, got a ‘list’ object  
> pyarrow.lib.ArrowInvalid: JSON parse error: Column() changed from object to array in row 0

What’s wrong with my procedure? The only thing I can imagine is that load\_dataset() doesn’t support list of list. But I didn’t find this in the documentation, and the error message is also not explicit enough to see if that’s the reason.

Thanks in advance for any input 🙂

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [February 6, 2023, 2:11pm UTC](https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350/2 "2023-02-06T14:11:53Z")

</div>

Hi! What does "used : " mean? Can you please specify the structure inside a code block?

JSON arrays/lists are supported, but this is (still) not documented.

---

<div class="post-metadata">

**Author:** ![nanami](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nanami/32/18907_2.png) [@nanami](https://discuss.huggingface.co/u/nanami)\
**Post date:** [February 19, 2023, 2:57pm UTC](https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350/3 "2023-02-19T14:57:32Z")

</div>

Thanks for your response!  
The “used” was a typo, the structure is

```auto
[{ “A” : string, “B”: list of string, “C”: list of list of bool }, sample2, sample3, …]

```

So essentially the json is a list of dictionaries. For each dictionary the keys are strings and there are three key-value pairs; a string, a list of strings, and a nested list of bool.

---

<div class="post-metadata">

**Author:** ![merlinyx](https://avatars.discourse-cdn.com/v4/letter/m/c89c15/32.png) [@merlinyx](https://discuss.huggingface.co/u/merlinyx)\
**Post date:** [June 20, 2023, 7:58pm UTC](https://discuss.huggingface.co/t/load-dataset-fail-for-custom-json-format/30350/4 "2023-06-20T19:58:33Z")

</div>

Hi, this is rather late but I also ran into this issue and realized that the JSON format should be

```auto
{ “A” : string, “B”: list of string, “C”: list of list of bool }
{ “A” : string, “B”: list of string, “C”: list of list of bool }
...
{ “A” : string, “B”: list of string, “C”: list of list of bool }

```

basically each dictionary’s JSON str is delimited with a newline character, instead of having all the dictionaries in a list when being dumped to a file.