Hello everyone! I am currently training a GPT-2 model from scratch using my own cif-tokenizer . The goal is to be able to generate crystallographic information files using an LLM. Since some of the CIFs have more tokens than context length, I am using strided tokenization with returned overflowing t…

I found a similar case, but it’s different… [Charlie Parker] multiprocess.pool.RemoteTraceback and TypeError: Couldn't cast array of type string to null when loading Hugging Face dataset python, python-multiprocessing, huggingface-datasets asked by Charlie P…

Hi, thanks for the help. I did see these before posting, but in my case all the entries strings so I don’t have these different datatypes. the only places I would have different datatypes would be in the return overflowing token mappings which I removed, at least to my understanding. So I’m not sure…

To begin with, Python’s language specification includes None but not null, so I think that null probably comes from pandas or pyarrow. In pandas, for example, just having “” creates null. I think that such data is causing problems when implicit conversion is performed inside the datasets library. O…

TypeError: Couldn't cast array of type int64 to null

🤗Datasets

John6666 February 5, 2025, 11:53am 2

I found a similar case, but it’s different…

Topic		Replies	Views
TypeError: Couldn't cast array of type int64 while mapping the dataset 🤗Datasets	6	5840	March 22, 2023
TypeError: Couldn't cast to null 🤗Datasets	1	125	November 6, 2025
Strange Error While Attempting to Load DataSet 🤗Datasets	7	4018	March 28, 2025
Dataset map() raises value error when mapping list to dict-like class 🤗Datasets	6	166	August 15, 2024
multiprocess.pool.RemoteTraceback and TypeError: Couldn't cast array of type string to null when loading Hugging Face dataset Beginners	1	192	September 24, 2024

TypeError: Couldn't cast array of type int64 to null

Related topics