How to load parquet files where some columns are lists

I am trying to work with parquet files between python and MATLAB. Unfortutanely it seems matlab does not support reading parquet files if some of the columns are "unsupported types" such as lists/arrays/JSON (see https://ch.mathworks.com/help/releases/R2021a/matlab/import_export/datatype-mappings-matlab-parquet.html).

Is there maybe a way to force MATLAB to read in those columns as strings?

I thoughts about getting the parquet file header info with `parquetinfo` and force the VariableTypes of the columns with lists to be "string" but it seems the returned info file is read-only.

Thanks,

+Enea

[Maik Berchten]

We're faced a similar problem and ended up converting the array to a CSV string and storing that as UTF8 string while generating the parquet file. Thanks for reporting this (and we're looking forward to a solution that doesn't need a workaround)!

--maik

Answer 1

[Divyanshu]

Hi Enea,

The Parquet Files or Parquet Formatted data was not supported in the MATLAB version R2021a. However, in the later versions these file formats are supported as mentioned in the release notes of R2021b:

https://www.mathworks.com/help/releases/R2021b/matlab/import_export/supported-file-formats-for-import-and-export.html

Also there are two functions 'parquetread' and 'parquetwrite' through which you can work with parquet files from MATLAB. Please refer the following documentation links to read more about these functions:

• parquetwrite - https://www.mathworks.com/help/releases/R2021b/matlab/ref/parquetwrite.html
• parquetread - https://www.mathworks.com/help/releases/R2021b/matlab/ref/parquetread.html