can MATLA read very large CSVs (>150GB)?
조회 수: 1(최근 30일)
I need to read and analyze six very large CSV files. In each file, I'm interested in only two columns. I already used opt to try reducing the amount of data, but my PC is constantly collapsing (since it reached the max memory of 128GB and the allocated memory as well).
My question is, do you know another approach to handle such large CSV files?
Eventually, I want to split these files, but I need to read them first.
Is that right?
Cris LaPierre 2022년 11월 23일
This sounds like what is called Big Data to me. MATLAB does have functionality for working with this data, A good starting place may be the Big Data with MATLAB page. You may be particularly intersted in the functionality for Tall Arrays.
Here's a good example you may be able to follow to get started.