How should I get a plot of my data if the data matrix is too large ?

Hi All

my data have 160000 rows ( with 2 columns time vs displacement), that I obtain from some calculation on my data acquisition, the problem is that after the calculations, I get memory error, cause despite that MATLAB can read a 160 000 row matrix from my excel file, it can not save it to a new 160000 row matrix during the calculations(possibly in temporary memory) so I won't be able to plot my results.

what is the solution to this ? This is the first part of the code where I read my file, XP is the parameter in which I will have to write the results :

A = xlsread(filename);

dn=1;
la= A(1:dn:end,2:7);
t = A(1:dn:end,1);
XP = zeros(size(la,1),9);

and this is the second section :

where I define a new parameter after having written results on XP :

Theta=zeros(size(XP,1));

I get the error here , seems like the new parameter can't be created with this size

and this is how MATLAB reads my inputs from excel :

and the output :

[Jan]

Without seeing your code, it is impossible to guess, what's going on. It does not matter, if the physical meaning of the data is "time" or "displacement", but the type of the values matter: double or single? 160'000 does not sound large. I cannot imagine, why such a small data set exhausts your memory. Maybe a forgotten pre-allocation or something like n = 16e4; x = zeros(n), when x = zeros(1, n) is meant. How could the readers know?

Please post the relevant part of the code to give the readers a chance to understand what's going on and to suggest an improvement. Post the complete error message and high-light the failing line.

See Tutorial: How to ask a question to get a fast answer

[farzad]

dear Jan, thank you for your notes, I updated my question

[farzad]

This question is the most urgent

[Jan]

@farzad: If you think, that this question is "urgent" for you, take the time to the the suggested tutorial about formulating a good question. Please remember, that no question is "urgent" for the users in the forum, see Why your question is not urgent. Some members of the community react allergic to this term for good reasons.

See my answer: I still have to guess the error message and the intention of the failing command. Not explaining the actual problem wastes your time.

[farzad]

Thank you Jan, no I just meant most priority among those 9 question links I had put, I know what you are saying, sorry and thank you

Answer 1

[Jan]

As I had guessed:

Theta=zeros(size(XP,1));

This creates a square matrix:

n     = size(XP, 1);
Theta = zeros(n, n);

With n=163'840, you try to allocate a 215 GB array (163840^2*8, because you need 8 byte per double). If you do not have this memory available as contiguous free block, the allocation must fail.

Either you have to start working of a massive machine, or maybe it is a typo and you mean:

Theta = zeros(size(XP,1), 1);

By the way: Please care for posting a copy or the complete error message, if you mention an error in the forum. A rough rephrasing is less useful

If you know the failing line already, remember that the readers in the forum cannot guess, what the code is intended to do. Maybe it contains an error, but we can only check this, if you tell us, what you expect the code to do.