concatenating matrices

I have 1 large correlation file with 24000x24000 elements. In order to be able to import into matlab I created 3 sub-matrices using the following-

file = ('text.txt');
r = 24000;
c = 24000;
fid = fopen(file,'r');
a = r*c;
m = textscan(fid,'%f',a);
b = m{1}(4:6403);   %needed to exclude the first 3 elements
mat1 = reshape(b,80,80); %[similarly for mat2 and mat3].

Now I need to combine these matrices into one large matrix; how do I do that?

-thanks.

[Edited, code formatted, Jan S]

[per isakson]

How large is your file? r=24000 what's that?

[Walter Roberson]

rows and columns maybe?

Answer 1

[Walter Roberson]

It is not clear what the rule is for mat2 and so on. Are you excluding the first 3 elements of each group of 6403? If so then 24000*24000 is the wrong size to read to get a complete final matrix. Even if the rule is to skip the first 3 only and to take 80x80 after that, r*c is the wrong number of elements to read.

What is the rule for combining into one large matrix? 80 rows total, and 80+80+80+... columns? 3D, 80 x 80 x however-many?

reshape(m{1}(4:end), [80, 80, r*c-3])

would be one interpretation.

Answer 2

[veeus18]

Hi, yes, r=24000 rows and c=24000 columns(probably a little more than that). I am working on a 64-bit m/c with 8gb ram, but Matlab can't read the data when I import it as one large text file. I only need to exclude the very first 3 elements from my data file. My end matrix needs to be 24000x24000. Hence trying to see if I can break it down. But then I don't know how to put it back. I can upload the text file if that would help! Thanks for any suggestions.

[Oleg Komarov]

you're repeating yourself w/o giving additional information.

How did you break the 24000 by 24000 matrix?

What are the sizes of mat1, mat2, mat3?

Did YOU break the big matrix?

[veeus18]

I just picked an arbitrary number for mat1, mat2, mat3..I figured if I read in a 4000x4000 matrix first, I would need to do it 6 times(6 sub-matrices), if I read in 6400x6400, then read it in 4 sub-matrices..etc..maybe it is not the best way to do it..but that is why I am here..

Answer 3

[Titus Edelhofer]

Hi Maithili,

just a warning: the final matrix will need 24000*24000*8/(1024^3)GB, so approx 4.3 GB of memory. Should work but you better make sure you don't copy it around ...

Why don't you read your text.txt in chunks? Preallocate a large (24000x24000) matrix, and read in a loop 500 lines (about 90MByte), put it into your matrix and continue. textscan allows a max number of elements to be read.

Titus

Answer 4

[veeus18]

Thanks, but not sure I know how to preallocate and read in a loop...