struct function in Matlab

Hi there,

I want to generate more than 10^4 times of a random 3 dimension matrix, is it a good and efficient way to store all those 10^4 3D matrices with 'struct' function? My understanding of 'struct' function is a group of variables with different types. Not sure if it is suitable to store numerous datas. And would it be easy and efficiency to access those datas? And how does it compare with a 4D matrix?

[Dyuman Joshi]

What is the size of a 3D matrix that you want to generate?

[DDDD]

Let's say, 16*16*16. complex variables.

[Stephen23]

Why not just use a cell array or a simple 4D array?

Assuming that you have sufficient memory, of course.

[Dyuman Joshi]

Memory POV -

For storing the data as a 4D complex double (2*8 bytes per element) array, with exactly 10^4 random 3D matrices, you would require

sprintf('%0.4f GB of memory of contiguous memory', 16*16*16*1e4*2*8/1e9)

ans = '0.6554 GB of memory of contiguous memory'

If you have, say 2*10^4 random 3D matrices, the memory required would double, i.e. ~1.3108 GB.

Double vs Struct (or Cell array)

Storing in a structure (or a cell array) does not require contiguous memory and the memory required to make a structure (or a cell array) depends on how it is constructed.

Read more here - https://in.mathworks.com/help/matlab/matlab_prog/memory-allocation.html

(Assuming each 3D matrix is stored as a field)

%Structure
S1.R = zeros(10,20,30);
S1.G = zeros(10,20,30);
S1.B = zeros(10,20,30);

%double array
S2 = zeros(10,20,30,3);
whos

  Name         Size                  Bytes  Class     Attributes

  S1           1x1                  144504  struct              
  S2          10x20x30x3            144000  double              
  ans          1x40                     80  char                
  cmdout       1x33                     66  char                

As you can see, that for the same amount of data, the structure requires more data than the double array, as for structures and cell arrays, MATLAB creates a header not only for the array, but also for each field of the structure or each cell of the cell array.

Accessing the data POV -

Accessing the data in a struct, again, depends how it is constructed, but in general accessing the data via indexing for the 4D array will certainly be easier.

In conclusion, using a 4D array will the best option.

[Bruno Luong]

Typo

16*16*16*1e4*2*8/1e8

divided by 1e9 to get Gb unit

[Dyuman Joshi]

Thank you for pointing out the mistake Bruno, I have edited my comment accordingly.

[DDDD]

Thank you all guys for your comments, really appreciate them!

Answer 1

[Bruno Luong]

I recommand ro Store them in 4D array

16 x 16 x 16 x 1e4

No reason to use struct or cell or any other structure. It would make the processing more cumbursome.

Answer 2

[Walter Roberson]

It looks to me as if the memory occupied by a struct is:

• for each field, 160 bytes plus 8 bytes times the number of struct elements
• for each extra field entry that has been written to, 96 bytes of overhead, plus the memory required for the data

So for example, foo(100).bar = []; requires 160 + 8 * 100 = 960 bytes. If you now assign to foo(99).baz = []; then that requires another 106 * 8 + 100 = 960 bytes for the second field. If you then assign to foo(42).bar = []; then that requires an additional 96 bytes plus the 0 bytes for the data.

This despite the fact that after you assign to foo(100).bar = [] that you create an implicit recall of foo(42).bar is [] because it is unassigned -- the explicit foo(42).bar = [] needs the 96 bytes overhead.

If you investigate cell arrays, you can deduce that 96-ish bytes (sometimes it seems like 100 or 108 bytes) is the number of bytes needed to store the dimensions and type and "iscomplex" and "isglobal" flag and other information about a variable -- the 96 bytes is the overhead for describing an anonymous variable.

And we notice that the 8 bytes (from "times the number of struct elements") is the exact same size as a pointer.

This suggests that each field of a struct is implemented as an array of pointers; if the pointer is empty then that offset of the field contains no explicit data and should read back as double precision []. If the pointer is not empty, it is the descriptor of an anonymous variable.

The size() of a struct does not appear to depend upon the length of the variable name for the field, suggesting that the field names are stored fixed-width. With 64 characters fixed width and 96 bytes of descriptor of some sort that would add up to 160 bytes, which does not sound like a coincidence.

... The point of all of this (besides being of interest for future reference) is that storing as a struct array can have significant overhead compared to just storing as a 4D array.

[Bruno Luong]

See https://www.mathworks.com/matlabcentral/answers/2011352-memory-required-for-struct-array