Speed up this fread loop

I need your help! I am going to read a lot of big files with fread. Any ideas on how to speed this up? Also if there is a way to estimate N from the length of the file before allocating B1 to B8? Every advice is greatly appreciated

        fid = fopen(Filename,'r'); 
        B1=zeros(m, N,'single');
        B2=zeros(m, N,'single');
        B3=zeros(m, N,'single');
        B4=zeros(m, N,'single');
        B5=zeros(m, N,'single');
        B6=zeros(m, N,'single');
        B7=zeros(m, N,'single');
        B8=zeros(m, N,'single');

    fseek(fid,headerlength, 'bof');
    fseek(fid, offset, 0);      

    for k = 1 : N

        try
          tag(k)= fread(fid,1,'int64');
        catch
            disp('End of file')
            break
        end

        Idontneedthis = fread(fid,2,'uint64');
        Idontneedthis= fread(fid,1,'int'); 
        Idontneedthis = fread(fid,1,'int');

        try
            A = fread(fid,m*8,'short','ieee-be'); 
            A = reshape(A,m,8);
            A=single(A);
        catch
            disp('End of file')
            break
        end

        B1(:,k) = A(:,1); 
        B2(:,k) = A(:,2); 
        B3(:,k) = A(:,3); 
        B4(:,k) = A(:,4); 
        B5(:,k) = A(:,5); 
        B6(:,k) = A(:,6); 
        B7(:,k) = A(:,7); 
        B8(:,k) = A(:,8);

    end
    fclose(fid);

Thanks a lot!!!

Answer 1

[Carl]

Hi T,

1. If you are creating the files yourself, you can store the value of N at the beginning of the file.
2. You can replace the assignments to 'Idontneedthis' with a call to fseek.
3. You can also look into using the Parallel Computing Toolbox for doing the read in parallel. Here is an example of doing so with the 'parfor' loop: http://www.mathworks.com/matlabcentral/newsreader/view_thread/307594#835402

Answer 2

[Jan]

I simply added the bytes per block and divided the file size by this considering the "headerlength" and the "offset". This seems to be trivial maths.

m        = 16;  % Guessed
FileInfo = dir(Filename);
N        = (FileInfo.bytes - headerlength - offset) / (3 * 8 + 4 + 4 + m * 8 * 2);

fid = fopen(Filename, 'r', 'ieee-be'); 
if fid == -1  % I've seen too many code failing silently...
  error('Cannot open file: %s', Filename);
end

B   = zeros(m, 8, N, 'single');
tag = zeros(1, N, 'uint64');

fseek(fid, headerlength + offset, 'bof');
for k = 1:N
  tag(k)     = fread(fid, 1, 'int64');
  fread(fid, 3, 'uint64');   % Skip 24 bytes
  B(:, :, k) = fread(fid, [m, 8], 'int16=>single');
end
fclose(fid);

Do not create 8 different variables with an index in the name. Prefer a 3D array instead. Please test this - due to the missing of test data I cannot run this.