# Writing a matfile in a parloop

조회 수: 2(최근 30일)
Angelo Cuzzola 2020년 5월 4일
댓글: Angelo Cuzzola 2020년 5월 5일
I have a very simple code that works pretty well when the size of the involved objects is manageable by the memory.
%%% Variables
% Z -> matrix of dimensions (r,T)
% V -> matrix of dimensions (r,r,T)
% nanY -> matrix of dimensions (n,T)
% y -> matrix of dimensions (n,T)
nanY = isnan(y);
y(nanY) = 0;
denom = sparse(zeros(n*r,n*r));
nom = sparse(zeros(n,r));
parfor t=1:T
nanYt = sparse(diag(~nanY(:,t)))p;
denom = denom + kron(Z(:,t+1)*Z(:,t+1)'+Vsmooth(:,:,t+1),nanYt);
nom = nom + y(:,t)*Z(:,t+1)';
end
vec = denom\nom(:);
In practical application, that bunch of code has to run with the parameter n taking values around 80k leading quickly to an unfeasible memory load. The solution I figured out relies on matfiles, at the expenses of paralellizionation and general performance. This is the version with which I am able to handle it for large n.
nanY = isnan(y);
y(nanY) = 0;
denom = zeros(n*r,n*r,T);
nom = zeros(n,r,T);
save var_cyc.mat denom nom y nanY -v7.3;
v = matfile('var_cyc.mat', 'Writable', true);
for t=1:T
v.nanYt = sparse(diag(~v.nanY(:,t)))p;
v.denom(:,:,T) = kron(Z(:,t+1)*Z(:,t+1)' + V(:,:,t+1),v.nanYt);
v.nom(:,:,T) = v.y(:,t)*Z(:,t+1)';
end
vec = sum(v.denom,3)\sum(v.nom(:),3);
Unfortunately this script cannot be paralellizable because conflicts generate when workers try to write on the matfile. I'm wondering if there is a way to recover the initial parallelization structure using matlab file to handle huge file without incurring in 'out of memory errors'.
Thanks.

댓글을 달려면 로그인하십시오.

### 답변(1개)

Jason Ross 2020년 5월 4일
Have you looked into using tall arrays?
##### 댓글 수: 1표시숨기기 없음
Angelo Cuzzola 2020년 5월 5일
Yes, but nom and denom are not exactly tall. They are 60kx60k matrices

댓글을 달려면 로그인하십시오.

R2020a

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by