Parsing files and computing in parfor - GA

Hey, I've been working on some optimization problems within GA. There are some problems with files when i try use my "work remote" cluster (MJS Cluster).

I've got 2 main files:

'Fused_VP_40u.mat" - 1.26 GB - 1x1688 cell, where every cell contains couple thousands rows and always 50 columns,

''Fused_VP_40u_20t.mat" - 652 MB - 1x1688 cell, where every cell contains couple thousands rows and always 50 columns.

My task is to optimize these 50 columns to find the best combination (So I'm trying to lower the number of columns and also increase the accuracy - finding the best solution)

When I'm using it on my PC, with parfors inside it's working. Not so fast but working. Now i've got remote cluster with 32 workers which I want to use in my computations. I've got problems with parsing these files to workers and my script ends.

Here is the main code snippet:

load('Fused_VPs_40u.mat')
load('Fused_VPs_40u_20t.mat')
load('Features_chosen1.mat')
load('Features_chosen_new.mat','Feats_chosen')
%%
Feats_num = 50;
x_start = ones(1,50); 
ub = ones(1,Feats_num)
lb = zeros(1,Feats_num);
myFitnessFunction = @(XX) Fit_GMM(XX, VP_merged_u_fused, VP_merged_t_fused);
options = optimoptions("ga","PlotFcn","gaplotbestf",UseParallel=false,Display="iter",PopulationType="bitstring",InitialPopulationMatrix=x_start,PopulationSize=1,MaxStallGenerations=2000,MaxGenerations=5000) 
%,MutationFcn="mutationadaptfeasible",CrossoverFraction=0.7,MutationFcn="mutationadaptfeasible",SelectionFcn="selectionroulette",CreationFcn="gacreationlinearfeasible")
% Call GA with the wrapped fitness function
[Best_feats, ff,exitflag,output,population,scores] = ga(myFitnessFunction, Feats_num, [], [], [], [], lb, ub, [], [], options);

% Calculate the accuracy
Acc = 1 - ff;

function errorK = Fit_GMM(XX, VP_merged_t_fused, VP_merged_u_fused,Feats_chosen)
    classes = 1:1688;
    for g = 1:size(VP_merged_u_fused, 2)
        VP_test{g} = VP_merged_t_fused{g} .* XX;
        VP_ucz{g} = VP_merged_u_fused{g} .* XX;
        GMMs{g} = GMM_VP(VP_ucz{g});
    end
    clearvars VP_ucz
    xxx = size(GMMs, 2);
    nrcx = zeros(1, xxx);
    for z = 1 : 1688
        parfor i = 1 : size(GMMs, 2)
            [~, nllx(i)] = posterior(GMMs{i}, VP_test{z});
        end
        nllx = nllx / size(VP_test{z}, 1);
        [~, nrcx(z)] = min(nllx);
    end
    
    Acc = sum(nrcx == classes) / length(classes);
    errorK = 1 - Acc;
    XX_n = [XX,Acc];
    Feats_chosen = [Feats_chosen; XX_n];

    save Features_chosen_new Feats_chosen
end

Here is problem i've faced:

Warning: Could not find appropriate function on path loading function handle C:\Users\XXX\AppData\Local\Temp\Editor_rlrqf\LiveEditorEvaluationHelperE1805384047.m>makeF%1/F%0
> In parallel.internal.pool.optionallyDeserialize (line 7)
In parallel.internal.parfor.cppRemoteParallelFunction (line 25)
Error using GA_fusion>Fit_GMM
The source code (D:\XXXXX\XXXXX\GA_fusion.m) for the parfor-loop that is trying to execute on the worker could not be found.

Error in GA_fusion>@(XX)Fit_GMM(XX,VP_merged_u_fused,VP_merged_t_fused,Feats_chosen) (line 24)
myFitnessFunction = @(XX) Fit_GMM(XX, VP_merged_u_fused, VP_merged_t_fused,Feats_chosen);

Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)
fcn_handle = @(x) fcn(x,FcnArgs{:});

Error in makeState (line 58)
            firstMemberScore = FitnessFcn(state.Population(initScoreProvided+1,:));

Error in gaunc (line 47)
state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);

Error in ga (line 417)
            [x,fval,exitFlag,output,population,scores] = gaunc(FitnessFcn,nvars, ...

Error in GA_fusion (line 28)
[Best_feats, ff,exitflag,output,population,scores] = ga(myFitnessFunction, Feats_num, [], [], [], [],
lb, ub, [], [], options);

Caused by:
    Undefined function handle.
    Error using parallel.internal.parfor/cppRemoteParallelFunction
    Worker unable to find file.
        Undefined function handle.
    Failure in initial user-supplied fitness function evaluation. GA cannot continue.

I've tried using "matfile" instead of load - didn't work.

Tried using AddAttachedFiles - also didn't work (connection is taking very long and lost over some time).

Maybe I should load these variables in other way?

Answer 1

[Edric Ellis]

When working with a remote cluster parpool, usually Parallel Computing Toolbox is able to work out automatically which files need to be transferred to the cluster to make things work. However, this isn't always the case. A couple of things to try:

1. use addAttachedFiles to add the file containing Fit_GMM to your parallel pool
2. If that doesn't help - try running the entire script from the command-line. It looks like you might be using the editor to run sections of your script rather than the whole thing. This can sometimes interfere with the auto-attaching of files.

Answer 2

[Dominik Maly]

So today I solved this problem.

To make things work i've replaced:

load('Fused_VPs_40u.mat')

load('Fused_VPs_40u_20t.mat')

with:

var1 = matfile('Fused_VPs_40u.mat')

var2 = matfile('Fused_VPs_40u_20t.mat')

var_to_ga1 = var1.variable_name_we_want_to_process_from_matfile;

var_to_ga2 = var2.variable_name_we_want_to_process_from_matfile;

After this change everything works and runs smoothly.