Error: File: Parallelanalysis.m Line: 60 Column: 1 This statement is not inside any function.

Line 60, T2 variable is giving an error and I don't think this should be insie any function. Does anyone know where this needs to be or if not does it need to be removed?

[Ahash Thayalan]

This si the full code of Parallelanalysis function;

function[EnsambleVectorPar, Total] = Parallelanalysis(RadLat, RadLon, RadO3)

%% 6: Pre-process the data for parallel processing

% This takes the 3D array of data [model, lat, lon] and generates the

% data required to be processed at each location.

% ## This process is defined by the customer ##

% If you want to know the details, please ask, but this is not required

% for the module or assessment.

[Data2Process, LatLon] = PrepareData(Hourlydata, Lat, Lon);

%% Parallel Analysis

%% 7: Create the parallel pool and attache files for use

PoolSize = 4 ; % define the number of processors to use in parallel

if isempty(gcp('nocreate'))

parpool('local',PoolSize);

end

poolobj = gcp;

% attaching a file allows it to be available at each processor without

% passing the file each time. This speeds up the process. For more

% information, ask your tutor.

addAttachedFiles(poolobj,{'EnsembleValue'});

% %% 8: Parallel processing is difficult to monitor progress so we define a

% % special function to create a wait bar which is updated after each

% % process completes an analysis. The update function is defined at the

% % end of this script. Each time a parallel process competes it runs the

% % function to update the waitbar.

DataQ = parallel.pool.DataQueue; % Create a variable in the parallel pool

%

% % Create a waitbar and handle top it:

hWaitBar = waitbar(0, sprintf('Time period %i, Please wait ...', idxTime));

% % Define the function to call when new data is received in the data queue

% % 'DataQ'. See end of script for the function definition.

afterEach(DataQ, @nUpdateWaitbar);

N = Num2Process/Steps; % the total number of data to process

p = 1; % offset so the waitbar shows some colour quickly.

%% 9: The actual parallel processing!

% Ensemble value is a function defined by the customer to calculate the

% ensemble value at each location. Understanding this function is not

% required for the module or the assessment, but it is the reason for

% this being a 'big data' project due to the processing time (not the

% pure volume of raw data alone).

T4 = toc;

parfor idx = 1: Num2Process % size(Data2Process,1)

[EnsembleVectorPar(idx, idxTime)] = EnsembleValue(Data2Process(idx,:,:,:), LatLon, RadLat, RadLon, RadO3);

if idx/Steps == ceil(idx/Steps)

send(DataQ, idx/Steps);

end

end

close(hWaitBar); % close the wait bar

T3(idxTime) = toc - T4; % record the parallel processing time for this hour of data

fprintf('Parallel processing time for hour %i : %.1f s\n', idxTime, T3(idxTime))

end % end time loop

T2 = toc;

delete(gcp);

%% 10: Reshape ensemble values to Lat, lon, hour format

EnsembleVectorPar = reshape(EnsembleVectorPar, 696, 396, []);

fprintf('Total processing time for %i workers = %.2f s\n', PoolSize, sum(T3));

Total = sum(T3);

end

[Steven Lord]

If you have the file open in the MATLAB Editor, select all the text then right-click and select "Smart Indent" from the context menu. Once you've done that, scroll to the bottom of the file. If you click on an end keyword MATLAB by default will highlight the keyword with which that end is associated.

If you click on the last end MATLAB will show it crossed out, indicating it doesn't match with anything. If you click on the end on line 57 (the one with the comment "end time loop") MATLAB will show that matches the function keyword on line 1. Therefore the call to toc on the next line is not part of the function and could never be executed. As such, MATLAB knows that there is an error somewhere and so throws an error.

So you likely have either one extra end that you don't need or you are missing the start of an if, for, while, etc. statement.

Answer 1

[Walter Roberson]

end % end time loop

What time loop? You are not inside any loop there. The parfor already ended.

[Ahash Thayalan]

I've created a time loop on thsi function file. This is on the following file attached 'loadDataparallel.m'

[Ahash Thayalan]

Should the end time loop statement be removed?

[Walter Roberson]

for idx = [1, 2, 4, 5, 6, 7, 8]
    HourlyData(DataLayer,:,:) = ncread(FileName, Contents.Variables(idx).Name,...
        [StartLon, StartLat, idxTime], [NumLon, NumLat, 1]);
    DataLayer = DataLayer + 1;
end

The loop in that code is complete; there is no need for another end statement to terminate it. And besides, the end time loop statement is in a different function completely; you cannot start a loop in one function and end it in a different function.

[Ahash Thayalan]

it is now showing a different error; Error: File: Parallelanalysis.m Line: 48 Column: 10

Unable to classify the variable 'EnsembleVectorPar' in the body of the parfor-loop. For more information, see Parallel for

Loops in MATLAB, "Solve Variable Classification Issues in parfor-Loops".

Error in ParallelProcessing (line 31)

[EnsambleVectorPar] = Parallelanalysis(RadLat, RadLon, RadO3);

I have attached file files already.

[Walter Roberson]

idxTime is used several times in that function but is not defined in that context.

When you define a variable in a function and then call a second function, then the variable does not get defined in the second function — except, that is, for the case where the second function is a nested function that is defined within the first function.

[Ahash Thayalan]

It is already been defined in the parallel processing file. What is needed to be added?

[Walter Roberson]

You have several choices:

1. You can pass idxTime to Parallelanalysis in your function calls;
2. You can construct Parallelanalysis as a nested function inside ParallelProcessing.m, at some point after the idxTime variable has been assigned to (such as at the end of the file), so that idxTime would become a shared variable. For this, the code currently in Parallelanalysis.m would have to be moved to be inside ParallelProcessing.m . I am not certain at the moment whether parfor will accept this possibility
3. You can use one of the techniques described at http://matlab.wikia.com/wiki/FAQ#How_can_I_share_data_between_callback_functions_in_my_GUI.28s.29.3F to share variables with the code. However, this will be tricky, as parfor starts the routine in a different process that does not have access to global variables... you might possibly not be able to get any of these techniques to work with parfor
4. You can use a pair of parallel.pool.DataQueue so as to send the current value of idxTime to the workers
5. Instead of using parfor you could use parfeval() and pass idxTime to the function. The function would have to expect to receive it as a parameter
6. Instead of using parfor, you could open the pool once and keep it open, and at the point where you would normally use parfor, you would use parfevalOnAll() to send the current value of idxTime to every worker, followed by using parfeval() to create the tasks

The only one of these that I recommend is the first, making idxTime a parameter to the function.