Does the MATLAB Interface to SQLite support concurrency?

SQLite as a database supports varying degrees of concurrency depending on version and settings, therefore I was expecting the MATLAB Interface to SQLite in the Database Toolbox would support some level of concurrency. And when a database access fails, it should at the very least show errors.

However, when I used the following sniplet,

conn=sqlite("sqlite_concurr_test.db","create");
conn.close();

ppool=parpool(4);
ff=parallel.Future.empty();
disp("Write 500 numbers")
for ii=1:500
    ff(ii)=parfeval(ppool,@writeOne,0,ii);
end
for ii=1:500
    ff(ii).wait()
end
delete(ppool);
conn=sqlite("sqlite_concurr_test.db");
readback=conn.sqlread("test");
disp("Readback "+num2str(size(readback,1))+" numbers");

function writeOne(ii)
conn=sqlite("sqlite_concurr_test.db");
conn.sqlwrite("test",array2table(ii));
conn.close();
end

I got the weird result of

Starting parallel pool (parpool) using the 'Processes' profile ...

Connected to the parallel pool (number of workers: 4).

Write 500 numbers

Parallel pool using the 'Processes' profile is shutting down.

Readback 74 numbers

This indicates that some database write did not occur, and there are no error reports.

What can I do to change this behavior? Is there anything I should do to assure parallel access, or at the very least get notifications if anything goes wrong?

Answer 1

[Sandeep]

Hi Xiaoxing Zhang,

Given the details, It is my understanding that you are looking for a solution to assure parallel access and expect an error message whenever database write fail to happen. It is recommended to use parfor in place of for-loop. parfor splits the execution of for-loop iterations over the workers in a parallel pool.

To know about the Status of intermediate writes, you can use the State property of parfeval function.

State property becomes 'finished' for completed Futures. You can distinguish between futures which are cancelled and complete normally by using the Error property.

fprintf("ff(%d): %s\n",ii, ff(ii).Error.message); % To display the completion status

Try pasting the above snippet in the loop to get the error status of the Future.