SQL query limits on matlab database toolbox

Hello, I have an instance where I suspect Matlab might be not allowing a large SQL Select query to be processed.

Are there any settings in setdbpref or elsewhere that constrains matlab to small queries?

Thanks, AMK

Answer 1

[the cyclist]

This page

http://www.mathworks.com/help/database/ug/preference-settings-for-large-data-import.html

discusses Java heap memory issues with importing large query results.

I personally have not had success in processing queries in batches via the method described there. For my large queries (~millions of rows), I have needed to wrap the query in a for loop, and use SQL's "offset" and "limit" clauses to do the batching within SQL.

[Adam]

Okay good, thanks for the link. My issue is one of my matlab instances processes this query with not glitches, another instance (which is installed on a virtual machine) won't process more than 105000 rows. I'll look closely at these prefs.

Thanks, AMK

[the cyclist]

In that case, on the machine where this does not work, check

Preferences -> General -> Java heap space

and make sure it is at 256MB.

You may need to restart MATLAB afterward.

[Adam]

I increased the Java Heap Memory on the virtual machine. Your link definitely explained the problem. Thanks for pointing that out.

[the cyclist]

It might be nice to "accept" this answer, which may help a future user seeking similar help.