Start with pre-allocating the output. Letting an array grow iteratively consumes a lot of resources, because Matlab has to create a new vector in each iteration.
Z = [1, sin(Y(m)); 2, cos(Y(m))];
The code can be simplified and no loop is required:
Y = linspace(1, 10, 1000);
X2 = cos(Y) - 2 * sin(Y);
It is not useful to optimize a code, which is a massive simplification of the real code. Calling DET() for a 2x2 matrix is a waste of time, because DET([a,b; c,d]) is simply: q*c - b*d. So either your problem can be solved in a cheap way without a loop, or you have hidden the important parts by posting an example which has been simplified too much.