How to make loops run faster

I have a 3d cashflow matrix, which have number of prices simulations as the first dimension, days left in a contract as the second dimension and opportunities left in the contract as the third dimension. I want to sum the cashflows per price simulation over the days left in the contract and the opportunities left, and then I want to discount the cashflows. I have written the code for it:

for j=days_left_in_contract-1:-1:2,
    counter=days_left_in_contract-j;

      if days_left_in_contract-j<days_left_with_opportunities,
          difference=days_left_in_contract-j;
      else
          difference=days_left_with_opportunities;
      end

      for w=1:difference,

          i=1:1:counter;
          interest_rate_vector=exp(-interest_rate*i);

          k=1;
          for i=1:simulations,
              if Eon_pris_t(i,j)>Gaspris_t(i,j), % Two different price simulations
                  Y(k,1)=sum(cashflow(i,j+1:j+counter,w).*interest_rate_vector);
                  k=k+1;
              end % close if
          end % close i

      end % close w

end % close j

However, the code runs very slowly when the number of prices simulations is over 1000. Do any of you guys know how to make the code run faster?

Thank you very much in advance.

[David Sanchez]

In your code you have:

counter=days_left_in_contract-j;

Y(k,1)=sum(cashflow(i,j+1:j+counter,w).*interest_rate_vector);

Did you realize that if

counter=days_left_in_contract-j;

then

j+counter == days_left_in_contract

and you could save an operation by writing

Y(k,1)=sum(cashflow(i,j+1:days_left_in_contract,w).*interest_rate_vector);

Answer 1

[Betina Isbak]

Thanks for your help Elige! I have tried to run the code improvements you suggested. However, I get the error:

Error using .* Matrix dimensions must agree.

I have tried different things, but I can’t seem to make it work. I can get the code to run if I change

for i = idx

to

for i = 1:idx

In the populate Y loop. But then I only get the result for the first simulation, and then the code stops.

This is only a part of the code. Y will be used in later calculations, but they are inside the w loop.

Again thank you very much for your help. I really appreciate it!

[Dr. Seis]

That error should not have anything to do with idx (even if idx is empty it should effectively skip over the for loop). idx should already be a list of numbers, so the error has more to do with the size of

cashflow(i,j+1:j+counter,w)

versus the size of

interest_rate_vector

(i.e., you can't .* a 1x5 matrix with a 1x6 matrix). If you can determine which loop/iteration you get the error, then I would suggest trying to investigate the size information for each of the above at that specific loop/iteration.

[Betina Isbak]

I got the code to work with a few modifications. The code is now

for j=days_left_in_contract-1:-1:2

      counter=days_left_in_contract-j;
      interest_rate_vector=exp(-interest_rate*(1:counter));

      difference=min(counter,days_left_with_opportunities);

      for w=1:difference

          % Get indices that contribute to Y
          idx = zeros(simulations,1);
          for i=1:simulations
              if Eon_pris_t(i,j)>Gaspris_t(i,j)
                  idx(i) = i;
              end % close if
          end % close i

          % Populate Y
          Y = zeros(numel(idx),1);
          k=1;
          for i = 1:idx(end),
              if idx>0
                  Y(k)=sum(cashflow(i,j+1:j+counter,w).*interest_rate_vector);
                  k=k+1;
              else
                  Y(k)=0;
                  k=k+1;
              end
          end
      end % close w
    end % close j

This code is a factor of 3 faster than my originale code, so I'm happy. Thanks for your help!

[Jan]

My answer is based on the above code.

Answer 2

[Hugo]

Try this

for j=days_left_in_contract-1:-1:2,
  counter=days_left_in_contract-j;
  difference=min(counter,days_left_with_opportunities);

    i=1:1:counter;
    interest_rate_vector=exp(-interest_rate*i);

    for w=1:difference,
   Y(:,1)=sum(cashflow(Eon_pris_t(:,j)>Gaspris_t(:,j),j+1:j+counter,w)*interest_rate_vector);
      end 
    end 
  end

The most important change is that I've moved the lines

i=1:1:counter;
interest_rate_vector=exp(-interest_rate*i);

outside the for loop. The values of this vectors are not initialize in each for loop actually, so they can be moved safely.

I have done other changes to make the code shorter and to save some for loops. matlab prefers things in matrix form.

By the way, the error that you mentioned before should occur in your original code as well. The error occurs in the line

Y(k,1)=sum(cashflow(i,j+1:j+counter,w).*interest_rate_vector);

and it occurs because sum(...) gives you a number while interest_rate_vector is a vector, and therefore cannot be multiplied using .*. In the code I wrote, this is replaced by *. Anyway, there is still the problem that you are trying to save a vector in Y(k,1), which is only one element, and that cannot be done. So you should check that the your code is right or clarify what Y really is (perhaps a cell array or so).

[Betina Isbak]

Hi Hugo, Thanks a lot for your help..

In my original code I don’t get an error in the line

Y(k,1)=sum(cashflow(i,j+1:j+counter,w)).*interest_rate_vector);

Because the cashflow matrix is multiplied with the interest_vector before the sum is taken. The cashflow matrix has the same length as the interest_vector and therefore I don’t get an error. It’s not that important to discount the cashflow back, so we can leave that one out. If I run the code you have suggested I get an error in

Y(:,1)=sum(cashflow(Eon_pris_t(:,j)>Gaspris_t(:,j),j+1:j+counter,w)*interest_rate_vector);

Saying

Error using * Inner matrix dimensions must agree

And if I take the interest rate out of the code I get the error

Subscripted assignment dimension mismatch.

Which is properly what you mean about me trying to save one cell in a vector. However, I’m not trying to save one element in Y(k,1), but a vector of sums over each price simulation. Maybe I should clarify what is done in the code. I have two different price simulations: Eon_pris_t and Gaspris_t. The cashflow matrix consist of max(Eon_pris_t(i,j)-Gaspris_t(i,j),0), and it is a 3d matrix which have number of prices simulations as the first dimension, days left in a contract as the second dimension and opportunities left in the contract as the third dimension. In the code I want to sum the cashflow matrix on each price simulation. This have to be done for every day left in the contract, the j for loop, and for every day with opportunities left in the contract, the w for loop. This is only a part of the code and might be why is doesn’t make sense that I overwrite Y in every loop. Hope this makes it more clear what I’m trying to do. I really appreciate your help.

[Hugo]

Ok, please correct me if I'm wrong, but in the first line of code you wrote, I see two parentesis opening and three closing, therefore that should be an error.

Nevertheless, as you said, in your original code the things are right. My mistake.

With respect to the first error, yes, now I see, please replace * with ".*". With the corrections, it should read like this:

for j=days_left_in_contract-1:-1:2,
  counter=days_left_in_contract-j;
  difference=min(counter,days_left_with_opportunities);

      i=1:1:counter;
      interest_rate_vector=exp(-interest_rate*i);

      for w=1:difference,
     Y(:,1)=sum(cashflow(Eon_pris_t(:,j)>Gaspris_t(:,j),j+1:j+counter,w).*interest_rate_vector);
        end 
      end 
    end

About moving

i=1:1:counter; interest_rate_vector=exp(-interest_rate*i);

above the

for w=1:...

I don't know what the problem could be, since it does not depend on w in any way. Can it be that the error comes from another thing? Try to move it before any other modification.

Answer 3

[David Sanchez]

Your problems could lay on the Eon_pris_t(i,j) Gaspris_t(i,j) functions: The calling itself takes its time, although it should not be what slows your code. What do these functions exactly do?

Answer 4

[Dr. Seis]

Could have something to do with pre-allocation of Y

There are more elegant ways to do this (I am sure), but if the below modifications do not cause a noticeable decrease in your current runtime then we will need to look at other things:

 for j=days_left_in_contract-1:-1:2
    counter=days_left_in_contract-j;
      if days_left_in_contract-j<days_left_with_opportunities
          difference=days_left_in_contract-j;
      else
          difference=days_left_with_opportunities;
      end
      for w=1:difference

          interest_rate_vector=exp(-interest_rate*(1:counter));

          % Get indices that contribute to Y
          idx = zeros(simulations,1);
          for i=1:simulations
              if Eon_pris_t(i,j)>Gaspris_t(i,j)
                 idx(i) = i;
              end % close if
          end % close i

          % Populate Y
          idx=idx(idx>0)); 
          Y = zeros(numel(idx),1);
          k=1;
          for i = idx
             Y(k)=sum(cashflow(i,j+1:j+counter,w).*interest_rate_vector);
             k=k+1;
          end
      end % close w
end % close j

Are you planning on keeping the results in Y for later calculations? Right now Y is overwritten after each step in w

Answer 5

[Jan]

Y = zeros(numel(idx), 1);
for j = days_left_in_contract-1:-1:2
    counter = days_left_in_contract - j;
    interest_rate_vector = exp(-interest_rate:-1:-interest_rate * counter));
    interest_rate_vector = interest_rate_vector.';  % Column vector

      difference = min(counter,days_left_with_opportunities);
      for w = 1:difference
         % Get indices that contribute to Y
         idx = Eon_pris_t(:, j) > Gaspris_t(:, j);  % Logical index

         % Populate Y
         Y(:) = 0;  % Reset values
         for k = 1:numel(idx)
           if idx(k)
              Y(k) = cashflow(k, j+1:j+counter, w) * interest_rate_vector;
           end
         end

         ... now hyou need to use the values of Y here, because they are
         ... overwritten in the next iteration
      end % close w
    end % close j

The DOT product is usually faster than the sum of the elementwise multiplication.

[Betina Isbak]

Hi Jan, I cant get this code to work.. Can you maybe go through it and make sure there aren't any mistakes in it? I can't get the interest_rate_vector to work right, and if I replace it with the code where it works I get the error:

Error using * Inner matrix dimensions must agree.

Thanks a lot for your help!