Bellman Equation with two independent variables

Imagine I have a bellman equation with two independent variables:

V(a,b) = max{u(a,b)+βV(a',b')}

where the maximization is with respect to both a and b, β is the discount rate and a' is the future value of a.

What is the code to get matlab to solve this equation?

Answer 1

[Harsh Mahalwar]

Hi Laura,

From what I can gather, you are looking for an implementation of Bellman equation with 2 independent variables in MATLAB.

V(a, b) = max{u(a, b) + βV(a’, b’)};

Here’s a snippet of code that tries to implement this Bellman equation in MATLAB:

beta = 0.9; % Discount rate
A = linspace(0, 10, 50); 
B = linspace(0, 10, 50);  
% directions for future states (You can add or remove values from here)
dir = [1 0; -1 0; 0 1; 0 -1];

Here, I have created 2 example matrices (A and B) of size 1x50 each. For this example, I have added 4 directions to dir. This will help us to traverse the reward function V.

function V = bellmanEQwith2Vars(beta, A, B, dir)
    % Sample utility function
    u = @(x, y) x + y;
    V = zeros(length(A) + 2, length(B) + 2);

    % For this example we are going to do 10 iterations
    for iter = 1:10
        for i = 1:length(A)
            for j = 1:length(B)
                
                for k = 1:length(dir)
                V(i + 1, j + 1) = max(V(i + 1, j + 1), u(A(i), B(j)) ...
                    + beta * V(i + dir(k, 1) + 1, j + dir(k, 2) + 1));
                end
            
            end
        end
    end
end

Change the utility function(u) according to your needs.

Please use the following link to learn more about bellman equations and dynamic programming:

https://login.cs.utexas.edu/sites/default/files/legacy_files/research/documents/6%20Bellman%20Eqs%20and%20DP.pdf

I hope this help, thanks!