How the transition probability matrix can be calculated in 2nd order markov model?

Suppose I have 3-states system; state space = {1 2}. Also, I have state sequence

1 2 2 1 1 1 1 2 2 2 2 2 1 1 1 1 1 1 2 2 2 2 2 1 1 1 1 1 2 2 2 2 2 1 1 1 1 1 2 2 2 2 1 1 2 1 2 1 2 1 2 1 2 1 2 1 2 2 2 2 2 2 1 1 1 1 1 1 1 2 2 2 2 2 1 1 2 2 2 1 2 2 2 1 1 2 2 2.

I already calculated the transition matrix for first order markov model. Now, I want to calculate the transition matrix for 2nd order markov model but I don't know how to proceed.

If I am not wrong one way is to count different 3 combinations : 111 112 121 122 211 212 221 222. So that I will have 4x2 transition matrix for 2nd order Markov model. I am not sure how to write a code to count these three pairs and store them in the form of matrix.

Your help will be greatly appreciated.

Answer 1

[Poorna]

Hi Sushil Pokharel,

I understand that you want to compute the 2nd order transition matrix for the given transition sequence. You could follow the below steps to achieve that.

1. Create a 4x2 matrix called ‘tran_mat’ with zeros.
2. Iterate over the sequence starting from index 3. Let the indexing variable be called ‘i’ and the sequence be stored in a variable called ‘s’.
3. Now inside the for loop do as below:

tran_mat(s(i-2)*2 + s(i-1) – 2, s(i)) = tran_mat(s(i-2)*2 + s(i-1) – 2, s(i)) + 1; 

4. Essentially, we are selecting the row based on the previous two observations and selecting the column based on the current observation and then incrementing the count of that occurrence.

5. Finally normalize the rows of the matrix to get the probabilities.

Hope this helps!

[Sushil Pokharel]

Hey purna,

Thank you so much for your help. really appreciated.