Stationary probabilities.(Markov chain).

[MathematicalPhysicist]

We are given two states 1,2 in an irreducible and positive recurrent Markov chain, and their stationary probabilities [tex]\pi_1[/tex] and [tex]\pi_2[/tex] respectively, try to characterise in general the probability (distribution) of the number of visits in state 2 after two consecutive visits in state 1.

Any hints?

 

[Enuma_Elish]

Write out the ways this can happen, then turn it into a formula:

1,1,2
2,1,1,2
2,2,1,1,2
1,2,1,1,2
...

 

[MathematicalPhysicist]

Yes I thought in this direction but not sure how to get to the formula.
I mean for 1,1,2 the probability is: [tex]P_{1,1}\frac{\pi_1}{\pi_1+\pi_2}P_{1,2}[/tex]
2,1,1,2 [tex]\frac{\pi_2}{\pi_1+\pi_2}P_{1,1}P_{2,1}P_{1,2}[/tex]
1,2,1,1,2 [tex]\frac{\pi_1}{\pi_1+\pi_2}P^2_{1,2}P_{1,1}P_{2,1}[/tex]

So my hunch is that if we first get to 1 or 2, we should multiply by pi1/(pi1+pi2) or pi2/(pi1+pi2), and we should always multiply by P1,1, but other than this I don't see a general equation for all cases.

 

[Enuma_Elish]

I don't have an immediate answer, but you might find the approach in this paper to be helpful:

http://smu.edu/statistics/TechReports/TR211.pdf

The authors derive the unconditional distribution of the number of successes (e.g., state 2) in n+1 trials. It seems to me that you need to derive a similar distribution, conditional on having obtained (exactly? or at least?) two consecutive failures (1,1).