Hi all, I have another Markov chain problem that I'm working off of as review and want to make sure I'm understanding the concepts properly and any correct mistakes. Would anyone be willing to help? Thank you in advance.
Problem with my answers:
Reasoning:
(a) [imath] P_{00} = P_{02} = P_{20} = 0 [/imath] are impossible events, since a swap must happen, and starting from state 0 black pens in the right pocket can only to state 1. Similarly, starting with 2 black pens in the right pocket cannot go to 0 (only 1 or stay at 2).
(b) [imath] P_{01} = 1 [/imath], per reasoning in part (a)
[imath] P_{10} = P(\text{taking black from right})P(\text{taking white from left}) = (1/2)(1/3) = 1/6 [/imath]
[imath] P_{11} = P(\text{taking white from right})P(\text{taking white from left}) + P(\text{taking black from right})P(\text{taking blackfrom left}) = (1/2)(1/3) + (1/2)(2/3) = 3/6 [/imath]
[imath] P_{12} = P(\text{taking white from right})P(\text{taking black from left}) = (1/2)(2/3) = 2/6 [/imath]
[imath] P_{21} = P(\text{taking black from right})P(\text{taking white from left}) = (1)(2/3) = 2/3 = 4/6 [/imath]
[imath] P_{22} = P(\text{taking black from right})P(\text{taking black from left}) = (1)(1/3) = 1/3 = 2/6 [/imath]
(c) Labelled above using the transition probabilities
(d) True. Since all states are are reachable from all others, the chain is one big communicating class [imath] C = {0, 1, 2} [/imath]
(e) True. If the chain is irreducible then every state is recurrent; there's always a way to get back to each state
(f) True. No state has a self-loop with [imath] P_{ii} = 1 [/imath]
(g) True. The presence of a self-loop ensures that the chain is aperiodic, i.e. the [imath] gcd(1, 2, 3, ...) = 1 [/imath]
(h) Solving the balance equation [imath] \pi = \pi P [/imath], where [imath] \pi = [\pi_0 \pi_1 \pi_2 ] [/imath] and [imath] P [/imath], is the transition matrix, I get a system of equations and adding the fact that [imath] \pi_0 + \pi_1 + \pi_2 = 1 [/imath] I get [imath] \pi_2 [/imath] = 0.3
Problem with my answers:
Reasoning:
(a) [imath] P_{00} = P_{02} = P_{20} = 0 [/imath] are impossible events, since a swap must happen, and starting from state 0 black pens in the right pocket can only to state 1. Similarly, starting with 2 black pens in the right pocket cannot go to 0 (only 1 or stay at 2).
(b) [imath] P_{01} = 1 [/imath], per reasoning in part (a)
[imath] P_{10} = P(\text{taking black from right})P(\text{taking white from left}) = (1/2)(1/3) = 1/6 [/imath]
[imath] P_{11} = P(\text{taking white from right})P(\text{taking white from left}) + P(\text{taking black from right})P(\text{taking blackfrom left}) = (1/2)(1/3) + (1/2)(2/3) = 3/6 [/imath]
[imath] P_{12} = P(\text{taking white from right})P(\text{taking black from left}) = (1/2)(2/3) = 2/6 [/imath]
[imath] P_{21} = P(\text{taking black from right})P(\text{taking white from left}) = (1)(2/3) = 2/3 = 4/6 [/imath]
[imath] P_{22} = P(\text{taking black from right})P(\text{taking black from left}) = (1)(1/3) = 1/3 = 2/6 [/imath]
(c) Labelled above using the transition probabilities
(d) True. Since all states are are reachable from all others, the chain is one big communicating class [imath] C = {0, 1, 2} [/imath]
(e) True. If the chain is irreducible then every state is recurrent; there's always a way to get back to each state
(f) True. No state has a self-loop with [imath] P_{ii} = 1 [/imath]
(g) True. The presence of a self-loop ensures that the chain is aperiodic, i.e. the [imath] gcd(1, 2, 3, ...) = 1 [/imath]
(h) Solving the balance equation [imath] \pi = \pi P [/imath], where [imath] \pi = [\pi_0 \pi_1 \pi_2 ] [/imath] and [imath] P [/imath], is the transition matrix, I get a system of equations and adding the fact that [imath] \pi_0 + \pi_1 + \pi_2 = 1 [/imath] I get [imath] \pi_2 [/imath] = 0.3