19th Ave New York, NY 95822, USA

# 统计代写| Recap stat代写

## 统计代考

$11.5$ Recap
A Markov chain is a sequence of r.v.s $X_{0}, X_{1}, X_{2}, \ldots$ satisfying the Markov property, which states that given the present, the past and future are conditionally independent:
$$P\left(X_{n+1}=j \mid X_{n}=i, X_{n-1}=i_{n-1}, \ldots, X_{0}=i_{0}\right)=P\left(X_{n+1}=j \mid X_{n}=i\right)=q_{i j}$$
The transition matrix $Q=\left(q_{i j}\right)$ gives the probabilities of moving from any state to any other state in one step. The ith row of the transition matrix is the conditional PMF of $X_{n+1}$ given $X_{n}=i$. The $n$th power of the transition matrix gives the $n$ step transition probabilities. If we specify initial conditions $s_{i}=P\left(X_{0}=i\right)$ and let $\mathrm{s}=\left(s_{1}, \ldots, s_{M}\right)$, then the marginal PMF of $X_{n}$ is $\mathrm{s} Q^{n}$.
States of a Markov chain can be classified as recurrent or transient: recurrent if the chain will return to the state over and over, and transient if it will eventually leave forever. States can also be classified according to their periods; the period of state $i$ is the greatest common divisor of the numbers of steps it can take to return to $i$, state in a finite number of steps, and aperiodic if each state has period $1 .$

A stationary distribution for a finite Markov chain is a PMF s such that s $Q=\mathbf{s}$. Under various conditions, the stationary distribution of a finite Markov chain exists and is unique, and the PMF of $X_{n}$ converges to $\mathrm{s}$ as $n \rightarrow \infty$. If state $i$ has stationary probability $s$ is $r_{i}=1 / s_{i}$.
If a PMF s satisfies the reversibility condition $s_{i} q_{i j}=s_{j} q_{j i}$ for all $i$ and $j$, it guarantees that $s$ is a stationary distribution of the Markov chain with transition matrix $Q=\left(q_{i j}\right)$. Markov chains for which there exists s satisfying the reversibility condition are called reversible. We discussed three types of reversible chains:

1. If the transition matrix is symmetric, then the stationary distribution is uniform over all states.
2. If the chain is a random walk on an undirected network, then the stationary distribution is proportional to the degree sequence, i.e.,
$$s_{j}=\frac{d_{j}}{\sum_{i} d_{i}}$$
3. If the chain is a birth-death chain, then the stationary distribution satisfies
$$s_{j}=\frac{s_{1} q_{12} q_{23} \ldots q_{j-1, j}}{q_{j, j-1} q_{j-1, j-2} \ldots q_{21}}$$
for $j>1$, where $s_{1}$ is solved for at the end to make $s_{1}+\cdots+s_{M}=1$.
521 FIGURE $11.6$ Given a transition matrix $Q$ and a distribution $t$ over the states, we can generate a Markov chain $X_{0}, X_{1}, \ldots$ by choosing $X_{0}$ according to $t$ and then running the chain according to the transition probabilities. An important event is $X_{n}=i$, the event that the chain is visiting state $i$ at time $n .$ We can then find the PMF of $X$ in terms of $Q$ and $\mathbf{t}$, and (under conditions discussed in the chapter) the PMF will converge to the stationary distribution s. If instead we start the chain out according to s, then the chain will stay stationary forever. Figure $11.6$ compares two ways to run a Markov chain with transition matrix $Q$ : choosing the initial state according to an arbitrary distribution $t$ over the states, or choosing the initial state according to the stationary distribution s. In the former case, the exact PMF after $n$ steps can be found in terms of $Q$ and $t$, and the PMF case, the exact PMF after $n$ steps can be found in terms of $Q$ and $t$, and the PMIF the latter case, the chain is stationary forever.

## 统计代考

$11.5$ 回顾

$$P\left(X_{n+1}=j \mid X_{n}=i, X_{n-1}=i_{n-1}, \ldots, X_{0}=i_{0}\right) =P\left(X_{n+1}=j \mid X_{n}=i\right)=q_{ij}$$

1. 如果转移矩阵是对称的，则平稳分布在所有状态上是均匀的。
2. 如果链是无向网络上的随机游走，则平稳分布与度数序列成正比，即
$$s_{j}=\frac{d_{j}}{\sum_{i} d_{i}}$$
3. 如果链是生死链，则平稳分布满足
$$s_{j}=\frac{s_{1} q_{12} q_{23} \ldots q_{j-1, j}}{q_{j, j-1} q_{j-1, j-2} \ldots q_{21}}$$
对于 $j>1$，其中 $s_{1}$ 在最后得到解决，使得 $s_{1}+\cdots+s_{M}=1$。
521 图 $11.6$ 给定转移矩阵 $Q$ 和状态分布 $t$，我们可以通过选择 $X_{0}$ 来生成马尔可夫链 $X_{0}, X_{1}, \ldots$到$t$，然后根据转移概率运行链。一个重要事件是 $X_{n}=i$，即链在时间 $n 访问状态$i$的事件。然后我们可以根据$Q$和$\mathbf 找到 $X$ 的 PMF {t}$，并且（在本章讨论的条件下）PMF 将收敛到平稳分布 s。相反，如果我们根据 s 启动链，那么链将永远保持静止。图$11.6$比较了两种运行带有转移矩阵$Q$的马尔可夫链的方法：根据状态上的任意分布$t$选择初始状态，或者根据平稳分布 s 选择初始状态。在前一种情况下，$n$步后的确切 PMF 可以用$Q$和$t\$ 和 PMIF 后一种情况，链永远是静止的。