- if $y \in S$ is a transient state then, for all $x \in S$,
$$
\operatorname{Pr}{x}(N(y)<\infty)=1 \text { and } E\left[N(y) \mid \theta^{(0)}=x\right]=\frac{\rho{x y}}{1-\rho_{y y}}<\infty .
$$ - if $y \in S$ is a recurrent state then
$$
\operatorname{Pr}_{y}(N(y)=\infty)=1 \text { and } E\left[N(y) \mid \theta^{(0)}=y\right]=\infty
$$
So, recurrent states are infinitely often (i.o.) visited with probability one. The expected number of visits is finite if the state is transient.
It is interesting to investigate possible decompositions of $S$ in subsets of recurrent and transient states. From this decomposition, probabilities of the chain hitting a given set of states can be evaluated. For states $x$ and $y$ in $S, x \neq y, x$ is said to hit $y$, denoted $x \rightarrow y$, if $\rho_{x y}>0$. A set $C \subseteq S$ is said to be closed if
$$
\rho_{x y}=0 \text { for } x \in C \text { and } y \notin C .
$$
STAT0017 COURSE NOTES ：
The conditional transition probability over $m$ steps is given by
$$
P^{m}(x, y)=\operatorname{Pr}\left(\theta^{(m+n)} \leq y \mid \theta^{(n)}=x\right), \text { for } x, y \in S,
$$
the transition kernel over $m$ steps is given by
$$
p^{m}(x, y)=\frac{\partial P^{m}(x, y)}{\partial y}, \text { for } x, y \in S
$$
and the equivalent equation to (4.2) has the form
$$
P^{n+m}(x, y)=\int_{-\infty}^{\infty} P^{m}(z, y) p^{n}(x, z) d z, \quad m, n \geq 0 .
$$
This is the continuous version of the Chapman-Kolmogorov equations. For $m=1$, it reduces to
$$
P^{n+1}(x, y)=\int_{-\infty}^{\infty} P(z, y) p^{n}(x, z) d z, \quad n \geq 0
$$