Recursive State Estimation¶

• Assume a state space model $p(x_t,z_t|z_{t-1})=p(x_t|z_t) p(z_t|z_{t-1})$.

• Find a recursive state update $p(z_t|x^t)$ from a prior estimate $p(z_{t-1}|x^{t-1})$ and a new observation $x_t$

$$\begin{align*} \underbrace{p(z_t|x^t)}_{\text{posterior}} &= \left(1/p(x^t)\right) p(z_t,x^t) \\ &= \left(1/p(x^t)\right) \int p(z_t,x_t,x^{t-1},z_{t-1}) \mathrm{d} z_{t-1} \\ &= \left(1/p(x^t)\right) \int p(z_t,x_t|x^{t-1},z_{t-1}) p(x^{t-1},z_{t-1}) \mathrm{d} z_{t-1} \\ &= \frac{p(x^{t-1})}{p(x^t)} \int p(z_t,x_t|x^{t-1},z_{t-1}) p(z_{t-1}|x^{t-1}) \mathrm{d} z_{t-1} \\ &= \underbrace{\frac{1}{p(x_t|x^{t-1})}}_{\text{normalization}} \underbrace{p(x_t|z_t)}_{\text{observation}}\int \underbrace{p(z_t|z_{t-1})}_{\text{transition}} \underbrace{p(z_{t-1}|x^{t-1})}_{\text{prior}} \mathrm{d} z_{t-1} \end{align*}$$

Gaussian Mixture Models¶

• Assume a model $p(x_n,\mathcal{C}_k) = \pi_k \mathcal{N}(x_n|\mu_k,\Sigma_k)$
• Let's rewrite this with a one-hot coding variable $z_k$:

$$\begin{align*} p(x_n|z_{nk}) &= \mathcal{N}(x_n|\mu_k,\Sigma_k) \\ p(z_{nk}) &= \pi_k \end{align*}$$

• This leads to

$$\begin{align*} p(x_n|z_{n}) &= \prod_k \mathcal{N}(x_n|\mu_k,\Sigma_k)^{z_{nk}} \\ p(z_{n}) &= \prod_k \pi_k^{z_{nk}} \end{align*}$$

• and generative model

$$\begin{align*} p(x,z) &= \prod_n p(z_{n}) p(x_n|z_{n}) \\ &= \prod_n \prod_k \left( \pi_k \mathcal{N}(x_n|\mu_k,\Sigma_k)\right)^{z_{nk}} \end{align*}$$