$$\frac{\partial \mathbf{b}^T \mathbf{A} \mathbf{b} }{\partial \mathbf{b}} = (\mathbf{A} + \mathbf{A}^T) \mathbf{b}$$
Proof:
Proof:
let $\mathbf{B}$ be a $m \times n$ matrix, and $\mathbf{A}$ be a $n \times m$ matrix
Then
Then
Proof for 2-D matrix:
\begin{align*} A &= \begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} \\ |A| &= ad - bc \\ A^{-1} &= \begin{bmatrix} d & -b \\ -c & a \\ \end{bmatrix} \\ \frac{\partial \ln|A|}{\partial A} &= \frac{1}{|A|} \begin{bmatrix} \frac{\partial |A|}{a} & \frac{\partial |A|}{b} \\ \frac{\partial |A|}{c} & \frac{\partial |A|}{d} \end{bmatrix} \\ &= \frac{1}{|A|} \begin{bmatrix} d & -c \\ -b & a \end{bmatrix} \\ &= (A^{-1})^T \end{align*}Proof:
First, calculate the element of a general three-matrix multiplication
At row $x$, column $y$, the element of $\mathbf{ABC}$ is $\sum_j^p \sum_i^n a_{xi} b_{ij} c_{jy}$, then
Now, to proove the cyclic property, the shape of the matrices need to satisfy:
Then, we have
Therefore, the three are equal.
Then,
Then,
So
$$\text{tr}(\mathbf{AB}) == \text{tr}(\mathbf{BA})$$Now, we extend it to the more general case, In the case of $\mathbf{tr}(\mathbf{A_1} \dots \mathbf{A}_n)$,
Let $\mathbf{A}_1 \dots A_{n-k} = \mathbf{X}$, and $\mathbf{A}_{n-k + 1} \dots \mathbf{A}_n = \mathbf{Y}$, then
where $k \in \{1, ... n\}$.