#!/usr/bin/env python
# coding: utf-8

# # 行向量对元素求导
# 
# 设$y^T=[y_1 ... y_n]$是n维行向量，$x$是元素
# 
# $\frac{\partial y^T}{\partial x}=[ \frac{\partial y_1}{\partial x} ... \frac{\partial y_n}{\partial x} ] $
# 
# 

# # 列向量对元素求导
# 
# 设$y=\begin{bmatrix}y_1\\ ...\\ y_n\end{bmatrix}$是m维列向量，$x$是元素
# 
# 则$\frac{\partial y}{\partial x}=\begin{bmatrix} \frac{\partial y_1}{\partial x}\\ ... \\
# \frac{\partial y_m}{\partial x} \end{bmatrix} $

# # 矩阵对元素求导
# 
# 设$Y=\begin{bmatrix}y_1 & ... & y_{1n}\\...&...& ...\\ y_{m1} & ... & y_{mn}\end{bmatrix}$是$m*n$维矩阵，$x$是元素
# 
# 则$\frac{\partial Y}{\partial x}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x} & ... &\frac{\partial y_{1n}}{\partial x}\\...&...& ... \\
# \frac{\partial y_{m1}}{\partial x}& ... & \frac{\partial y_{mn}}{\partial x} \end{bmatrix} $

# # 元素对行向量求导
# 
# 设$y$是元素，$x^T=[x_1 ... x_q]$是q维行向量
# 
# 则$\frac{\partial y}{\partial x^T}=[ \frac{\partial y}{\partial x_1} ... \frac{\partial y}{\partial x_q} ] $

# # 元素对列向量求导
# 
# 设$y$是元素，$x=\begin{bmatrix}x_1\\ ...\\ x_n\end{bmatrix}$是p维列向量
# 
# 则$\frac{\partial y}{\partial x}=\begin{bmatrix} \frac{\partial y}{\partial x_1}\\ ... \\
# \frac{\partial y}{\partial x_p} \end{bmatrix} $

# # 元素对矩阵求导
# 
# 设$y$是元素，$X=\begin{bmatrix}x_1 & ... & x_{1q}\\...&...& ...\\ x_{p1} & ... & x_{pq}\end{bmatrix}$是$p*q$维矩阵
# 
# 则$\frac{\partial y}{\partial X}=\begin{bmatrix} \frac{\partial y}{\partial x_{11}} & ... &\frac{\partial y}{\partial x_{1q}}\\...&...& ... \\
# \frac{\partial y}{\partial x_{p1}}& ... & \frac{\partial y}{\partial x_{pq}} \end{bmatrix} $

# # 行向量对列向量求导
# 
# 设$y^T=[y_1 ... y_n]$是n维行向量，$x=\begin{bmatrix}x_1\\ ...\\ x_p\end{bmatrix}$是p维列向量
# 
# 则$\frac{\partial y^T}{\partial x}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} & ... &\frac{\partial y_{n}}{\partial x_{1}}\\...&...& ... \\
# \frac{\partial y_1}{\partial x_{p}}& ... & \frac{\partial y_n}{\partial x_{p}} \end{bmatrix} $

# # 列向量对行向量求导
# 
# 设$y=\begin{bmatrix}y_1\\ ...\\ y_m\end{bmatrix}$是m维列向量，$x^T=[x_1 ... x_q]$是q维行向量
# 
# 则$\frac{\partial y}{\partial x^T}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} & ... &\frac{\partial y_{1}}{\partial x_{q}}\\...&...& ... \\
# \frac{\partial y_m}{\partial x_{1}}& ... & \frac{\partial y_m}{\partial x_{q}} \end{bmatrix} $

# # 行向量对行向量求导
# 
# 设$y^T=[y_1 ... y_n]$是n维行向量，$x^T=[x_1 ... x_q]$是q维行向量
# 
# 则$\frac{\partial y^T}{\partial x^T}=[ \frac{\partial y^T}{\partial x_1} ... \frac{\partial y^T}{\partial x_q} ] $

# # 列向量对列向量求导
# 
# 设$y=\begin{bmatrix}y_1\\ ...\\ y_m\end{bmatrix}$是m维列向量，$x=\begin{bmatrix}x_1\\ ...\\ x_p\end{bmatrix}$是p维列向量
# 
# 则$\frac{\partial y}{\partial x}=\begin{bmatrix} \frac{\partial y_1}{\partial x} \\...\\
# \frac{\partial y_m}{\partial x} \end{bmatrix} $

# # 矩阵对行向量求导
# 
# 设$Y=\begin{bmatrix}y_1 & ... & y_{1n}\\...&...& ...\\ y_{m1} & ... & y_{mn}\end{bmatrix}$是$m*n$维矩阵,$x^T=[x_1 ... x_q]$是q维行向量
# 
# 则$\frac{\partial Y}{\partial x^T}=[ \frac{\partial Y}{\partial x_1} ... \frac{\partial Y}{\partial x_q} ] $

# # 矩阵对列向量求导
# 
# 设$Y=\begin{bmatrix}y_1 & ... & y_{1n}\\...&...& ...\\ y_{m1} & ... & y_{mn}\end{bmatrix}$是$m*n$维矩阵,$x=\begin{bmatrix}x_1\\ ...\\ x_n\end{bmatrix}$是p维列向量
# 
# 则$\frac{\partial Y}{\partial x}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x} & ... &\frac{\partial y_{1n}}{\partial x}\\...&...& ... \\
# \frac{\partial y_{m1}}{\partial x}& ... & \frac{\partial y_{mn}}{\partial x} \end{bmatrix} $

# # 行向量对矩阵求导
# 
# 设$y^T=[y_1 ... y_n]$是n维行向量，$X=\begin{bmatrix}x_1 & ... & x_{1q}\\...&...& ...\\ x_{p1} & ... & x_{pq}\end{bmatrix}$是$p*q$维矩阵
# 
# 则$\frac{\partial y^T}{\partial X}=\begin{bmatrix} \frac{\partial y^T}{\partial x_{11}} & ... &\frac{\partial y^T}{\partial x_{1q}}\\...&...& ... \\
# \frac{\partial y^T}{\partial x_{p1}}& ... & \frac{\partial y^T}{\partial x_{pq}} \end{bmatrix} $

# # 列向量对矩阵求导
# 
# 设$y=\begin{bmatrix}y_1\\ ...\\ y_m\end{bmatrix}$是m维列向量，$X=\begin{bmatrix}x_1 & ... & x_{1q}\\...&...& ...\\ x_{p1} & ... & x_{pq}\end{bmatrix}$是$p*q$维矩阵
# 
# 则$\frac{\partial y}{\partial X}=\begin{bmatrix} \frac{\partial y_1}{\partial X}\\ ... \\
# \frac{\partial y_m}{\partial X} \end{bmatrix} $

# # 矩阵对矩阵求导
# 
# 设$Y=\begin{bmatrix}y_1 & ... & y_{1n}\\...&...& ...\\ y_{m1} & ... & y_{mn}\end{bmatrix}=\begin{bmatrix}y_1^T\\ ...\\ y_m^T\end{bmatrix}$是$m*n$维矩阵，$X=\begin{bmatrix}x_1 & ... & x_{1q}\\...&...& ...\\ x_{p1} & ... & x_{pq}\end{bmatrix}=[x_1 ... x_q]$是$p*q$维矩阵
# 
# 则$\frac{\partial Y}{\partial X}
# =[ \frac{\partial Y}{\partial x_1} ... \frac{\partial Y}{\partial x_q} ]
# =\begin{bmatrix}
# \frac{\partial y_1^T}{\partial X}\\
# ... \\
# \frac{\partial y_m^T}{\partial X} 
# \end{bmatrix} 
# =\begin{bmatrix}
# \frac{\partial y_{1}^T}{\partial x_{1}} & ... &\frac{\partial y_{1}^T}{\partial x_{q}}\\
# ...&...& ... \\
# \frac{\partial y_m^T}{\partial x_{1}}& ... & \frac{\partial y_m^T}{\partial x_{q}} \end{bmatrix} $