矩阵微积分

分子记法：结果的形状与分子维度相同，与分母相反；分母记法反之。下面的例子都是分子记法。

  1. 标量求导
    1. 标量对标量求导: 1×11×1→1×1\frac{1 \times 1}{1 \times 1} \rightarrow {1 \times 1}
      • dydx\frac{dy}{dx}
  2. 向量求导
    1. 向量对标量求导: m×11×1→m×1\frac{m \times 1}{1 \times 1} \rightarrow {m \times 1}
      • ∂[y1y2…ym]T∂x=[∂y1∂x∂y2∂x…∂ym∂x]T\frac{\partial\begin{bmatrix} y_1 & y_2 & \ldots & y_m\end{bmatrix}^T}{\partial x}=\begin{bmatrix}\frac{\partial y_1}{\partial x} & \frac{\partial y_2}{\partial x} & \ldots & \frac{\partial y_m}{\partial x}\end{bmatrix}^T
    2. 标量对向量求导（梯度）: 1×1n×1→1×n\frac{1 \times 1}{n \times 1} \rightarrow {1 \times n}
      • ∇uf=(∂f∂x)⊤u{\displaystyle \nabla _{\mathbf {u} }f=\left({\frac {\partial f}{\partial \mathbf {x} }}\right)^{\top }\mathbf {u} }
      • ∂y∂[x1x2…xn]T=[∂y∂x1∂y∂x2…∂y∂xn]\frac{\partial y}{\partial \begin{bmatrix} x_1 & x_2 & \ldots & x_n\end{bmatrix}^T} = \begin{bmatrix}\frac{\partial y}{\partial x_1} & \frac{\partial y}{\partial x_2} & \ldots & \frac{\partial y}{\partial x_n}\end{bmatrix}
    3. 向量对向量求导: m×1n×1→m×n\frac{m \times 1}{n \times 1} \rightarrow {m \times n}
      • d f(v)=∂f∂vd v{\displaystyle d\,\mathbf {f} (\mathbf {v} )={\frac {\partial \mathbf {f} }{\partial \mathbf {v} }}d\,\mathbf {v} }
      • ∂[y1y2…ym]T∂[x1x2…xn]T=[∂y1∂x1∂y1∂x2⋯∂y1∂xn∂y2∂x1∂y2∂x2⋯∂y2∂xn⋮⋮⋱⋮∂ym∂x1∂ym∂x2⋯∂ym∂xn]\frac{\partial\begin{bmatrix} y_1 & y_2 & \ldots & y_m\end{bmatrix}^T}{\partial \begin{bmatrix} x_1 & x_2 & \ldots & x_n\end{bmatrix}^T}=\begin{bmatrix} \frac{\partial y_1}{\partial x_1} & \frac{\partial y_1}{\partial x_2} & \cdots & \frac{\partial y_1}{\partial x_n}\\ \frac{\partial y_2}{\partial x_1} & \frac{\partial y_2}{\partial x_2} & \cdots & \frac{\partial y_2}{\partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial y_m}{\partial x_1} & \frac{\partial y_m}{\partial x_2} & \cdots & \frac{\partial y_m}{\partial x_n} \end{bmatrix}
  3. 矩阵求导
    1. 矩阵对标量求导
      • ∂Y∂x=[∂y11∂x∂y12∂x⋯∂y1n∂x∂y21∂x∂y22∂x⋯∂y2n∂x⋮⋮⋱⋮∂ym1∂x∂ym2∂x⋯∂ymn∂x]{\displaystyle {\frac {\partial \mathbf {Y} }{\partial x}}={\begin{bmatrix}{\frac {\partial y_{11}}{\partial x}}&{\frac {\partial y_{12}}{\partial x}}&\cdots &{\frac {\partial y_{1n}}{\partial x}}\\{\frac {\partial y_{21}}{\partial x}}&{\frac {\partial y_{22}}{\partial x}}&\cdots &{\frac {\partial y_{2n}}{\partial x}}\\\vdots &\vdots &\ddots &\vdots \\{\frac {\partial y_{m1}}{\partial x}}&{\frac {\partial y_{m2}}{\partial x}}&\cdots &{\frac {\partial y_{mn}}{\partial x}}\\\end{bmatrix}}}
    2. 标量对矩阵求导
      • ∂y∂X=[∂y∂x11∂y∂x21⋯∂y∂xp1∂y∂x12∂y∂x22⋯∂y∂xp2⋮⋮⋱⋮∂y∂x1q∂y∂x2q⋯∂y∂xpq]{\displaystyle {\frac {\partial y}{\partial \mathbf {X} }}={\begin{bmatrix}{\frac {\partial y}{\partial x_{11}}}&{\frac {\partial y}{\partial x_{21}}}&\cdots &{\frac {\partial y}{\partial x_{p1}}}\\{\frac {\partial y}{\partial x_{12}}}&{\frac {\partial y}{\partial x_{22}}}&\cdots &{\frac {\partial y}{\partial x_{p2}}}\\\vdots &\vdots &\ddots &\vdots \\{\frac {\partial y}{\partial x_{1q}}}&{\frac {\partial y}{\partial x_{2q}}}&\cdots &{\frac {\partial y}{\partial x_{pq}}}\\\end{bmatrix}}}
      • 标量函数f(X)关于矩阵X在方向Y的方向导数可写成 ∇Yf=tr(∂f∂XY){\displaystyle \nabla _{\mathbf {Y} }f= {tr} \left({\frac {\partial f}{\partial \mathbf {X} }}\mathbf {Y} \right)}

常用结论

如下结论按照分母记法给出。

  • ∂xTA∂x=∂ATx∂x=A\frac{\partial \mathbf{x}^T \mathbf{A}}{\partial \mathbf{x}} = \frac{\partial \mathbf{A}^T\mathbf{x}}{\partial \mathbf{x}}=\mathbf{A}
  • ∂xTx∂x=x\frac{\partial\mathbf{x}^T\mathbf{x}}{\partial\mathbf{x}}=\mathbf{x}
  • 对于方阵B\mathbf{B}，有∂xTBx∂x=(B+BT)x\frac{\partial\mathbf{x}^T\mathbf{B}\mathbf{x}}{\partial\mathbf{x}}=\left(\mathbf{B}+\mathbf{B}^T\right)\mathbf{x}

results matching ""

    No results matching ""