设A∈Rm×n,x∈Rn×1A\in\mathbb{R}^{m\times n},x\in \mathbb{R}^{n\times 1}

矩阵求导的本质是元素对元素一对一求导

  • 分子布局

∂[Ax]m×1∂[xT]1×n=[∂Axx1∂Axx2⋯∂Axxn]=[∂A1xx1∂A1xx2⋯∂A1xxn∂A2xx1∂A2xx2⋯∂A2xxn⋮⋮⋱⋮∂Amxx1∂Amxx2⋯∂Amxxn]m×n=[a11a12⋯a1na21a22⋯a2n⋮⋮⋱⋮am1am2⋯amn]m×n=A\begin{split} \frac{\partial [Ax]_{m\times 1}}{\partial [x^T]_{1\times n}} &= \begin{bmatrix} \frac{\partial Ax}{x_{1}} & \frac{\partial Ax}{x_{2}} & \cdots & \frac{\partial Ax}{x_{n}} \\ \end{bmatrix}\\ &= \begin{bmatrix} \frac{\partial A_1x}{x_{1}} & \frac{\partial A_1x}{x_{2}} & \cdots & \frac{\partial A_1x}{x_{n}} \\ \frac{\partial A_2x}{x_{1}} & \frac{\partial A_2x}{x_{2}} & \cdots & \frac{\partial A_2x}{x_{n}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial A_mx}{x_{1}} & \frac{\partial A_mx}{x_{2}} & \cdots & \frac{\partial A_mx}{x_{n}} \\ \end{bmatrix}_{m\times n} = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix}_{m\times n} = A \end{split}

其中AiA_i表示 矩阵AA的第ii行

  • 分母布局

∂[(Ax)T]1×m∂[x]n×1=[∂(Ax)Tx1∂(Ax)Tx2⋮∂(Ax)Txn]=[∂A1xx1∂A2xx1⋯∂Amxx1∂A1xx2∂A2xx2⋯∂Amxx2⋮⋮⋱⋮∂A1xxn∂A2xxn⋯∂Amxxn]n×m=[a11a21⋯am1a12a22⋯am2⋮⋮⋱⋮a1na2n⋯amn]n×m=AT\frac{\partial [(Ax)^T]_{1\times m}}{\partial [x]_{n\times 1}} = \begin{bmatrix} \frac{\partial (Ax)^T}{x_{1}} \\ \frac{\partial (Ax)^T}{x_{2}} \\ \vdots \\ \frac{\partial (Ax)^T}{x_{n}} \\ \end{bmatrix} = \begin{bmatrix} \frac{\partial A_1x}{x_{1}} & \frac{\partial A_2x}{x_{1}} & \cdots & \frac{\partial A_mx}{x_{1}} \\ \frac{\partial A_1x}{x_{2}} & \frac{\partial A_2x}{x_{2}} & \cdots & \frac{\partial A_mx}{x_{2}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial A_1x}{x_{n}} & \frac{\partial A_2x}{x_{n}} & \cdots & \frac{\partial A_mx}{x_{n}} \\ \end{bmatrix}_{n\times m} = \begin{bmatrix} a_{11} & a_{21} & \cdots & a_{m1} \\ a_{12} & a_{22} & \cdots & a_{m2} \\ \vdots & \vdots & \ddots & \vdots \\ a_{1n} & a_{2n} & \cdots & a_{mn} \\ \end{bmatrix}_{n\times m} = A^T

其中AiA_i表示 矩阵AA的第ii行

矩阵求导

A=[F11(X)F12(X)⋯F1n(X)F21(X)F22(X)⋯F2n(X)⋮⋮⋱⋮Fm1(X)Fm2(X)⋯Fmn(X)](m×n)A = \begin{bmatrix} F_{11}(X)&F_{12}(X)&\cdots&F_{1n}(X)\\ F_{21}(X)&F_{22}(X)&\cdots&F_{2n}(X)\\ \vdots&\vdots&\ddots&\vdots\\ F_{m1}(X)&F_{m2}(X)&\cdots&F_{mn}(X) \end{bmatrix}_{(m\times n)}