循环神经网络Recurrent Neural Networks

img

RNN的数学描述

参考零基础入门深度学习(5) - 循环神经网络

输入层

网络的输入是一串m维向量序列 x1,x2,⋯ ,xt,⋯\boldsymbol{x^1},\boldsymbol{x^2},\cdots,\boldsymbol{x^t},\cdots

x1=[x11x21⋮xm1],x2=[x12x22⋮xm2],⋯ ,xt=[x1tx2t⋮xmt],⋯\boldsymbol{x^1} = \begin{bmatrix} x^1_1\\x^1_2\\\vdots\\x^1_m \end{bmatrix}, \boldsymbol{x^2} = \begin{bmatrix} x^2_1\\x^2_2\\\vdots\\x^2_m \end{bmatrix}, \cdots, \boldsymbol{x^t} = \begin{bmatrix} x^t_1\\x^t_2\\\vdots\\x^t_m \end{bmatrix}, \cdots

循环层

网络的状态是一串n维向量序列 s0,s1,s2⋯ ,st,⋯\boldsymbol{s^0},\boldsymbol{s^1},\boldsymbol{s^2}\cdots,\boldsymbol{s^t},\cdots

[s1ts2t⋮snt]=f([u11u12⋯u1mu21u22⋯u2m⋮⋮⋱⋮un1un2⋯unm][x1tx2t⋮xmt]+[w11w12⋯w1nw21w22⋯w2n⋮⋮⋱⋮wn1wn2⋯wnn][s1t−1s2t−1⋮snt−1]+[b1Rb2R⋮bnR])t=1,2,⋯\begin{gather*} \begin{bmatrix} s^t_1\\s^t_2\\\vdots\\s^t_n \end{bmatrix}= f\left( \begin{bmatrix} u_{11}&u_{12}&\cdots&u_{1m}\\ u_{21}&u_{22}&\cdots&u_{2m}\\ \vdots&\vdots&\ddots&\vdots\\ u_{n1}&u_{n2}&\cdots&u_{nm}\\ \end{bmatrix} \begin{bmatrix} x^t_1\\x^t_2\\\vdots\\x^t_m \end{bmatrix} + \begin{bmatrix} w_{11}&w_{12}&\cdots&w_{1n}\\ w_{21}&w_{22}&\cdots&w_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ w_{n1}&w_{n2}&\cdots&w_{nn}\\ \end{bmatrix} \begin{bmatrix} s^{t-1}_1\\s^{t-1}_2\\\vdots\\s^{t-1}_n \end{bmatrix} + \begin{bmatrix} b^R_1\\b^R_2\\\vdots\\b^R_n \end{bmatrix} \right) \\ t = 1,2,\cdots \end{gather*}

输出层

网络的输出是一串m维的向量序列 o1,o2,⋯ ,ot,⋯\boldsymbol{o^{1}},\boldsymbol{o^{2}},\cdots,\boldsymbol{o^{t}},\cdots

[o1to2t⋮omt]=g([v11v12⋯v1nv21v22⋯v2n⋮⋮⋱⋮vm1vm2⋯vmn][s1ts2t⋮snt]+[b1Ob2O⋮bmO])t=1,2,⋯\begin{gather*} \begin{bmatrix} o^t_1\\o^t_2\\\vdots\\o^t_m \end{bmatrix}= g\left( \begin{bmatrix} v_{11}&v_{12}&\cdots&v_{1n}\\ v_{21}&v_{22}&\cdots&v_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ v_{m1}&v_{m2}&\cdots&v_{mn}\\ \end{bmatrix} \begin{bmatrix} s^t_1\\s^t_2\\\vdots\\s^t_n \end{bmatrix} + \begin{bmatrix} b^O_1\\b^O_2\\\vdots\\b^O_m \end{bmatrix} \right) \\ t = 1,2,\cdots \end{gather*}

网络的输出

网络在 tt 时刻的输出 ot\boldsymbol{o^t} 由前面各时刻的输入 xt,xt−1,⋯ ,x1\boldsymbol{x^t},\boldsymbol{x^{t-1}},\cdots,\boldsymbol{x^1}和初始状态 s0\boldsymbol{s^0} 决定

(下面的推导式中省略了偏置项 b\boldsymbol{b})

ot=g(Vst)=g(Vf(Uxt+Wst−1))=g(Vf(Uxt+Wf(Uxt−1+Wst−2)))⋮=g(Vf(Uxt+Wf(Uxt−1+Wf(Uxt−2+⋯+Wf(Ux1+Ws0)))))\begin{split} \boldsymbol{o^t} &= g\left( V\boldsymbol{s^t}\right) \\ &= g\left( Vf\left(U\boldsymbol{x^t}+W\boldsymbol{s^{t-1}}\right)\right) \\ &=g\left( Vf\left(U\boldsymbol{x^t}+Wf\left(U\boldsymbol{x^{t-1}}+W\boldsymbol{s^{t-2}}\right)\right)\right) \\ &\vdots\\ &=g\left( Vf\left(U\boldsymbol{x^t}+Wf\left(U\boldsymbol{x^{t-1}}+Wf\left(U\boldsymbol{x^{t-2}}+\cdots+ Wf\left(U\boldsymbol{x^1}+W\boldsymbol{s^0}\right)\right)\right)\right)\right) \\ \end{split}

网络输出的误差

网络在每个 tt 时刻的输出 ot\boldsymbol{o^t} 都对应一个目标向量 tt\boldsymbol{t}^t (target), 每个时刻都对应一个误差, 用EtE^t来表示 , EtE^t 是关于 ot\boldsymbol{o^t}和 tt\boldsymbol{t}^t 的函数, 例如采用二范数的平方表示误差, 误差函数如下计算

Et=12∥ot−tt∥22=12∑i=1m(oit−tit)2\begin{split} E^t &= \frac{1}{2}\|\boldsymbol{o}^t-\boldsymbol{t}^t\|_2^2 \\ &= \frac{1}{2}\sum_{i=1}^m (o^t_i-t^t_i)^2 \end{split}

梯度的计算(Back Propagate Through Time, BPTT)

循环层到输出层

记输出层 tt 时刻的输入向量为 ξt\boldsymbol{\xi}^{t}

[o1to2t⋮omt]=g([ξ1tξ2t⋮ξmt]),[ξ1tξ2t⋮ξmt]=[v11v12⋯v1nv21v22⋯v2n⋮⋮⋱⋮vm1vm2⋯vmn][s1ts2t⋮snt]+[b1Ob2O⋮bmO]\begin{split} \begin{bmatrix} o^t_1\\o^t_2\\\vdots\\o^t_m \end{bmatrix}= g\left( \begin{bmatrix} \xi^t_1\\ \xi^t_2\\ \vdots\\ \xi^t_m \end{bmatrix} \right) ,\quad \begin{bmatrix} \xi^t_1\\ \xi^t_2\\ \vdots\\ \xi^t_m \end{bmatrix}= \begin{bmatrix} v_{11}&v_{12}&\cdots&v_{1n}\\ v_{21}&v_{22}&\cdots&v_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ v_{m1}&v_{m2}&\cdots&v_{mn}\\ \end{bmatrix} \begin{bmatrix} s^t_1\\s^t_2\\\vdots\\s^t_n \end{bmatrix} + \begin{bmatrix} b^O_1\\b^O_2\\\vdots\\b^O_m \end{bmatrix} \end{split}

∂Et∂vij=∂Et∂ξit⋅∂ξit∂vij=∂Et∂ξit⋅sjt∂Et∂biO=∂Et∂ξit⋅∂ξit∂biO=∂Et∂ξit⋅1i=1,⋯ ,mj=1,⋯ ,n\begin{split} \frac{\partial E^t}{\partial v_{ij}} &= \frac{\partial E^t}{\partial \xi^t_i}\cdot\frac{\partial \xi^t_i}{\partial v_{ij}} =\frac{\partial E^t}{\partial \xi^t_i}\cdot s^t_j \\ \frac{\partial E^t}{\partial b^O_{i}} &= \frac{\partial E^t}{\partial \xi^t_i}\cdot\frac{\partial \xi^t_i}{\partial b^O_{i}} =\frac{\partial E^t}{\partial \xi^t_i}\cdot 1 \end{split} \qquad i=1,\cdots,m\quad j=1,\cdots,n

向量化计算梯度

∂Et∂bO=[∂Et∂ξ1t∂Et∂ξ2t⋮∂Et∂ξmt],∂Et∂V=[∂Et∂ξ1t∂Et∂ξ2t⋮∂Et∂ξmt][s1ts2t⋯snt]=[∂Et∂ξ1ts1t∂Et∂ξ1ts2t⋯∂Et∂ξ1tsnt∂Et∂ξ2ts1t∂Et∂ξ2ts2t⋯∂Et∂ξ2tsnt⋮⋮⋱⋮∂Et∂ξmts1t∂Et∂ξmts2t⋯∂Et∂ξmtsnt]\frac{\partial E^t}{\partial \boldsymbol{b^O}} = \begin{bmatrix} \frac{\partial E^t}{\partial \xi^t_1}\\\frac{\partial E^t}{\partial \xi^t_2}\\\vdots\\\frac{\partial E^t}{\partial \xi^t_m} \end{bmatrix}, \qquad \frac{\partial E^t}{\partial V} = \begin{bmatrix} \frac{\partial E^t}{\partial \xi^t_1}\\\frac{\partial E^t}{\partial \xi^t_2}\\\vdots\\\frac{\partial E^t}{\partial \xi^t_m} \end{bmatrix} \begin{bmatrix} s^t_1&s^t_2&\cdots&s^t_n \end{bmatrix}= \begin{bmatrix} \frac{\partial E^t}{\partial \xi^t_1} s^t_1 & \frac{\partial E^t}{\partial \xi^t_1} s^t_2 & \cdots & \frac{\partial E^t}{\partial \xi^t_1} s^t_n \\ \frac{\partial E^t}{\partial \xi^t_2} s^t_1 & \frac{\partial E^t}{\partial \xi^t_2} s^t_2 & \cdots & \frac{\partial E^t}{\partial \xi^t_2} s^t_n \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial E^t}{\partial \xi^t_m} s^t_1 & \frac{\partial E^t}{\partial \xi^t_m} s^t_2 & \cdots & \frac{\partial E^t}{\partial \xi^t_m} s^t_n \\ \end{bmatrix}

输入层到循环层

记循环层 tt 时刻的输入向量为 ηt\boldsymbol{\eta}^t

[s1ts2t⋮snt]=f([η1tη2t⋮ηnt]),[η1tη2t⋮ηnt]=[u11u12⋯u1mu21u22⋯u2m⋮⋮⋱⋮un1un2⋯unm][x1tx2t⋮xmt]+[w11w12⋯w1nw21w22⋯w2n⋮⋮⋱⋮wn1wn2⋯wnn][s1t−1s2t−1⋮snt−1]+[b1Rb2R⋮bnR]\begin{bmatrix} s^t_1\\s^t_2\\\vdots\\s^t_n \end{bmatrix}= f\left( \begin{bmatrix} \eta^t_1 \\ \eta^t_2 \\ \vdots \\ \eta^t_n \end{bmatrix} \right), \qquad \begin{bmatrix} \eta^t_1 \\ \eta^t_2 \\ \vdots \\ \eta^t_n \end{bmatrix}= \begin{bmatrix} u_{11}&u_{12}&\cdots&u_{1m}\\ u_{21}&u_{22}&\cdots&u_{2m}\\ \vdots&\vdots&\ddots&\vdots\\ u_{n1}&u_{n2}&\cdots&u_{nm}\\ \end{bmatrix} \begin{bmatrix} x^t_1\\x^t_2\\\vdots\\x^t_m \end{bmatrix} + \begin{bmatrix} w_{11}&w_{12}&\cdots&w_{1n}\\ w_{21}&w_{22}&\cdots&w_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ w_{n1}&w_{n2}&\cdots&w_{nn}\\ \end{bmatrix} \begin{bmatrix} s^{t-1}_1\\s^{t-1}_2\\\vdots\\s^{t-1}_n \end{bmatrix} + \begin{bmatrix} b^R_1\\b^R_2\\\vdots\\b^R_n \end{bmatrix}

关于矩阵U的偏导

由上面的记号, tt 时刻循环层的输入为ηt\boldsymbol{\eta}^t, ηt\boldsymbol{\eta}^t 是网络在 tt 时刻的输入 xt\boldsymbol{x}^t 和 上一时刻的状态 st−1\boldsymbol{s}^{t-1} 的线性变换

ηt=Uxt+Wst−1+bRst−1=f(ηt−1)\begin{gather} \boldsymbol{\eta}^t = U\boldsymbol{x}^t + W\boldsymbol{s}^{t-1}+\boldsymbol{b}^R\\ \boldsymbol{s}^{t-1} = f(\boldsymbol{\eta}^{t-1}) \end{gather}

下面的公式推导出一个 ∂Et/∂U\partial E^t/\partial U 关于时间的递推式, 我们记 ∂Et∂U(t)\frac{\partial E^t}{\partial U}(t) 为 tt 时刻网络输出的误差 EE 关于

∂Et∂U=∂Et∂ηt∂ηt∂U(ηt=Uxt+Wst−1+bR)→=∂Et∂ηt(∂Uxt∂U+∂Wst−1∂U)=∂Et∂ηt(∂Uxt∂U+W∂st−1∂ηt−1∂ηt−1∂U)将∂Et∂ηt乘进括号中去→=∂Et∂ηt∂Uxt∂U+∂Et∂ηt∂Wst−1∂ηt−1∂ηt−1∂U(∂Wst−1∂ηt−1=∂ηt∂ηt−1)→=∂Et∂ηt∂Uxt∂U+∂Et∂ηt∂ηt∂ηt−1∂ηt−1∂U=∂Et∂ηt∂Uxt∂U+∂Et∂ηt−1∂ηt−1∂U\begin{split} \frac{\partial E^t}{\partial U} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial U} \\ (\boldsymbol{\eta}^t = U\boldsymbol{x}^t + W\boldsymbol{s}^{t-1}+\boldsymbol{b}^R)\rightarrow % 第二个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} % 第二个等号括号中的内容 \left( \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial W\boldsymbol{s}^{t-1}}{\partial U} \right) \\ % 第三个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} % 第三个等号括号中的内容 \left( \frac{\partial U\boldsymbol{x}^t}{\partial U} + W\frac{\partial \boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial U} \right) \\ % 第四个等号 将\frac{\partial E^t}{\partial \boldsymbol{\eta}^t}乘进括号中去\rightarrow &= % 第四个等号加号左边的内容 \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + % 第四个等号加号右边的内容 \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial U} \\ \left(\frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}}= \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}}\right)\rightarrow % 第五个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial U} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial U} \end{split}

由这个递推式可以得到

∂Et∂U=∂Et∂ηt∂ηt∂U=∂Et∂ηt∂Uxt∂U+∂Et∂ηt−1∂ηt−1∂U=∂Et∂ηt∂Uxt∂U+∂Et∂ηt−1∂Uxt−1∂U+∂Et∂ηt−2∂ηt−2∂U=∂Et∂ηt∂Uxt∂U+∂Et∂ηt−1∂Uxt−1∂U+∂Et∂ηt−2∂Uxt−2∂U+⋯+∂Et∂η2∂Ux2∂U+∂Et∂η1∂Ux1∂U\begin{split} \frac{\partial E^t}{\partial U} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial U} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial U} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial U\boldsymbol{x}^{t-1}}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial \boldsymbol{\eta}^{t-2}}{\partial U} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial U\boldsymbol{x}^t}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial U\boldsymbol{x}^{t-1}}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial U\boldsymbol{x}^{t-2}}{\partial U} + \cdots + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{2}} \frac{\partial U\boldsymbol{x}^{2}}{\partial U} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{1}} \frac{\partial U\boldsymbol{x}^{1}}{\partial U} \end{split}

计算∂Et∂ηk∂Uxk∂U\frac{\partial E^t}{\partial \boldsymbol{\eta}^k}\frac{\partial U\boldsymbol{x}^k}{\partial U}

计算∂Et∂ηt\frac{\partial E^t}{\partial \boldsymbol{\eta}^t}

∂Et∂ηt=∂Et∂ξt∂ξt∂st∂st∂ηt=[∂Et∂ξ1t∂Et∂ξ2t⋯∂Et∂ξmt][∂ξ1t∂s1t∂ξ1t∂s2t⋯∂ξ1t∂snt∂ξ2t∂s1t∂ξ2t∂s2t⋯∂ξ2t∂snt⋮⋮⋱⋮∂ξmt∂s1t∂ξmt∂s2t⋯∂ξmt∂snt][∂s1t∂η1t∂s1t∂s2t⋯∂s1t∂snt∂s2t∂η1t∂s2t∂η2t⋯∂s2t∂ηnt⋮⋮⋱⋮∂snt∂η1t∂snt∂η2t⋯∂snt∂ηnt]=[∂Et∂ξ1t∂Et∂ξ2t⋯∂Et∂ξmt][v11v12⋯v1nv21v22⋯v2n⋮⋮⋱⋮vm1vm2⋯vmn][∂s1t∂η1t0⋯00∂s2t∂η2t⋯0⋮⋮⋱⋮00⋯∂snt∂ηnt]=[∂s1t∂η1t∑i=1m(∂Et∂ξitvi1),∂s2t∂η2t∑i=1m(∂Et∂ξitvi2),⋯ ,∂snt∂ηnt∑i=1m(∂Et∂ξitvin)]记为=[δ1ttδ2tt⋯δntt]\begin{split} \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} &= \frac{\partial E^t}{\partial \boldsymbol{\xi}^t} \frac{\partial \boldsymbol{\xi}^t}{\partial \boldsymbol{s}^t} \frac{\partial \boldsymbol{s}^t}{\partial \boldsymbol{\eta}^t} \\ &= % 第二行的行向量 \begin{bmatrix} \frac{\partial E^t}{\partial \xi^t_1}&\frac{\partial E^t}{\partial \xi^t_2}&\cdots&\frac{\partial E^t}{\partial \xi^t_m} \end{bmatrix} % 第二行的第一个矩阵 \begin{bmatrix} \frac{\partial \xi^t_1}{\partial s^t_1} & \frac{\partial \xi^t_1}{\partial s^t_2} & \cdots & \frac{\partial \xi^t_1}{\partial s^t_n} \\ \frac{\partial \xi^t_2}{\partial s^t_1} & \frac{\partial \xi^t_2}{\partial s^t_2} & \cdots & \frac{\partial \xi^t_2}{\partial s^t_n} \\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial \xi^t_m}{\partial s^t_1} & \frac{\partial \xi^t_m}{\partial s^t_2} & \cdots & \frac{\partial \xi^t_m}{\partial s^t_n} \\ \end{bmatrix} % 第二行的第二个矩阵 \begin{bmatrix} \frac{\partial s^t_1}{\partial \eta^t_1} & \frac{\partial s^t_1}{\partial s^t_2} & \cdots & \frac{\partial s^t_1}{\partial s^t_n} \\ \frac{\partial s^t_2}{\partial \eta^t_1} & \frac{\partial s^t_2}{\partial \eta^t_2} & \cdots & \frac{\partial s^t_2}{\partial \eta^t_n} \\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial s^t_n}{\partial \eta^t_1} & \frac{\partial s^t_n}{\partial \eta^t_2} & \cdots & \frac{\partial s^t_n}{\partial \eta^t_n} \\ \end{bmatrix} \\ &= % 第三行的行向量 \begin{bmatrix} \frac{\partial E^t}{\partial \xi^t_1}&\frac{\partial E^t}{\partial \xi^t_2}&\cdots&\frac{\partial E^t}{\partial \xi^t_m} \end{bmatrix} % 第三行的V矩阵 \begin{bmatrix} v_{11}&v_{12}&\cdots&v_{1n}\\ v_{21}&v_{22}&\cdots&v_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ v_{m1}&v_{m2}&\cdots&v_{mn}\\ \end{bmatrix} % 第三行的对角矩阵 \begin{bmatrix} \frac{\partial s^t_1}{\partial \eta^t_1} & 0 & \cdots & 0 \\ 0 & \frac{\partial s^t_2}{\partial \eta^t_2} & \cdots & 0 \\ \vdots&\vdots&\ddots&\vdots\\ 0 & 0 & \cdots & \frac{\partial s^t_n}{\partial \eta^t_n} \\ \end{bmatrix} \\ &= \left[ \frac{\partial s^t_1}{\partial \eta^t_1} \sum_{i=1}^m(\frac{\partial E^t}{\partial \xi^t_i}v_{i1}) ,\quad \frac{\partial s^t_2}{\partial \eta^t_2} \sum_{i=1}^m(\frac{\partial E^t}{\partial \xi^t_i}v_{i2}) ,\quad \cdots ,\quad \frac{\partial s^t_n}{\partial \eta^t_n} \sum_{i=1}^m(\frac{\partial E^t}{\partial \xi^t_i}v_{in}) \right] \\ 记为&= \begin{bmatrix} \delta^{tt}_1&\delta^{tt}_2&\cdots&\delta^{tt}_n \end{bmatrix} \end{split}

∂Et∂ηt\frac{\partial E^t}{\partial \boldsymbol{\eta}^t} 的结果记为 δtt\boldsymbol{\delta^{tt}}, 称为循环层 tt 时刻(第二个 tt)的输入的误差项 (网络 tt 时刻输出的误差关于循环层 tt 时刻输入的偏导数)

计算∂Et∂ηk\frac{\partial E^t}{\partial \boldsymbol{\eta}^k}

∂ηt∂ηt−1=∂Wst−1∂ηt−1=W∂st−1∂ηt−1=W[∂s1t−1∂η1t−1∂s1t−1∂η2t−1⋯∂s1t−1∂ηnt−1∂s2t−1∂η1t−1∂s2t−1∂η2t−1⋯∂s2t−1∂ηnt−1⋮⋮⋱⋮∂snt−1∂η1t−1∂snt−1∂η2t−1⋯∂snt−1∂ηnt−1]=W[∂s1t−1∂η1t−10⋯00∂s2t−1∂η2t−1⋯0⋮⋮⋱⋮00⋯∂snt−1∂ηnt−1]=W[f′(η1t−1)0⋯00f′(η2t−1)⋯0⋮⋮⋱⋮00⋯f′(ηnt−1)]\begin{split} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}} &= \frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}}= W\frac{\partial \boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}}= W % 第一个矩阵 \begin{bmatrix} \frac{\partial s^{t-1}_{1}}{\partial \eta^{t-1}_{1}}& \frac{\partial s^{t-1}_{1}}{\partial \eta^{t-1}_{2}}& \cdots& \frac{\partial s^{t-1}_{1}}{\partial \eta^{t-1}_{n}} \\ \frac{\partial s^{t-1}_{2}}{\partial \eta^{t-1}_{1}}& \frac{\partial s^{t-1}_{2}}{\partial \eta^{t-1}_{2}}& \cdots& \frac{\partial s^{t-1}_{2}}{\partial \eta^{t-1}_{n}} \\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial s^{t-1}_{n}}{\partial \eta^{t-1}_{1}}& \frac{\partial s^{t-1}_{n}}{\partial \eta^{t-1}_{2}}& \cdots& \frac{\partial s^{t-1}_{n}}{\partial \eta^{t-1}_{n}} \end{bmatrix}= % 第二个矩阵 W\begin{bmatrix} \frac{\partial s^{t-1}_{1}}{\partial \eta^{t-1}_{1}}&0&\cdots&0 \\ 0&\frac{\partial s^{t-1}_{2}}{\partial \eta^{t-1}_{2}}&\cdots&0 \\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\frac{\partial s^{t-1}_{n}}{\partial \eta^{t-1}_{n}} \end{bmatrix} \\ &= W\begin{bmatrix} f'(\eta^{t-1}_{1})&0&\cdots&0 \\ 0&f'(\eta^{t-1}_{2})&\cdots&0 \\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&f'(\eta^{t-1}_{n}) \end{bmatrix} \end{split}

∂Et∂ηk=∂Et∂ξt∂ξt∂st∂st∂ηt(∂ηt∂ηt−1⋯∂ηk+1∂ηk)=[δ1ttδ2tt⋯δntt]∏i=(t−1)k(W[f′(η1i)⋯0⋮⋱⋮0⋯f′(ηni)])记为=[δ1tkδ2tk⋯δntk](t≥k≥1)\begin{split} \frac{\partial E^t}{\partial \boldsymbol{\eta}^k} &= \frac{\partial E^t}{\partial \boldsymbol{\xi}^t} \frac{\partial \boldsymbol{\xi}^t}{\partial \boldsymbol{s}^t} \frac{\partial \boldsymbol{s}^t}{\partial \boldsymbol{\eta}^t} \left( \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}} \cdots \frac{\partial \boldsymbol{\eta}^{k+1}}{\partial \boldsymbol{\eta}^{k}} \right)\\ &= \begin{bmatrix} \delta^{tt}_1&\delta^{tt}_2&\cdots&\delta^{tt}_n \end{bmatrix} % 连乘 \prod_{i=(t-1)}^{k} \left( W\begin{bmatrix} f'(\eta^{i}_{1})&\cdots&0\\ \vdots&\ddots&\vdots\\ 0&\cdots&f'(\eta^{i}_{n}) \end{bmatrix} \right)\\ 记为&= \begin{bmatrix} \delta^{tk}_1&\delta^{tk}_2&\cdots&\delta^{tk}_n \end{bmatrix} \qquad (t \ge k \ge 1) \end{split}

∂Et∂ηk\frac{\partial E^t}{\partial \boldsymbol{\eta}^k} 的结果记为 δtk\boldsymbol{\delta^{tk}}, 称为循环层 kk 时刻输入的误差项 (网络 tt 时刻输出的误差关于循环层 kk 时刻输入的偏导数)

实际计算中我们会一步一步地计算δtt,δt(t−1),⋯ ,δt1\boldsymbol{\delta}^{tt},\boldsymbol{\delta}^{t(t-1)},\cdots,\boldsymbol{\delta}^{t1}, 而不是使用连乘运算

[δ1t(t−1)δ2t(t−1)⋯δnt(t−1)]=[δ1tkδ2tk⋯δntk]W[f′(η1t−1)⋯0⋮⋱⋮0⋯f′(ηnt−1)][δ1t(t−2)δ2t(t−2)⋯δnt(t−2)]=[δ1t(t−1)δ2t(t−1)⋯δnt(t−1)]W[f′(η1t−2)⋯0⋮⋱⋮0⋯f′(ηnt−2)]⋮[δ1t1δ2t1⋯δnt1]=[δ1t(2)δ2t(2)⋯δnt(2)]W[f′(η11)⋯0⋮⋱⋮0⋯f′(ηn1)]\begin{split} % 第一行 \begin{bmatrix} \delta^{t(t-1)}_1&\delta^{t(t-1)}_2&\cdots&\delta^{t(t-1)}_n \end{bmatrix} &= \begin{bmatrix} \delta^{tk}_1&\delta^{tk}_2&\cdots&\delta^{tk}_n \end{bmatrix} W\begin{bmatrix} f'(\eta^{t-1}_{1})&\cdots&0 \\ \vdots&\ddots&\vdots\\ 0&\cdots&f'(\eta^{t-1}_{n}) \end{bmatrix} % 第二行 \\ \begin{bmatrix} \delta^{t(t-2)}_1&\delta^{t(t-2)}_2&\cdots&\delta^{t(t-2)}_n \end{bmatrix} &= \begin{bmatrix} \delta^{t(t-1)}_1&\delta^{t(t-1)}_2&\cdots&\delta^{t(t-1)}_n \end{bmatrix} W\begin{bmatrix} f'(\eta^{t-2}_{1})&\cdots&0 \\ \vdots&\ddots&\vdots\\ 0&\cdots&f'(\eta^{t-2}_{n}) \end{bmatrix} \\ &\vdots \\ % 第三行 \begin{bmatrix} \delta^{t1}_1&\delta^{t1}_2&\cdots&\delta^{t1}_n \end{bmatrix} &= \begin{bmatrix} \delta^{t(2)}_1&\delta^{t(2)}_2&\cdots&\delta^{t(2)}_n \end{bmatrix} W\begin{bmatrix} f'(\eta^{1}_{1})&\cdots&0 \\ \vdots&\ddots&\vdots\\ 0&\cdots&f'(\eta^{1}_{n}) \end{bmatrix} \end{split}

计算∂Uxk∂U\frac{\partial U\boldsymbol{x}^k}{\partial U}

∂Uxk∂U=[(∂η1k∂u11⋯∂η1k∂u1m⋮⋱⋮∂η1k∂un1⋯∂η1k∂unm)⋮(∂ηik∂u11⋯∂ηik∂u1m⋮⋱⋮∂ηik∂un1⋯∂ηik∂unm)⋮(∂ηnk∂u11⋯∂ηnk∂u1m⋮⋱⋮∂ηnk∂un1⋯∂ηnk∂unm)]=[(x1kx2k⋯xmk00⋯0⋮⋮⋱⋮00⋯0)⋮(00⋯0⋮⋮⋮x1kx2k⋯xmk⋮⋮⋮00⋯0)1⋮i⋮n⋮(00⋯0⋮⋮⋱⋮00⋯0x1kx2k⋯xmk)](t≥k≥1)\frac{\partial U\boldsymbol{x}^k}{\partial U}= % 第一个大矩阵 \begin{bmatrix} \left(\begin{smallmatrix} \frac{\partial \eta^k_1}{\partial u_{11}} & \cdots & \frac{\partial \eta^k_1}{\partial u_{1m}}\\ \vdots & \ddots & \vdots \\ \frac{\partial \eta^k_1}{\partial u_{n1}} & \cdots & \frac{\partial \eta^k_1}{\partial u_{nm}}\\ \end{smallmatrix}\right) \\ \vdots \\ \left(\begin{smallmatrix} \frac{\partial \eta^k_i}{\partial u_{11}} & \cdots & \frac{\partial \eta^k_i}{\partial u_{1m}}\\ \vdots & \ddots & \vdots \\ \frac{\partial \eta^k_i}{\partial u_{n1}} & \cdots & \frac{\partial \eta^k_i}{\partial u_{nm}}\\ \end{smallmatrix}\right) \\ \vdots \\ \left(\begin{smallmatrix} \frac{\partial \eta^k_n}{\partial u_{11}} & \cdots & \frac{\partial \eta^k_n}{\partial u_{1m}}\\ \vdots & \ddots & \vdots \\ \frac{\partial \eta^k_n}{\partial u_{n1}} & \cdots & \frac{\partial \eta^k_n}{\partial u_{nm}}\\ \end{smallmatrix}\right) \end{bmatrix}= % 第二个大矩阵 \begin{bmatrix} \left(\begin{smallmatrix} x^k_1 & x^k_2 &\cdots & x^k_m\\ 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0\\ \end{smallmatrix}\right) \\ \vdots \\ \left(\begin{smallmatrix} 0 & 0 & \cdots & 0 \\ \vdots & \vdots & & \vdots \\ x^k_1 & x^k_2 &\cdots & x^k_m\\ \vdots & \vdots & & \vdots \\ 0 & 0 & \cdots & 0\\ \end{smallmatrix}\right) \begin{smallmatrix} 1\\\vdots\\i\\\vdots\\n \end{smallmatrix} \\ \vdots \\ \left(\begin{smallmatrix} 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0\\ x^k_1 & x^k_2 &\cdots & x^k_m\\ \end{smallmatrix}\right) \end{bmatrix} \qquad (t \ge k \ge 1)

计算∂Et∂ηk∂Uxk∂U\frac{\partial E^t}{\partial \boldsymbol{\eta}^k}\frac{\partial U\boldsymbol{x}^k}{\partial U}

∂Et∂ηk⋅∂Uxk∂U=[δ1tkδ2tk⋯δntk][(x1kx2k⋯xmk00⋯0⋮⋮⋱⋮00⋯0)⋮(00⋯0⋮⋮⋮x1kx2k⋯xmk⋮⋮⋮00⋯0)1⋮i⋮n⋮(00⋯0⋮⋮⋱⋮00⋯0x1kx2k⋯xmk)]=[δ1tkδ2tk⋮δntk][x1kx2k⋯xmk](t≥k≥1)\frac{\partial E^t}{\partial \boldsymbol{\eta}^k} \cdot \frac{\partial U\boldsymbol{x}^k}{\partial U} = \begin{bmatrix} \delta^{tk}_1&\delta^{tk}_2&\cdots&\delta^{tk}_n \end{bmatrix} % 大矩阵 \begin{bmatrix} \left(\begin{smallmatrix} x^k_1 & x^k_2 &\cdots & x^k_m\\ 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0\\ \end{smallmatrix}\right) \\ \vdots \\ \left(\begin{smallmatrix} 0 & 0 & \cdots & 0 \\ \vdots & \vdots & & \vdots \\ x^k_1 & x^k_2 &\cdots & x^k_m\\ \vdots & \vdots & & \vdots \\ 0 & 0 & \cdots & 0\\ \end{smallmatrix}\right) \begin{smallmatrix} 1\\\vdots\\i\\\vdots\\n \end{smallmatrix} \\ \vdots \\ \left(\begin{smallmatrix} 0 & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 0\\ x^k_1 & x^k_2 &\cdots & x^k_m\\ \end{smallmatrix}\right) \end{bmatrix} % 第二个等号 =\begin{bmatrix} \delta^{tk}_1 \\ \delta^{tk}_2 \\ \vdots \\ \delta^{tk}_n \end{bmatrix} \begin{bmatrix} x^k_1 & x^k_2 & \cdots & x^k_m \end{bmatrix} \qquad (t \ge k \ge 1)

最后结果U的梯度

∂Et∂U=∑k=1t([δ1tkδ2tk⋮δntk][x1kx2k⋯xmk])\frac{\partial E^t}{\partial U} % 第一个等号 =\sum_{k=1}^t \left( \begin{bmatrix} \delta^{tk}_1 \\ \delta^{tk}_2 \\ \vdots \\ \delta^{tk}_n \end{bmatrix} \begin{bmatrix} x^k_1 & x^k_2 & \cdots & x^k_m \end{bmatrix} \right)

关于矩阵W的偏导

∂Et∂W=∂Et∂ηt∂ηt∂W(ηt=Uxt+Wst−1+bR)→=∂Et∂ηt(∂Wst−1∂W)(莱布尼茨法则)→=∂Et∂ηt(∂W∂Wst−1+W∂st−1∂W)=∂Et∂ηt∂W∂Wst−1+∂Et∂ηtW∂st−1∂ηt−1∂ηt−1∂W(∂Wst−1∂ηt−1=∂ηt∂ηt−1)→=∂Et∂ηt∂W∂Wst−1+∂Et∂ηt∂ηt∂ηt−1∂ηt−1∂W=∂Et∂ηt∂W∂Wst−1+∂Et∂ηt−1∂ηt−1∂W\begin{split} \frac{\partial E^t}{\partial W} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial W} \\ (\boldsymbol{\eta}^t = U\boldsymbol{x}^t + W\boldsymbol{s}^{t-1}+\boldsymbol{b}^R)\rightarrow % 第二个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} % 第二个等号括号中的内容 \left( \frac{\partial W\boldsymbol{s}^{t-1}}{\partial W} \right) \\ (莱布尼茨法则)\rightarrow &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} % 第二个等号括号中的内容 \left( \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + W\frac{\partial \boldsymbol{s}^{t-1}}{\partial W} \right) \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} W\frac{\partial \boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial W} \\ \left(\frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}}= \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}}\right)\rightarrow &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial W} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial W} \end{split}

∂Et∂W=∂Et∂ηt∂ηt∂W=∂Et∂ηt∂W∂Wst−1+∂Et∂ηt−1∂ηt−1∂W=∂Et∂ηt∂W∂Wst−1+∂Et∂ηt−1∂W∂Wst−2+∂Et∂ηt−2∂ηt−2∂W=∂Et∂ηt∂W∂Wst−1+∂Et∂ηt−1∂W∂Wst−2+∂Et∂ηt−2∂W∂Wst−3+⋯+∂Et∂η2∂W∂Ws1+∂Et∂η1∂W∂Ws0\begin{split} \frac{\partial E^t}{\partial W} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial W} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial W} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-2} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial \boldsymbol{\eta}^{t-2}}{\partial W} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-2} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial W}{\partial W}\boldsymbol{s}^{t-3} + \cdots + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{2}} \frac{\partial W}{\partial W}\boldsymbol{s}^{1} + \frac{\partial E^t}{\partial \boldsymbol{\eta}^{1}} \frac{\partial W}{\partial W}\boldsymbol{s}^{0} \end{split}

计算∂Et∂ηk∂W∂Wsk−1\frac{\partial E^t}{\partial \boldsymbol{\eta}^k}\frac{\partial W}{\partial W}\boldsymbol{s}^{k-1}

计算∂W∂W\frac{\partial W}{\partial W}

∂W∂W=∂(w11⋯wn1⋮⋱⋮wn1⋯wnn)∂(w11⋯wn1⋮⋱⋮wn1⋯wnn)=[(10⋯000⋯0⋮⋮⋱⋮00⋯0)⋯(00⋯100⋯0⋮⋮⋱⋮00⋯0)⋮⋱⋮(00⋯000⋯0⋮⋮⋱⋮10⋯0)⋯(00⋯000⋯0⋮⋮⋱⋮00⋯1)]\begin{split} \frac{\partial W}{\partial W}&= \frac {\partial \left(\begin{smallmatrix} w_{11}&\cdots&w_{n1}\\ \vdots&\ddots&\vdots\\ w_{n1}&\cdots&w_{nn} \end{smallmatrix}\right)} {\partial \left(\begin{smallmatrix} w_{11}&\cdots&w_{n1}\\ \vdots&\ddots&\vdots\\ w_{n1}&\cdots&w_{nn} \end{smallmatrix}\right)} \\ &= % 大矩阵 \begin{bmatrix} \left( \begin{smallmatrix} 1 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) & \cdots & \left( \begin{smallmatrix} 0 & 0 & \cdots & 1 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) \\ \vdots&\ddots&\vdots\\ \left( \begin{smallmatrix} 0 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 1 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) & \cdots & \left( \begin{smallmatrix} 0 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 1 \\ \end{smallmatrix} \right) \end{bmatrix} \end{split}

计算∂Et∂ηk∂W∂Wsk−1\frac{\partial E^t}{\partial \boldsymbol{\eta}^k}\frac{\partial W}{\partial W}\boldsymbol{s}^{k-1}

∂Et∂ηk∂W∂Wsk−1=[δ1tkδ2tk⋯δntk][(10⋯000⋯0⋮⋮⋱⋮00⋯0)⋯(00⋯100⋯0⋮⋮⋱⋮00⋯0)⋮⋱⋮(00⋯000⋯0⋮⋮⋱⋮10⋯0)⋯(00⋯000⋯0⋮⋮⋱⋮00⋯1)][s1k−1s2k−1⋮snk−1]=[δ1tkδ2tk⋮δntk][s1k−1s2k−1⋯snk−1](t≥k≥1)\begin{split} \frac{\partial E^t}{\partial \boldsymbol{\eta}^k}\frac{\partial W}{\partial W}\boldsymbol{s}^{k-1} &= \begin{bmatrix} \delta^{tk}_1&\delta^{tk}_2&\cdots&\delta^{tk}_n \end{bmatrix} % 大矩阵 \begin{bmatrix} \left( \begin{smallmatrix} 1 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) & \cdots & \left( \begin{smallmatrix} 0 & 0 & \cdots & 1 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) \\ \vdots&\ddots&\vdots\\ \left( \begin{smallmatrix} 0 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 1 & 0 & \cdots & 0 \\ \end{smallmatrix} \right) & \cdots & \left( \begin{smallmatrix} 0 & 0 & \cdots & 0 \\ 0 & 0 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & 1 \\ \end{smallmatrix} \right) \end{bmatrix} \begin{bmatrix} s^{k-1}_1\\s^{k-1}_2\\\vdots\\s^{k-1}_n \end{bmatrix} \\ &= \begin{bmatrix} \delta^{tk}_1\\\delta^{tk}_2\\\vdots\\\delta^{tk}_n \end{bmatrix} \begin{bmatrix} s^{k-1}_1&s^{k-1}_2&\cdots&s^{k-1}_n \end{bmatrix} \qquad (t \geq k \geq 1) \end{split}

最后结果W的梯度

∂Et∂W=∑k=1t([δ1tkδ2tk⋮δntk][s1k−1s2k−1⋯snk−1])\frac{\partial E^t}{\partial W} % 第一个等号 =\sum_{k=1}^t \left( \begin{bmatrix} \delta^{tk}_1\\\delta^{tk}_2\\\vdots\\\delta^{tk}_n \end{bmatrix} \begin{bmatrix} s^{k-1}_1&s^{k-1}_2&\cdots&s^{k-1}_n \end{bmatrix} \right)

关于偏置项bR\boldsymbol{b}^R的偏导

∂Et∂bR=∂Et∂ηt∂ηt∂bR(ηt=Uxt+Wst−1+bR)→=∂Et∂ηt(∂bR∂bR+∂Wst−1∂bR)=∂Et∂ηt∂bR∂bR+∂Et∂ηt∂Wst−1∂ηt−1∂ηt−1∂bR(∂Wst−1∂ηt−1=∂ηt∂ηt−1)→=∂Et∂ηt∂bR∂bR+∂Et∂ηt∂ηt∂ηt−1∂ηt−1∂bR=∂Et∂ηt∂bR∂bR+∂Et∂ηt−1∂ηt−1∂bR\begin{split} \frac{\partial E^t}{\partial \boldsymbol{b}^R} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{b}^R} \\ (\boldsymbol{\eta}^t = U\boldsymbol{x}^t + W\boldsymbol{s}^{t-1}+\boldsymbol{b}^R)\rightarrow % 第二个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} % 第二个等号括号中的内容 \left( \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{b}^R} \right) \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial \boldsymbol{b}^R} \\ \left(\frac{\partial W\boldsymbol{s}^{t-1}}{\partial \boldsymbol{\eta}^{t-1}}= \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}}\right)\rightarrow &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial \boldsymbol{b}^R} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial \boldsymbol{b}^R} \end{split}

∂Et∂bR=∂Et∂ηt∂ηt∂bR=∂Et∂ηt∂bR∂bR+∂Et∂ηt−1∂ηt−1∂bR=∂Et∂ηt∂bR∂bR+∂Et∂ηt−1∂bR∂bR+∂Et∂ηt−2∂ηt−2∂bR=∂Et∂ηt∂bR∂bR+∂Et∂ηt−1∂bR∂bR+∂Et∂ηt−2∂bR∂bR+⋯+∂Et∂η1∂bR∂bR\begin{split} \frac{\partial E^t}{\partial \boldsymbol{b}^R} % 第一个等号 &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{\eta}^t}{\partial \boldsymbol{b}^R} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{\eta}^{t-1}}{\partial \boldsymbol{b}^R} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial \boldsymbol{\eta}^{t-2}}{\partial \boldsymbol{b}^R} \\ &= \frac{\partial E^t}{\partial \boldsymbol{\eta}^t} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-1}} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{t-2}} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}+ \cdots+ \frac{\partial E^t}{\partial \boldsymbol{\eta}^{1}} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R} \end{split}

计算∂Et∂ηk∂bR∂bR\frac{\partial E^t}{\partial \boldsymbol{\eta}^{k}}\frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}

∂Et∂ηk∂bR∂bR=∂Et∂ηk⋅Inn=∂Et∂ηk=[δ1tkδ2tk⋮δntk]\frac{\partial E^t}{\partial \boldsymbol{\eta}^{k}} \frac{\partial \boldsymbol{b}^R}{\partial \boldsymbol{b}^R}= \frac{\partial E^t}{\partial \boldsymbol{\eta}^{k}} \cdot I_{nn}= \frac{\partial E^t}{\partial \boldsymbol{\eta}^{k}}= \begin{bmatrix} \delta^{tk}_1\\\delta^{tk}_2\\\vdots\\\delta^{tk}_n \end{bmatrix}

最后结果 bR\boldsymbol{b}^R 的梯度

∂Et∂bR=∑k=1t([δ1tkδ2tk⋮δntk])\frac{\partial E^t}{\partial \boldsymbol{b}^R}= \sum_{k=1}^t \left( \begin{bmatrix} \delta^{tk}_1\\\delta^{tk}_2\\\vdots\\\delta^{tk}_n \end{bmatrix} \right)