在网上看到有人贴了如下求导公式：
Y = A * X --> DY/DX = A'
Y = X * A --> DY/DX = A
Y = A' * X * B --> DY/DX = A * B'
Y = A' * X' * B --> DY/DX = B * A'
于是把以前学过的矩阵求导部分整理一下：
1. 矩阵Y 对标量x 求导：
相当于每个元素求导数后转置一下，注意M×N 矩阵求导后变成N×M 了
Y = [y(ij)] --> dY/dx = [dy(ji)/dx]
2. 标量y 对列向量X 求导：
注意与上面不同，这次括号内是求偏导，不转置，对N×1向量求导后还是N×1向量
y = f(x1,x2,..,xn) --> dy/dX = (Dy/Dx1,Dy/Dx2,..,Dy/Dxn)'
3. 行向量Y' 对列向量X 求导：
注意1×M 向量对N×1向量求导后是N×M 矩阵。
将Y 的每一列对X 求偏导，将各列构成一个矩阵。
重要结论：
dX'/dX = I
d(AX)'/dX = A'
4. 列向量Y 对行向量X’求导：
转化为行向量Y’对列向量X 的导数，然后转置。
注意M×1向量对1×N 向量求导结果为M×N 矩阵。
dY/dX' = (dY'/dX)'
5. 向量积对列向量X 求导运算法则：
注意与标量求导有点不同。
d(UV')/dX = (dU/dX)V' + U(dV'/dX)
d(U'V)/dX = (dU'/dX)V + (dV'/dX)U'
重要结论：
d(X'A)/dX = (dX'/dX)A + (dA/dX)X' = IA + 0X' = A
d(AX)/dX' = (d(X'A')/dX)' = (A')' = A
d(X'AX)/dX = (dX'/dX)AX + (d(AX)'/dX)X = AX + A'X
6. 矩阵Y 对列向量X 求导：
将Y 对X 的每一个分量求偏导，构成一个超向量。
注意该向量的每一个元素都是一个矩阵。
7. 矩阵积对列向量求导法则：
d(uV)/dX = (du/dX)V + u(dV/dX)
d(UV)/dX = (dU/dX)V + U(dV/dX)
重要结论：
d(X'A)/dX = (dX'/dX)A + X'(dA/dX) = IA + X'0 = A
8. 标量y 对矩阵X 的导数：
类似标量y 对列向量X 的导数，
把y 对每个X 的元素求偏导，不用转置。
dy/dX = [ Dy/Dx(ij) ]
重要结论：
y = U'XV = ΣΣu(i)x(ij)v(j) 于是 dy/dX = [u(i)v(j)] = UV'
y = U'X'XU 则 dy/dX = 2XUU'
y = (XU-V)'(XU-V) 则 dy/dX = d(U'X'XU - 2V'XU + V'V)/dX = 2XUU' - 2VU' + 0 = 2(XU-V)U'
9. 矩阵Y 对矩阵X 的导数：
将Y 的每个元素对X 求导，然后排在一起形成超级矩阵。
10. 乘积的导数
d(f*g)/dx=(df'/dx)g+(dg/dx)f'
结论
d(x'Ax)=(d(x'')/dx)Ax+(d(Ax)/dx)(x'')=Ax+A'x （注意：'' 是表示两次转置
Matrix Reference Manual
Matrix Calculus
Go to: Introduction , Notation , Index Contents
Notation
∙ Derivatives of Linear Products
∙ Derivatives of Quadratic Products ∙
Notation
∙
∙
∙
∙
∙ d/dx (y ) is a vector whose (i) element is dy(i)/dx d/dx (y) is a vector whose (i) element is dy/dx(i) d/dx (y T ) is a matrix whose (i,j) element is dy(j)/dx(i) d/dx (Y ) is a matrix whose (i,j) element is dy(i,j)/dx d/dX (y) is a matrix whose (i,j) element is dy/dx(i,j)
Note that the Hermitian transpose is not used because complex conjugates are not analytic.
In the expressions below matrices and vectors A , B , C do not depend on X .
Derivatives of Linear Products
∙
∙
∙
∙
∙ d/dx (AYB ) =A * d/dx (Y ) * B o d/dx (Ay ) =A * d/dx (y ) d/dx (x T A ) =A T o d/dx (x ) =I T T o d/dx (x a ) = d/dx (a x ) = a d/dX (a T Xb ) = ab T T T T T o d/dX (a Xa ) = d/dX (a X a ) = aa d/dX (a T X T b ) = ba T d/dx (YZ ) =Y * d/dx (Z ) + d/dx (Y ) * Z Derivatives of Quadratic Products
d/dx (Ax +b ) T C (D x+e ) = A T C(Dx+e) + D T C T (Ax+b)
T T o d/dx (x Cx ) = (C +C ) x
T [C: symmetric]: d/dx (x Cx ) = 2Cx
T d/dx (x x ) = 2x
T T T o d/dx (Ax +b ) (D x+e ) = A (Dx+e) + D (Ax+b)
T T d/dx (Ax +b ) (A x+b ) = 2A (Ax+b)
T T o [C: symmetric]: d/dx (Ax +b ) C (A x+b ) = 2A C(Ax+b)
T T T T ∙ d/dX (a X Xb ) = X(ab + ba)
T T T o d/dX (a X Xa ) = 2Xaa
T T T T T ∙ d/dX (a X CXb ) = C Xab + CXba
T T T T o d/dX (a X CXa ) = (C + C)Xaa
T T T o [C :Symmetric ] d/dX (a X CXa ) = 2CXaa
T T T ∙ d/dX ((Xa+b)C(Xa+b)) = (C+C)(Xa+b)a ∙
Derivatives of Cubic Products
∙ d/dx (x T Axx T ) = (A+AT )xx T +xT AxI
Derivatives of Inverses
∙ d/dx (Y -1) = -Y -1d/dx (Y ) Y -1
Derivative of Trace
Note: matrix dimensions must result in an n*n argument for tr(). ∙
∙
∙
∙
∙ d/dX (tr(X)) = I d/dX (tr(Xk ) ) =k (Xk -1) T d/dX (tr(AXk ) ) = SUM r=0:k-1(Xr AX k-r -1) T d/dX (tr(AX-1B) ) = -(X-1BAX -1) T -1-1-T T -T o d/dX (tr(AX) ) =d/dX (tr(XA) ) = -X A X d/dX (tr(AT XB T )) = d/dX (tr(BXT A )) = AB
o d/dX (tr(XAT ) ) = d/dX (tr(AT X) ) =d/dX (tr(XT A) ) = d/dX (tr(AXT ) ) = A
d/dX (tr(AXBXT ) ) = AT XB T + AXB
T T o d/dX (tr(XAX) ) = X(A+A)
T T T o d/dX (tr(XAX) ) = X(A+A)
T T o d/dX (tr(AXX) ) = (A+A)X
d/dX (tr(AXBX)) = AT X T B T + B T X T A T
[C:symmetric ] d/dX (tr((XT CX) -1A ) = d/dX (tr(A (XT CX) -1) = -(CX(XT CX) -1)(A+AT )(XT CX) -1
[B,C:symmetric ] d/dX (tr((XT CX) -1(XT BX) ) = d/dX (tr( (XT BX)(XT CX) -1) = -2(CX(XT CX) -1)X T BX(XT CX) -1 + 2BX(XT CX) -1
∙ ∙ ∙ ∙ ∙ ∙
Derivative of Determinant
Note: matrix dimensions must result in an n*n argument for det(). ∙
∙
∙
∙ d/dX (det(X)) = d/dX (det(XT ) ) = det(X)*X-T -T o d/dX (det(AXB)) = det(AXB)*X -T o d/dX (ln(det(AXB))) = X d/dX (det(Xk ) ) = k *det(Xk )*X-T k -T o d/dX (ln(det(X) )) = k X [Real ] d/dX (det(XT CX) ) = det(XT CX)*(C+CT )X(XT CX) -1 T T T -1o [C : Real,Symmetric ] d/dX (det(XCX) ) = 2det(XCX)* CX(XCX) [C : Real,Symmetricc ] d/dX (ln(det(XT CX)) ) = 2CX(XT CX) -1 Jacobian
If y is a function of x , then d y T /dx is the Jacobian matrix of y with respect to x .
Its determinant, |d y T /dx |, is the Jacobian of y with respect to x and represents the ratio of the hyper-volumes d y and d x . The Jacobian occurs when changing variables in an integration:
Integral(f(y )d y )=Integral(f(y (x )) |d y T /d x | dx ).
Hessian matrix
If f is a function of x then the symmetric matrix d2f/dx 2 = d /dx T (df/dx ) is the Hessian matrix of f(x ). A value of x for which df/dx = 0 corresponds to a minimum, maximum or saddle point according to whether the Hessian is positive definite, negative definite or indefinite.
d 2/dx 2 (a T x ) = 0
22T T T T ∙ d /dx (Ax +b ) C (D x+e ) = A CD + D C A
22T T o d /dx (x Cx ) = C +C
22T d /dx (x x ) = 2I
22T T T o d /dx (Ax +b ) (D x+e ) = A D + D A
22T T d /dx (Ax +b ) (A x+b ) = 2A A
22T T o [C: symmetric]: d /dx (Ax +b ) C (A x+b ) = 2A CA ∙
在网上看到有人贴了如下求导公式：
Y = A * X --> DY/DX = A'
Y = X * A --> DY/DX = A
Y = A' * X * B --> DY/DX = A * B'
Y = A' * X' * B --> DY/DX = B * A'
于是把以前学过的矩阵求导部分整理一下：
1. 矩阵Y 对标量x 求导：
相当于每个元素求导数后转置一下，注意M×N 矩阵求导后变成N×M 了
Y = [y(ij)] --> dY/dx = [dy(ji)/dx]
2. 标量y 对列向量X 求导：
注意与上面不同，这次括号内是求偏导，不转置，对N×1向量求导后还是N×1向量
y = f(x1,x2,..,xn) --> dy/dX = (Dy/Dx1,Dy/Dx2,..,Dy/Dxn)'
3. 行向量Y' 对列向量X 求导：
注意1×M 向量对N×1向量求导后是N×M 矩阵。
将Y 的每一列对X 求偏导，将各列构成一个矩阵。
重要结论：
dX'/dX = I
d(AX)'/dX = A'
4. 列向量Y 对行向量X’求导：
转化为行向量Y’对列向量X 的导数，然后转置。
注意M×1向量对1×N 向量求导结果为M×N 矩阵。
dY/dX' = (dY'/dX)'
5. 向量积对列向量X 求导运算法则：
注意与标量求导有点不同。
d(UV')/dX = (dU/dX)V' + U(dV'/dX)
d(U'V)/dX = (dU'/dX)V + (dV'/dX)U'
重要结论：
d(X'A)/dX = (dX'/dX)A + (dA/dX)X' = IA + 0X' = A
d(AX)/dX' = (d(X'A')/dX)' = (A')' = A
d(X'AX)/dX = (dX'/dX)AX + (d(AX)'/dX)X = AX + A'X
6. 矩阵Y 对列向量X 求导：
将Y 对X 的每一个分量求偏导，构成一个超向量。
注意该向量的每一个元素都是一个矩阵。
7. 矩阵积对列向量求导法则：
d(uV)/dX = (du/dX)V + u(dV/dX)
d(UV)/dX = (dU/dX)V + U(dV/dX)
重要结论：
d(X'A)/dX = (dX'/dX)A + X'(dA/dX) = IA + X'0 = A
8. 标量y 对矩阵X 的导数：
类似标量y 对列向量X 的导数，
把y 对每个X 的元素求偏导，不用转置。
dy/dX = [ Dy/Dx(ij) ]
重要结论：
y = U'XV = ΣΣu(i)x(ij)v(j) 于是 dy/dX = [u(i)v(j)] = UV'
y = U'X'XU 则 dy/dX = 2XUU'
y = (XU-V)'(XU-V) 则 dy/dX = d(U'X'XU - 2V'XU + V'V)/dX = 2XUU' - 2VU' + 0 = 2(XU-V)U'
9. 矩阵Y 对矩阵X 的导数：
将Y 的每个元素对X 求导，然后排在一起形成超级矩阵。
10. 乘积的导数
d(f*g)/dx=(df'/dx)g+(dg/dx)f'
结论
d(x'Ax)=(d(x'')/dx)Ax+(d(Ax)/dx)(x'')=Ax+A'x （注意：'' 是表示两次转置
Matrix Reference Manual
Matrix Calculus
Go to: Introduction , Notation , Index Contents
Notation
∙ Derivatives of Linear Products
∙ Derivatives of Quadratic Products ∙
Notation
∙
∙
∙
∙
∙ d/dx (y ) is a vector whose (i) element is dy(i)/dx d/dx (y) is a vector whose (i) element is dy/dx(i) d/dx (y T ) is a matrix whose (i,j) element is dy(j)/dx(i) d/dx (Y ) is a matrix whose (i,j) element is dy(i,j)/dx d/dX (y) is a matrix whose (i,j) element is dy/dx(i,j)
Note that the Hermitian transpose is not used because complex conjugates are not analytic.
In the expressions below matrices and vectors A , B , C do not depend on X .
Derivatives of Linear Products
∙
∙
∙
∙
∙ d/dx (AYB ) =A * d/dx (Y ) * B o d/dx (Ay ) =A * d/dx (y ) d/dx (x T A ) =A T o d/dx (x ) =I T T o d/dx (x a ) = d/dx (a x ) = a d/dX (a T Xb ) = ab T T T T T o d/dX (a Xa ) = d/dX (a X a ) = aa d/dX (a T X T b ) = ba T d/dx (YZ ) =Y * d/dx (Z ) + d/dx (Y ) * Z Derivatives of Quadratic Products
d/dx (Ax +b ) T C (D x+e ) = A T C(Dx+e) + D T C T (Ax+b)
T T o d/dx (x Cx ) = (C +C ) x
T [C: symmetric]: d/dx (x Cx ) = 2Cx
T d/dx (x x ) = 2x
T T T o d/dx (Ax +b ) (D x+e ) = A (Dx+e) + D (Ax+b)
T T d/dx (Ax +b ) (A x+b ) = 2A (Ax+b)
T T o [C: symmetric]: d/dx (Ax +b ) C (A x+b ) = 2A C(Ax+b)
T T T T ∙ d/dX (a X Xb ) = X(ab + ba)
T T T o d/dX (a X Xa ) = 2Xaa
T T T T T ∙ d/dX (a X CXb ) = C Xab + CXba
T T T T o d/dX (a X CXa ) = (C + C)Xaa
T T T o [C :Symmetric ] d/dX (a X CXa ) = 2CXaa
T T T ∙ d/dX ((Xa+b)C(Xa+b)) = (C+C)(Xa+b)a ∙
Derivatives of Cubic Products
∙ d/dx (x T Axx T ) = (A+AT )xx T +xT AxI
Derivatives of Inverses
∙ d/dx (Y -1) = -Y -1d/dx (Y ) Y -1
Derivative of Trace
Note: matrix dimensions must result in an n*n argument for tr(). ∙
∙
∙
∙
∙ d/dX (tr(X)) = I d/dX (tr(Xk ) ) =k (Xk -1) T d/dX (tr(AXk ) ) = SUM r=0:k-1(Xr AX k-r -1) T d/dX (tr(AX-1B) ) = -(X-1BAX -1) T -1-1-T T -T o d/dX (tr(AX) ) =d/dX (tr(XA) ) = -X A X d/dX (tr(AT XB T )) = d/dX (tr(BXT A )) = AB
o d/dX (tr(XAT ) ) = d/dX (tr(AT X) ) =d/dX (tr(XT A) ) = d/dX (tr(AXT ) ) = A
d/dX (tr(AXBXT ) ) = AT XB T + AXB
T T o d/dX (tr(XAX) ) = X(A+A)
T T T o d/dX (tr(XAX) ) = X(A+A)
T T o d/dX (tr(AXX) ) = (A+A)X
d/dX (tr(AXBX)) = AT X T B T + B T X T A T
[C:symmetric ] d/dX (tr((XT CX) -1A ) = d/dX (tr(A (XT CX) -1) = -(CX(XT CX) -1)(A+AT )(XT CX) -1
[B,C:symmetric ] d/dX (tr((XT CX) -1(XT BX) ) = d/dX (tr( (XT BX)(XT CX) -1) = -2(CX(XT CX) -1)X T BX(XT CX) -1 + 2BX(XT CX) -1
∙ ∙ ∙ ∙ ∙ ∙
Derivative of Determinant
Note: matrix dimensions must result in an n*n argument for det(). ∙
∙
∙
∙ d/dX (det(X)) = d/dX (det(XT ) ) = det(X)*X-T -T o d/dX (det(AXB)) = det(AXB)*X -T o d/dX (ln(det(AXB))) = X d/dX (det(Xk ) ) = k *det(Xk )*X-T k -T o d/dX (ln(det(X) )) = k X [Real ] d/dX (det(XT CX) ) = det(XT CX)*(C+CT )X(XT CX) -1 T T T -1o [C : Real,Symmetric ] d/dX (det(XCX) ) = 2det(XCX)* CX(XCX) [C : Real,Symmetricc ] d/dX (ln(det(XT CX)) ) = 2CX(XT CX) -1 Jacobian
If y is a function of x , then d y T /dx is the Jacobian matrix of y with respect to x .
Its determinant, |d y T /dx |, is the Jacobian of y with respect to x and represents the ratio of the hyper-volumes d y and d x . The Jacobian occurs when changing variables in an integration:
Integral(f(y )d y )=Integral(f(y (x )) |d y T /d x | dx ).
Hessian matrix
If f is a function of x then the symmetric matrix d2f/dx 2 = d /dx T (df/dx ) is the Hessian matrix of f(x ). A value of x for which df/dx = 0 corresponds to a minimum, maximum or saddle point according to whether the Hessian is positive definite, negative definite or indefinite.
d 2/dx 2 (a T x ) = 0
22T T T T ∙ d /dx (Ax +b ) C (D x+e ) = A CD + D C A
22T T o d /dx (x Cx ) = C +C
22T d /dx (x x ) = 2I
22T T T o d /dx (Ax +b ) (D x+e ) = A D + D A
22T T d /dx (Ax +b ) (A x+b ) = 2A A
22T T o [C: symmetric]: d /dx (Ax +b ) C (A x+b ) = 2A CA ∙