.md .pdf

repository open issue suggest edit

Contents

微分

有矩阵 $\mathbf{A} = [\mathbf{a}_1, \mathbf{a}_2, \cdots, \mathbf{a}_m]{\top} \in \mathbb{R}^{m \times n}$ 和向量 $\mathbf{x} = [x_1, x_2, \cdots, x_n]{\top} \in \mathbb{R}^{n}$，则有：

$$\begin{split} \mathbf{Ax} = \begin{bmatrix} \langle \mathbf{a}_1, \mathbf{x} \rangle\\ \langle \mathbf{a}_2, \mathbf{x} \rangle\\ \vdots \\ \langle \mathbf{a}_m, \mathbf{x} \rangle \end{bmatrix} \end{split}$$

向量值函数的导数

令 $\mathbf{f}: \mathbb{R}^n \rightarrow \mathbb{R}^m$，则梯度记作：

$$\nabla_{\mathbf{x}} \mathbf{f(x)} = \bigg[\frac{\partial \mathbf{f(x)}}{\partial x_1}, \frac{\partial \mathbf{f(x)}}{\partial x_2}, \ldots, \frac{\partial \mathbf{f(x)}}{\partial x_n}\bigg]$$

若有 $\mathbf{y} = [y_1, y_2, \ldots, y_m]^\top$，$x \in \mathbb{R}$，则：

$$\frac{\partial \mathbf{y}}{\partial x} = \bigg[\frac{\partial y_1}{\partial x}, \frac{\partial y_2}{\partial x}, \ldots, \frac{\partial y_m}{\partial x}\bigg]^\top$$

还有，

$$\begin{split} \frac{\partial \mathbf{y}}{\partial \mathbf{x}} = \bigg[\frac{\partial y_1}{\partial \mathbf{x}}, \frac{\partial y_2}{\partial \mathbf{x}}, \ldots, \frac{\partial y_m}{\partial \mathbf{x}}\bigg]^\top = \begin{bmatrix} \frac{\partial y_1}{\partial x_1} & \frac{\partial y_1}{\partial x_2} &\cdots &\frac{\partial y_1}{\partial x_n} \\ \frac{\partial y_2}{\partial x_1} & \frac{\partial y_2}{\partial x_2} & \cdots & \frac{\partial y_2}{\partial x_n}\\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial y_m}{\partial x_1} & \frac{\partial y_m}{\partial x_2} & \cdots & \frac{\partial y_m}{\partial x_n} \end{bmatrix} \end{split}$$

可以记作：$\mathbf{f^{'}(x)}$ 或者 $\mathbf{D} \mathbf{f(x)}$。容易推出：

$$\begin{split} \nabla_\mathbf{x} \mathbf{f(x)} = \mathbf{f^{'}(x)}^{\top}\\ \end{split}$$

例1 计算：$\nabla_{\mathbf{x}} ||\mathbf{x}||^2$

因为，

$$\begin{split} \begin{align} \mathbf{d} ||\mathbf{x}||^2 &= \mathbf{d} \langle \mathbf{x}, \mathbf{x} \rangle\\ &= \mathbf{d} \sum_{i=1}^n x_i^2\\ &= \sum_{i=1}^n \mathbf{d} x_i^2\\ &= 2 \sum_{i=1}^n x_i \mathbf{d} x_i\\ &= 2 \mathbf{x}^{\top} \mathbf{d} \mathbf{x}\\ \end{align} \end{split}$$

所以，$\nabla_{\mathbf{x}} ||\mathbf{x}||^2 = 2 \mathbf{x}$

拓展

可以借助内积运算简化求解过程，比如：

$$\begin{split} \begin{align} \mathbf{d} ||\mathbf{x}||^2 &= \langle \mathbf{d} \mathbf{x}, \mathbf{x} \rangle + \langle \mathbf{x}, \mathbf{d} \mathbf{x} \rangle\\ &= \langle 2 \mathbf{x}, \mathbf{d} \mathbf{x} \rangle\\ &= \langle \nabla_{\mathbf{x}} ||\mathbf{x}||^2, \mathbf{dx} \rangle \end{align} \end{split}$$

微分学（废弃）