Mathematics
Table of Contents
As the titile indicates, this post is written to summarize the basic of mathematics.
Special functions
Gamma function
\begin{align*} \Gamma(x) \triangleq \int_0^{+\infty} e^{-t} t^{x - 1} dt \end{align*}Particularly, \(\forall x \in \mathbb{Z}^+\), \(\Gamma(x) = (x-1)!\).
- Bessel functions
First kind
\begin{align*} J_n(x) \triangleq \frac{1}{2\pi}\int_{-\pi}^{\pi} e^{j(x\sin\theta-n\theta)}d\theta \end{align*}Particularly, \(\forall \phi \in [0, 2\pi)\), \(J_0(x) = \dfrac{1}{2\pi} \int_\phi^{2\pi + \phi}e^{\pm jx\cos\theta}d\theta = \dfrac{1}{2\pi} \int_\phi^{2\pi + \phi}e^{\pm jx\sin\theta}d\theta\).
- Second kind
Q-function
\begin{align*} Q(x) &\triangleq \frac{1}{\sqrt{2\pi}}\int_x^{+\infty}e^{-\frac{t^2}{2}} dt \\ &= 1 - Q(-x) \end{align*}Error function and complementary error function
\begin{align*} erf(x) &= \frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2} dt \\ &= 2Q(\sqrt{2}x) \\ erfc(x)&= \frac{2}{\sqrt{\pi}}\int_x^{+\infty} e^{-t^2} dt \\ &= 1 - erf(x) \\ &= 1 - 2Q(\sqrt{2}x) \end{align*}Euler
\begin{align*} e^{j\theta} &= \cos\theta + j\sin\theta \\ \cos\theta &= \frac{e^{j\theta} + e^{-j\theta}}{2} \\ \sin\theta &= \frac{e^{j\theta} - e^{-j\theta}}{2j} \end{align*}
Matrix
Vectorization
\(\forall \mathbf{A} \in \mathbb{C}^{p \times n}\), \(\mathbf{B} \in \mathbb{C}^{n \times m}\), \(\mathbf{C} \in \mathbb{C}^{m \times q}\),
\begin{align} \text{vec}(\mathbf{ABC}) = (\mathbf{C}^T \otimes \mathbf{A}) \text{vec}(\mathbf{B}). \label{eq:vect} \end{align}Particularly,
- By setting \(\mathbf{A} = \mathbf{I}_n\) and \(\mathbf{C} = \mathbf{I}_m\) respectively, \eqref{eq:vect} becomes
If \(\mathbf{B} = \text{diag}(\mathbf{b})\) is a diagonal matrix (\(m = n\)), \(\mathbf{b} \in \mathbb{C}^n\), \eqref{eq:vect} becomes
\begin{align*} \text{vec}(\mathbf{ABC}) = (\mathbf{C}^T \odot \mathbf{A})\mathbf{b}, \end{align*}where \(\odot\) is the Khatri-Rao product operator.
Khatri-Rao product
\(\forall \mathbf{A} = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots &\mathbf{a}_k \end{bmatrix} \in \mathbb{C}^{n \times k}\) and \(\mathbf{B} = \begin{bmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_k \end{bmatrix} \in \mathbb{C}^{m \times k}\), their Khatri-Rao product can be denoted by
\begin{align*} \mathbf{A} \odot \mathbf{B} = \begin{bmatrix} \mathbf{a}_1 \otimes \mathbf{b}_1 & \mathbf{a}_2 \otimes \mathbf{b}_2 & \cdots & \mathbf{a}_k \otimes \mathbf{b}_k \end{bmatrix} \in \mathbb{C}^{mn \times k}, \end{align*}where \(\mathbf{a}_i \otimes \mathbf{b}_i = \text{vec}\left(\mathbf{b}_i \mathbf{a}_i^T\right) \in \mathbb{C}^{nm}\), \(\mathbf{a}_i \in \mathbb{C}^n\), \(\mathbf{b}_i \in \mathbb{C}^m\), \(i = 1, 2, \ldots, k\).
Calculus
Differentiation
Integral
Newton-Leibniz
Symmetry
Particularly when \(a = -b\), we have
\begin{align*} \int_{-b}^b f(x) dx &= \int_0^b \left[ f(x) + f(-x) \right] dx \\ &= \begin{cases} 0, & f(x) = -f(-x); \\ 2\int_0^b f(x) dx, & f(x) = f(-x). \end{cases} \end{align*}Multiple integral
Double integral
For a 2D region \(D\) parameterized by \(\begin{cases} a \leq x \leq b \\ y_1(x) \leq y \leq y_2(x)\end{cases}\), if function \(f(x, y)\) is integrable on \(D\), then
\begin{align*} \iint_D f(x, y)dxdy = \int_a^b dx \int_{y_1(x)}^{y_2(x)} f(x, y) dy. \end{align*}For a 2D region \(D\) parameterized by \(\begin{cases} c \leq y \leq d \\ x_1(y) \leq x \leq x_2(y)\end{cases}\), if function \(f(x, y)\) is integrable on \(D\), then
\begin{align*} \iint_D f(x, y)dxdy = \int_c^d dy \int_{x_1(y)}^{x_2(y)} f(x, y) dx. \end{align*}Given a 2D region parameterized by \(\begin{cases} x = x(u, v) \\ y = y(u, v) \end{cases}\), if \(x(u,v)\) and \(y(u,v)\) are continuous and differentiable, and Jacobian determinant \(J = \left| \cfrac{D(x,y)}{D(u,v)} \right| = \begin{vmatrix} \frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v} \end{vmatrix} \neq 0\), then we have
\begin{align*} \iint_{D(x, y)} f(x,y)dxdy = \iint_{D(u, v)} f \left(x(u, v), y(u, v)\right) |J| dudv, \end{align*}where \(D(u, v)\) is an one-to-one mapping of \(D(x, y)\).
Particularly, for the transformation from Cartesian to polar coordinate systems, \(\begin{cases} x = r\cos\theta \\ y = r\sin\theta \end{cases}\), we have \(J = r\) and
\begin{align*} \iint_{D(x, y)} f(x,y)dxdy = \iint_{D(r, \theta)} f(r\cos\theta, r\sin\theta) rdrd\theta. \end{align*}
Triple integral
For a 3D region \(V\) parameterized by \(\begin{cases} a \leq x \leq b \\ y_1(x) \leq y \leq y_2(x) \\ z_1(x,y) \leq z \leq z_2(x,y)\end{cases}\), if function \(f(x, y, z)\) is integrable on \(V\), then
\begin{align*} \iiint_V f(x,y,z)dxdydz = \int_a^b dx \int_{y_1(x)}^{y_2(x)} dy \int_{z_1(x,y)}^{z_2(x,y)} f(x, y, z)dz. \end{align*}Given a 3D region parameterized by \(\begin{cases} x = x(u, v, w) \\ y = y(u, v, w) \\ z = z(u, v, w) \end{cases}\), if \(x(u,v,w)\), \(y(u,v,w)\), and \(z(u,v,w)\) are continuous and differentiable, and Jacobian determinant \(J = \left| \cfrac{D(x,y,z)}{D(u,v,w)} \right| = \begin{vmatrix} \frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} & \frac{\partial x}{\partial w} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v} & \frac{\partial y}{\partial w} \\ \frac{\partial z}{\partial u} & \frac{\partial z}{\partial v} & \frac{\partial z}{\partial w}\end{vmatrix} \neq 0\), then we have
\begin{align*} \iiint_{V(x, y, z)} f(x,y,z)dxdydz = \iiint_{V(u,v,w)} f \left( x(u,v,w), y(u,v,w), z(u,v,w) \right) |J| dudvdw, \end{align*}where \(V(u,v,w)\) is an one-to-one mapping of \(V(x,y,z)\).
Particularly, for the transformation from Cartesian to cylindrical coordinate systems, \(\begin{cases} x = r\cos\theta \\ y = r\sin\theta \end{cases}\), we have \(J = r\) and
\begin{align*} \iiint_{V(x, y, z)} f(x,y,z)dxdydz = \iiint_{V(r, \theta, z)} f(r\cos\theta, r\sin\theta, z) rdrd\theta dz. \end{align*}Particularly, for the transformation from Cartesian to spherical coordinate systems, \(\begin{cases} x = r\sin\theta\cos\phi \\ y = r\sin\theta\sin\phi \\ z = r\cos\theta\end{cases}\), we have \(J = r^2 \sin\theta\) and
\begin{align*} \iiint_{V(x, y, z)} f(x,y,z)dxdydz = \iiint_{V(r, \theta, \phi)} f(r\sin\theta\cos\phi, r\sin\theta\sin\phi, r\cos\theta) r^2\sin\theta drd\theta d\phi. \end{align*}
Line integral
A line integral means to integrate a scalar function or a vector field over a smooth curve in a plane or space. The former is termed scalar line integral or 1st type of line integral, while the latter is termed vector line integral or 2nd type of line integral.
Curve parameterization
Generally, a smooth curve can be parameterized by
\begin{align*} \mathbf{r}(t) = \begin{bmatrix} x(t) \\ y(t) \\ z(t) \end{bmatrix}, \quad a \le t \le b. \end{align*}Regular parameterization
\(\forall t\in [a, b]\), if \(\mathbf{r}^\prime(t) \neq \mathbf{0}\), the parameterization is regular or smooth. The regularity ensures \(\mathbf{r}\) is a curve, but not just a point.
Scalar line integral
Given a smooth curve \(C\) between \(A\) and \(B\) parameterized by \(\begin{cases}x = x(t)\\ y = y(t)\\ z = z(t)\end{cases}\), \(t_A \leq t \leq t_B\), if integrand \(f(x, y, z)\) is continuous over \(C\), then
\begin{align*} \int_Cf(x,y,z)ds = \int_{t_A}^{t_B} f(x(t), y(t), z(t)) \sqrt{x^{\prime 2}(t) + y^{\prime 2}(t) + z^{\prime 2}(t)} dt, \end{align*}where \(ds \triangleq \left\| \begin{bmatrix}dx \\ dy \\ dz\end{bmatrix}\right\| = \left\|\begin{bmatrix} x^{\prime}(t) \\ y^{\prime}(t) \\ z^{\prime}(t) \end{bmatrix}\right\|dt = \sqrt{x^{\prime 2}(t) + y^{\prime 2}(t) + z^{\prime 2}(t)} dt\) is the arc length unit.
For scalar line integral, neither the orientation nor the parameterization of the curve matters, i.e., \(\int_{\overset{\frown}{AB}} f(x,y,z)ds = \int_{\overset{\frown}{BA}} f(x,y,z)ds\).
Vector line integral
Given a smooth and oriented curve \(C\) from \(A\) to \(B\) parameterized by \(\begin{cases}x = x(t)\\ y = y(t)\\ z = z(t)\end{cases}\), if a vector field \(\mathbf{F} = \begin{bmatrix} P(x,y,z) \\ Q(x,y,z) \\ R(x,y,z)\end{bmatrix}\) is continuous over \(C\), then
\begin{align*} \int_C \mathbf{F} \cdot d\mathbf{r} &= \int_C Pdx + Qdy + Rdz = \int_{t_A}^{t_B} Px^{\prime}(t) + Qy^{\prime}(t) + Rz^{\prime}(t) dt \\ &= \int_C \mathbf{F} \cdot \mathbf{T} ds, \end{align*}where
- \(d\mathbf{r} \triangleq \begin{bmatrix} dx \\ dy \\ dz \end{bmatrix} = \mathbf{r}^{\prime}(t)dt = \mathbf{T} ds\) is the arc unit;
- \(\mathbf{r}^{\prime}(t) \triangleq \begin{bmatrix} x^{\prime}(t) \\ y^{\prime}(t) \\ z^{\prime}(t) \end{bmatrix}\);
- \(\mathbf{T} \triangleq \cfrac{\mathbf{r}^{\prime}(t)}{\|\mathbf{r}^{\prime}(t)\|}\) is the unit tangent vector along \(C\);
- \(ds \triangleq \|d\mathbf{r}\| = \|\mathbf{r}^{\prime}(t)\| dt\) is the arc length unit.
For vector line integral, the orientation of the curve does matter, i.e., \(\int_{\overset{\frown}{AB}} \mathbf{F} \cdot d\mathbf{r} = -\int_{\overset{\frown}{BA}} \mathbf{F} \cdot d\mathbf{r}\).
Orientation
For a scalar line integral, neither the orientation nor the parameterization of a curve matters. So long as the curve is traversed once by the parameterization, the value of the line integral is unchanged. Differently, the orientation of a curve indeed matters for a vector line integral.
For an oriented curve \(C\) and a same curve but with reversed orientation \(-C\), following equation holds.
\begin{align*} \int_{-C} \mathbf{F} \cdot d\mathbf{r} = -\int_C \mathbf{F} \cdot d\mathbf{r}. \end{align*}Surface integral
A surface integral means to integrate a scalar function or a vector field over a smooth surface in space. A surface integral is very similar to a line integral, and the only difference lies the integration is performed over a surface rather than a path. If the integrand is a scalar function, the integral is termed scalar surface integral or 1st type of surface integral; if the integrand is a vector field, the integral is termed vector surface integral or 2nd type of surface integral.
Surface parameterization
Generally, a surface can be parameterized by
\begin{align*} \mathbf{r}(u,v) = \begin{bmatrix} x(u,v) \\ y(u,v) \\ z(u,v) \end{bmatrix}, \end{align*}with 2 parameters \(u\) and \(v\) involved since a surface is 2 dimensional.
For example, a sphere of radius \(\rho\) centered at the origin can be parameterized by
\begin{align*} \mathbf{r}(\theta, \phi) = \begin{bmatrix} \rho\sin\theta\cos\phi \\ \rho\sin\theta\sin\phi \\ \rho\cos\theta\end{bmatrix}; \quad 0 \le \theta \le \pi, 0 \le \phi \le 2\pi. \end{align*}Regular parameterization
\(\forall(u, v)\), if cross product \(\mathbf{r}_u \times \mathbf{r}_v \neq \mathbf{0}\), the parameterization is regular or smooth. Similarly, the regularity ensures \(\mathbf{r}(u,v)\) is a surface, but not just a curve or even point.
Scalar surface integral
Given a smooth surface \(S\), if integrand \(f(x, y, z)\) is continuous over \(S\), then
\begin{align*} \iint_Sf(x, y, z)dS &= \iint_{D(u,v)}f(x(u,v),y(u,v),z(u,v))\|\mathbf{t}_u \times \mathbf{t}_v\|dudv \\ &= \iint_{D(x,y)}f(x, y, z)\sqrt{1 + z_x^2 + z_y^2}dxdy \\ &= \iint_{D(y,z)}f(x, y, z)\sqrt{1 + x_y^2 + x_z^2}dydz \\ &= \iint_{D(x,z)}f(x, y, z)\sqrt{1 + y_x^2 + y_z^2}dxdz, \end{align*}where
- Tangent vectors \(\mathbf{t}_u \triangleq \begin{bmatrix} \cfrac{\partial x}{\partial u}, \cfrac{\partial y}{\partial u}, \cfrac{\partial z}{\partial u} \end{bmatrix}\) and \(\mathbf{t}_v \triangleq \begin{bmatrix} \cfrac{\partial x}{\partial v}, \cfrac{\partial y}{\partial v}, \cfrac{\partial z}{\partial v} \end{bmatrix}\) span the tangent plane.
- \(dS \triangleq \left\|\mathbf{t}_u \times \mathbf{t}_v \right\|dudv = \sqrt{1 + z_x^2 + z_y^2}dxdy = \sqrt{1 + x_y^2 + x_z^2}dydz = \sqrt{1 + y_x^2 + y_z^2}dxdz\) is the surface area unit.
Vector surface integral
Given a smooth surface \(S\), if a vector field \(\mathbf{F} = \begin{bmatrix} P(x,y,z) \\ Q(x,y,z) \\ R(x,y,z)\end{bmatrix}\) is continuous over \(S\), then,
\begin{align*} \iint_S \mathbf{F} \cdot d\mathbf{S} &= \iint_S Pdydz + Qdxdz + Rdxdy = \iint_S P\cos\alpha + Q\cos\beta + R\cos\gamma dS \\ &= \iint_S \mathbf{F} \cdot \mathbf{n} dS, \end{align*}where
- \(d\mathbf{S} \triangleq \begin{bmatrix}dydz \\ dxdz \\ dxdy\end{bmatrix} = \mathbf{n}dS\) is the surface unit.
- \(\mathbf{n} = \begin{bmatrix}\cos\alpha \\ \cos\beta \\ \cos\gamma \end{bmatrix}\) is the unit normal vector; \(\cos\alpha\), \(\cos\beta\), \(\cos\gamma\) are direction cosines; \(\alpha\), \(\beta\), \(\gamma\) are the angles relative to \(x\), \(y\), \(z\) axes respectively.
- \(dS \triangleq \|d\mathbf{S}\| = \left\| \begin{bmatrix}dydz \\ dxdz \\ dxdy\end{bmatrix}\right\|\) is the surface area unit.
Orientation
Similar to vector line integrals, we need to define an oriented surface for a surface integral of a vector field. The orientation means upward/downward for an oriented surface, or outward/inward for a sphere or a cylinder.
For a closed surface, e.g., sphere, we define outward unit normal vector as positive orientation; while for a portion of a graph, we define upward unit normal vector as positive orientation.
Maximum and minimum values
For a differentiable function \(f: \mathbb{R}^{n} \to \mathbb{R}\), zero gradient is the necessary condition of an extreme point.
- If Hessian matrix at an extreme point \(\mathbf{x}_0\) is positive definite, i.e., \(\nabla f(\mathbf{x}_0) = \mathbf{0}\), \(\mathbf{H}(\mathbf{x}_0) \succ 0\), the extreme point corresponds with a local minimum.
- If Hessian matrix at an extreme point \(\mathbf{x}_0\) is negative definite, i.e., \(\nabla f(\mathbf{x}_0) = \mathbf{0}\), \(\mathbf{H}(\mathbf{x}_0) \prec 0\), the extreme point corresponds with a local maximum.
Transform
Fourier
Laplace
Series
Taylor
For function \(f(x)\), if \(f^{(n)}(x_0)\) exists, its Taylor expansion can be written as
\begin{align*} f(x) = f(x_0) + f^{\prime}(x_0)(x - x_0) + \cdots + \frac{f^{(n)}(x_0)}{n!}(x - x_0)^n + o\left( (x - x_0)^n\right). \end{align*}When \(n \to \infty\), Taylor expansion above becomes power series.
\begin{align*} f(x) = f(x_0) + f^{\prime}(x_0)(x - x_0) + \cdots + \frac{f^{(n)}(x_0)}{n!}(x - x_0)^n + \cdots. \end{align*}The power series in the right side is termed Taylor series. Particularly, if \(x_0 = 0\), the series is termed Maclaurin series, i.e.,
\begin{align*} f(x) = f(0) + f^{\prime}(0)x + \cdots + \frac{f^{(n)}(0)}{n!}x^n + \cdots. \end{align*}Examples
\begin{align*} e^x &= \sum_{n=0}^{\infty} \frac{x^n}{n!} = 1 + x + \frac{x^2}{2} + \cdots; \quad \mathrm{ROC}: (-\infty, +\infty) \\ \sin x &= \sum_{n=0}^{\infty} \frac{(-1)^n x^{2n+1}}{(2n+1)!} = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \cdots; \quad \mathrm{ROC}: (-\infty, +\infty) \\ \cos x &= \sum_{n=0}^{\infty} \frac{(-1)^n x^{2n}}{(2n)!} = 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \cdots; \quad \mathrm{ROC}: (-\infty, +\infty) \\ (1+x)^a &= \sum_{n=0}^{\infty} C_a^nx^n = 1 + ax + \frac{a(a-1)}{2}x^2 + \cdots; \quad \mathrm{ROC}: (-1, 1) \\ \ln(1+x) &= \sum_{n=0}^{\infty} \frac{(-1)^n x^{n+1}}{n+1} = x - \frac{x^2}{2} + \frac{x^3}{3} - \cdots; \quad \mathrm{ROC}: (-1, 1] \\ \arctan x &= \sum_{n=0}^{\infty} \frac{(-1)^n x^{2n+1}}{2n+1} = x - \frac{x^3}{3} + \frac{x^5}{5} - \cdots; \quad \mathrm{ROC}: [-1, 1] \end{align*}