An important kind of equation to learn is those linear equations, which are those can be written in the form of

for some linear differential operator $L$ and unknown $y$. We begin by the example of second order homogeneous linear equations.

Second Order Linear Equations

Given a linear, second order, homogeneous ODE

Now, we are going to show the following two things:

1. If $y_1,y_2$ are solution to (1), then $y=c_1y_1+c_2y_2$ is also a solution to (1) for some constants $c_1,c_2\in \mathbb{R}$.

2. If $y_1$ and $y_2$ are two linearly independent solutions to (1), then all solution to (1) have the form $y=c_1y_1+c_2y_2$ for some constants $c_1,c_2\in\mathbb{R}$.

The first one relies on the linearity of differential operator

We can show that this operator is linear by observing that

• $L\left(cy\right)=cL\left(y\right)$ for any constant $c$.

• $L\left(y_1+y_2\right)=L\left(y_1\right)+L\left(y_2\right)$.

Then, for any constant $c_1,c_2$, we have

and the first one is proved.

The second one relies the Picard-Lindelof theorem also known as existence and uniqueness theorem:

Let $x_0\in\Omega$, where $\Omega\subset\mathbb{R}^n$ is open and let $t_0\in I$, where $I\subset\mathbb{R}$ be an interval.

Suppose $F:\Omega\times I\to\mathbb{R}^n$ is continuous in $t$ and Lipschitz continuous in $x$, i.e. there exist a constant $L>0$, such that for all $x,y\in\Omega$ and all $t\in I$,

Then, the initial value problem

has one and only one solution in some $t$ interval containing $t_0$.

An outline of the proof is as follows:

We can always construct a solution by using an iterative approximation method known as Picard iteration, which works as follows:

First, a solution to (2) will have to satisfy the integral equation

Then, by setting $x^{(0)}\left(t\right)=x_0$, and

yields a sequence of function $x^{(k)}\left(t\right)$. Then, under some condition on function $F$, we can use Contraction Mapping Principle to show that the sequence converges to a unique function $x\left(t\right)$.

Algebraic Structure of Linear Equation

An interesting question to consider in the proof is that why is there exactly two independent solutions to a second linear order equation? A simple answer to this might be that since it contain second order derivative, when we solve it we shall integrate twice and each gives an integrating constant $c$ which makes there are two constants $c_1,c_2$ to determine.

In fact, if we apply some knowledge from linear algebra, we may easy to think of the solutions as a vector space, known as solution space which happens to be 2 dimensional. And the general case would be, for a linear equation of order $n$, the dimension of its solution space is exactly $n$. Now, we want to prove this.

First notice that given an explicit ODE of order $n$,

and set new variables

Equation (3) can be written as a linear system of equations of order 1,

or, simply denoted as

Therefore, for any linear equation of order $n$, it is always equivalent to a system of $n$ first order linear equation.

Thence, we only need to consider the first order linear system of equations,

where $\boldsymbol{b}:\mathbb{R}\to\mathbb{R}^n$, and $\boldsymbol{A}:\mathbb{R}\to\mathbb{R}^n\times\mathbb{R}^n$ .

Consider the associated homogeneous equation

Notice that given any two solution $\boldsymbol{x}_1,\boldsymbol{x}_2$, and any number $\lambda,\mu\in\mathbb{R}$, their linear combination $\lambda\boldsymbol{x}_1+\mu\boldsymbol{x}_2$ is also a solution.

Suppose that a basis of vectors $\mathcal{B}=\left\{\boldsymbol{b}_1,\cdots,\boldsymbol{b}_n\right\}$, of $\mathbb{R}^n$ is given and fix $t_0\in \mathbb{R}$. Denote by $\boldsymbol{x}_k\left(t\right)$ the unique solution (theorem of Picard-Lindelof) to the initial value problem

and denote

Then, by superposition principle, any element of $\mathrm{span}\mathcal{F}$ is a solution to (4)

Conversely, let $\boldsymbol{x}$ be any solution to (4), and suppose that $\boldsymbol{x}\left(t_0\right)=\boldsymbol{x}_0\in\mathbb{R}^n$. We have

for some numbers $\lambda_1,\dots,\lambda_n\in\mathbb{R}$. Therefore, by theorem of Picard-Lindelof,

This shows that the solution space is given by $\mathrm{span}\mathcal{F}$.

In addition, we have,

which means $\mathcal{F}$ is a basis.

Now, let’s explain why a second order linear equation will have exactly two independent solutions. For equation (1), let $x_1=y,x_2=y'$, and denote $\boldsymbol{x}=\left(x_1,x_2\right)$, it is equivalent to

which is a first order linear system of equation in two dimension.

Solution to Linear System of Equations

Homogeneous, Constant Coefficients

First, consider the most basic case of linear system of equations, that is a homogeneous one with constant coefficients:

where $\boldsymbol{x}:\mathbb{R}\to\mathbb{R}^n$ and $\boldsymbol{A}\in \mathcal{M}\left(n\times n;\mathbb{R}\right)$.

With proper definition, it can be shown that the solution to the initial value problem (6) is

where

Now, the problem is how to calculate this matrix exponential. Recall from linear algebra that diagonal matrix, or Jordan matrix, are relative easy to calculate the matrix exponential. Fortunately, we have the Jordan Normal Form theorem which states that every complex matrix is similar to a Jordan matrix, i.e., for any matrix $\boldsymbol{A}$, there exists a matrix $\boldsymbol{U}$ and a Jordan matrix $\boldsymbol{J}$ such that

Then, we can easily see that

and then

The problem now becomes calculating $\displaystyle e^{\boldsymbol{J}t}$.

Let

be a Jordan matrix, where $\boldsymbol{J}_{k_i}\left(\lambda_i\right)$ is a Jordan block of size $k_i$ and diagonal element $\lambda_i$, i.e.,

It’s easy to show that

Hence, it’s sufficient to calculate the exponential of Jordan blocks. Notice that

where $\boldsymbol{N}$ is some nilpotent matrix whose power will vanish after finite many times, and that unit matrix commutes with any other matrix in terms of multiplication, i.e., $\boldsymbol{I}\boldsymbol{N}=\boldsymbol{N}\boldsymbol{I}$. Therefore,

However, the above calculation seems too complicated that it would be impractical. If we trace back what we are actually doing, we know that we are actually finding a basis of the solution space.

From previous discussion of the structure of the solution, we know immediately that for any basis $\mathcal{B}=\left\{\boldsymbol{b}_1,\cdots,\boldsymbol{b}_n\right\}$ of $\mathbb{R}^n$, the system of functions $\mathcal{F}=\left\{e^{\boldsymbol{A}t}\boldsymbol{b}_1,\cdots,e^{\boldsymbol{A}t}\boldsymbol{b}_n\right\}$ is a fundamental system of solution. We collect them into one matrix and denoted as $\boldsymbol{X}$, and is also called the fundamental matrix.

It turns out that if we choose the basis wisely, we don’t even need to calculate the whole $e^{\boldsymbol{A}t}$ to get a fundamental matrix. From theory of Jordan normal form, we know that there exists a basis consists of (generalized) eigenvectors $\boldsymbol{U}=\left\{\boldsymbol{u}_1,\cdots, \boldsymbol{u}_n\right\}$, such that the Jordan matrix of $\boldsymbol{A}$ is given by $\boldsymbol{J}=\boldsymbol{U}^{-1}\boldsymbol{AU}$. Using this basis, we have the fundamental matrix simply given by

The columns of $\boldsymbol{X}$ will therefore be the fundamental system of solutions.

Inhomogeneous, Constant Coefficients

Next, we solve an inhomogeneous equation with constant coefficients. The general solution is the superposition of a particular solution and the solution to the associated homogeneous equation.

Consider the following linear system of differential equation

suppose $\mathcal{F}=\left\{\boldsymbol{x}_1,\cdots,\boldsymbol{x}_n\right\}$ is a fundamental system for the associated homogeneous equation, then the general solution is given by

Variable Coefficients

There is no general method of solving homogeneous equations with variable coefficients,

However, if we are given a fundamental system of solution $\mathcal{F}=\left\{\boldsymbol{x}_1,\cdots,\boldsymbol{x}_n\right\}$, then we can find a particular solution to the inhomogeneous equation

by method of variation of parameter.

Let

and plug into equation (7), and get

leading to

Let $\boldsymbol{X}\left(t\right)=\left(\boldsymbol{x}_1\left(t\right),\cdots,\boldsymbol{x}_n\left(t\right)\right)$ be the fundamental matrix, and $\displaystyle \boldsymbol{c}\left( t \right) =\left( \begin{array}{c} c_1\left( t \right)\\ c_2\left( t \right)\\ \vdots\\ c_n\left( t \right)\\ \end{array} \right)$ then the previous equation is indeed the following linear system of equation

Using Cramer’s rule, we can find the solution to $c_k'\left(t\right)$.

where

is called the Wronskian.