12  Finite Differences

Lecture 12

Published

March 23, 2026

Note

I encourage you to play around with the juptyer notebook for this lecture - you can copy the code with the this notebook button on the side of this page.

Warning

These notes are mainly a record of what we discussed and are not a substitute for attending the lectures and reading books! If anything is unclear/wrong, let me know and I will update the notes.

Tip

This chapter is mostly based on

  • Burden, Faires, and Burden (2015) Chapter 11,
  • Driscoll and Braun (2018) Chapter 10

12.1 Boundary Value Problems

We will consider the following abstract formulation of a boundary value problem. Suppose we have a differential operator \mathcal L, a “nice” domain \Omega \subset \mathbb R^d, and data f (source or sink) and v (boundary values). Seek u : \Omega \to \mathbb R such that

\begin{align} \mathcal L[u] &= f &&\text{in } \Omega \nonumber\\ u &= v &&\text{on } \partial \Omega . \tag{BVP} \end{align}

This is a problem with Dirichlet boundary conditions (the values of the solution are prescribed on the boundary).

Example 12.1 (Linear Advection-Diffusion-Reaction Equations) Consider

\begin{align} \mathcal L[u] := -\Delta u + p \cdot \nabla u + q u \end{align}

In the context of semiconductor physics, u(x) represents the concentration of charge carriers at a position x \in \Omega. Each of these terms relates to the physics in question:

  • -\Delta : diffusion; electrons move from areas of high concentration to low,
  • p \cdot \nabla u : advection (drift); the solution is driven with velocity p (due to e.g. an external electric field),
  • q u : reaction; electrons and electron holes are being created or destroyed.

The same equation also models e.g pollutant transport in a river, drug delivery in a blood vessel, asset price in finance, heat transfer in fluids, ……

Here, we solve this problem numerically in one spatial dimension:

function adr_1d(n, p, q, f)
    h = 1/n 
    x = h:h:(1-h) 
    
    # Finite Difference Coefficients
    # -u'' ≈ (-u_{i-1} + 2u_i - u_{i+1}) / h^2
    # pu'  ≈ p * (u_{i+1} - u_{i-1}) / (2h)  [Central Difference]
    
    d = (2/h^2 + q) * ones(n-1)
    d₋  = (-1/h^2 - p/(2h)) * ones(n-2)
    d₊  = (-1/h^2 + p/(2h)) * ones(n-2)

    A = Tridiagonal(d₋, d, d₊)
    b = f.(x)
    u = A \ b

    # Dirichlet boundary conditions
    u = [0.0, u..., 0.0]
    x = [0.0, x..., 1.0]

    return x, u
end

# u'' = diffusion
# p = advection
# q = reaction
# f = source 

f(x) = 1.0    

# diffusion
p, q = 0., 0.
x, u = adr_1d(100, p, q, f)
P1 = plot(x, u, title=L"Diffusion: $-u′′ + %$p u′ + %$q u = 1$", 
     xlabel=L"Position $x$", ylabel=L"Concentration $u$",
     label=L"u(x)", lw=2, m=1)
display(P1)

# Advection (drift)
p, q = 50., 0.
x, u = adr_1d(100, p, q, f)
P2 = plot(x, u, title=L"Advection: $-u′′ + %$p u′ + %$q u = 1$", 
     xlabel=L"Position $x$", ylabel=L"Concentration $u$",
     label=L"u(x)", lw=2, m=1)
display(P2)

# Reaction (decay)
p, q = 0., 100.
x, u = adr_1d(100, p, q, f)
P3 = plot(x, u, title=L"Reaction: $-u′′ + %$p u′ + %$q u = 1$", 
     xlabel=L"Position $x$", ylabel=L"Concentration $u$",
     label=L"u(x)", lw=2, m=1)
display(P3)

# Advection + Reaction
p, q = 100., 100.
x, u = adr_1d(100, p, q, f)
P4 = plot(x, u, title=L"$-u′′ + %$p u′ + %$q u = 1$", 
     xlabel=L"Position $x$", ylabel=L"Concentration $u$",
     label=L"u(x)", lw=2, m=1)
display(P4)

Aim of today: Understand how did we did this

12.2 Finite Differences

We numerically solved the following problem

\begin{align} -u''(x) + p u'(x) + q u(x) = f(x) \end{align}

by solving a linear system of equations (more generally, p and q can also be functions of x).

In order to do this, we needed to approximate the linear differential operators u \mapsto u' and u \mapsto u'' with matrices. Suppose u : [0,1] \to \mathbb R solves the equation above with the boundary conditions u(0) = u(1) = 0. We try to approximate the solution of this equation on the grid \{x_j\}_{j=0}^n where x_j = \frac{j}{n}. We will let h:= \frac{1}{n} be the mesh size and let u_j \in \mathbb R^{n+1} be an approximation to u(x_j). Since u(0) = u(1) = 0, we can simply set u_0 = u_n = 0 and we want to write an equation for the remaining variables u_1,\dots,u_{n-1}.

Exercise 12.1 Explain why

\begin{align} u'(x_j) \approx \frac{u(x_j+h) - u(x_j)}{h}. \end{align}

We may therefore define the forward difference

\begin{align} \partial^+u_j := \frac{u_{j+1} - u_j}{h} \tag{forwards} \end{align}

There are many other ways of approximating this derivative:

\begin{align} \partial^-u_j := \frac{u_{j} - u_{j-1}}{h} \tag{backwards}\\ \partial^0u_j := \frac{u_{j+1} - u_{j-1}}{2h} \tag{central} \end{align}

(these are backwards and central differences, resp.)

Exercise 12.2 Show that \partial^0 = \frac{1}{2}( \partial^- + \partial^+).

Definition 12.1 We say a finite difference is consistent if the error E(h) between the exact derivative and the finite difference goes to zero as h \to 0. Moreover, we say the finite difference has order of accuracy p if E(h)=O(h^p) as h \to 0.

Exercise 12.3 What is the order of accuracy of \partial^+, \partial^-, \partial^0?

  • Example 5.4.1 from Driscoll and Braun (2018),

What scale would you use to plot the error curves to see the order of accuracy of \partial^0 method?

f(x) = exp(x) * (1 + sin(x)) 
f′(x) = exp(x) * (1 +  sin(x) + cos(x) )  

FD1(h) = (f(h)-f(0))/h 
FD2(h) = (-f(2h)/2 + 2f(h) - 3f(0)/2)/h
CD2(h) = (f(h) - f(-h))/2h
CD4(h) = ( -f(2h)/12 + 2f(h)/3 - 2f(-h)/3 + f(-2h)/12 )/h

h = .01:.001:.1 
plot( h, abs.(FD1.(h).-f′(0)); xaxis=:log, yaxis=:log, m=1, label="FD1" )
plot!( h, @. abs(FD2(h).-f′(0)); m=1, label="FD2" )
plot!( h, @. abs(CD2(h).-f′(0)); m=1, label="CD2" ) 
plot!( h, @. abs(CD4(h).-f′(0)); m=1, label="CD4" ) 

plot!( h, 2h; linestyle=:dash, label=L"O(h)")
plot!( h, 2h.^2; linestyle=:dash, label=L"O(h^2)")
plot!( h, (1/2)h.^4; linestyle=:dash, label=L"O(h^4)")

Exercise 12.4 Given u_1,\dots,u_{n-1} (recall that we are considering u_0 = u_n = 0), one can approximate the derivative of u at x_j by computing \partial^- u_j, \partial^+ u_j, or \partial^0 u_j (there are of course other choices!). Write these operations as matrices so that u'(x_j) \approx (D^- u)_j, (D^+ u)_j, and (D^0 u)_j.

Hint, these differentiation matrices are (n-1)\times(n-1).

Second differences:

In your assignment you will show that

\begin{align} u''(x_j) \approx (Lu)_j := \frac{u_{j-1} - 2 u_j + u_{j+1}}{h^2} \end{align}

is a second-order accurate finite difference for approximating the second derivative of u.

Exercise 12.5 Show that (Lu)_j = \partial^+ \partial^-u_j = \partial^-\partial^+ u_j.

Here are some other examples:

f′′(x) = exp(x) * (1 + 2cos(x) )  

F2D1(h) = (f(0)-2f(h)+f(2h))/h^2 
F2D2(h) = (2f(0)-5f(h)+4f(2h)-f(3h))/h^2 
C2D(h) = (f(-h)-2f(0)+f(h))/h^2

h = .01:.001:.1 
plot( h, abs.(F2D1.(h).-f′′(0)); xaxis=:log, yaxis=:log, m=1, label="F2D1" )
plot!( h, abs.(F2D2.(h).-f′′(0)); xaxis=:log, yaxis=:log, m=1, label="F2D2" ) 
plot!( h, abs.(C2D.(h).-f′′(0)); xaxis=:log, yaxis=:log, m=1, label="C2D" ) 

plot!( h, 4h; linestyle=:dash, label=L"O(h)")
plot!( h, h.^2/4; linestyle=:dash, label=L"O(h^2)")

Therefore, by replacing the continuous problem -u'' + p u' + q u = f with the discrete problem

\begin{align} Au := (-L + p D^0 + q I) u = f \end{align}

where f_j := f(x_j), we can approximate the solution u on the grid \{x_j\} by solving a linear system of equations.

Exercise 12.6 Look back at the code above and convince yourself that this is what is going on!

TipTO-DO

Chapter 1 reading:

Chapter 2 reading:

  • Recap matrices: sections 7.1 - 7.2 of Burden, Faires, and Burden (2015)
  • Jacobi & Gauss-Seidel: section 7.3 Burden, Faires, and Burden (2015)
  • CG: section 7.6 Burden, Faires, and Burden (2015)

Chapter 3 reading:

  • Sections 11.1 - 11.2 of Burden, Faires, and Burden (2015): shooting methods
Burden, Richard L., Douglas J. Faires, and Annette M. Burden. 2015. Numerical Analysis. 10th ed. CENGAGE Learning.
Driscoll, Tobin A, and Richard J Braun. 2018. Fundamentals of Numerical Computation. https://fncbook.com/.