Section 4.5 The Arzelà–Ascoli Theorem
Next we examine what the concept of relative compactness means in the space \(C_\bdd(X)\text{.}\) The following notions will be essential for this purpose.
Definition 4.28.
Let \((X,d)\) be a metric space and \(x_0 \in X\text{.}\) Let \(F\) be a set of functions \(f \maps X \to \R\text{.}\) We say that \(F\) is equicontinuous at \(x_0\) if
\begin{gather*}
\forall \varepsilon \gt 0
\,
\exists \delta \gt 0
\st
\forall x \in X
\,
\forall f \in F
,\\
d(x, x_0) \lt \delta \implies |f(x) - f(x_0)| \lt \varepsilon\text{.}
\end{gather*}
We say that \(F\) is equicontinuous if it is equicontinuous at every point of \(X\text{.}\)
This definition is quite analogous to continuity, but now \(\delta\) must not depend on the choice of \(f\) in \(F\text{.}\) For comparison, the statement ‘all functions in \(F\) are continuous at \(x_0\)’ means that
\begin{gather*}
\forall f \in F
\,
\forall \varepsilon \gt 0
\,
\exists \delta \gt 0
\st
\forall x \in X,\\
d(x, x_0) \lt \delta \implies |f(x) - f(x_0)| \lt \varepsilon\text{.}
\end{gather*}
Theorem 4.29. Arzelà–Ascoli.
Let \((X,d)\) be a compact metric space. A set of \(F\) of functions in \(C_\bdd(X)\) is relatively compact if and only if it is bounded and equicontinuous.
Proof.
Suppose that \(F\) is relatively compact. Then it is totally bounded by Theorem 4.27. In particular it is bounded by Lemma 4.9.
In order to prove that \(F\) is equicontinuous, let \(x_0 \in X\) and \(\varepsilon \gt 0\text{.}\) Using the fact that \(F\) is totally bounded, we find \(g_1, \dotsc, g_N \in C_\bdd(X)\) such that the balls in \(C_\bdd(X)\) of radius \(\frac{\varepsilon}{3}\) centred at \(g_1, \dotsc, g_N\) cover \(F\text{.}\) That is, for every \(f \in F\) there exists \(n \in \{1, \dotsc, N\}\) such that
\begin{equation*}
\|f - g_n\|_{\sup}\le \frac{\varepsilon}{3}.
\end{equation*}
Since every \(g_n\) is continuous, there exists \(\delta \gt 0\) such that \(|g_n(x) - g_n(x_0)| \lt \frac{\varepsilon}{3}\) for \(n=1, \dotsc, N\) and for all \(x \in X\) with \(d(x, x_0) \lt \delta\text{.}\) Now let \(f \in F\) be arbitrary. Choose \(n \in \{1, \dotsc, N\}\) such that \(\|f - g_n\|_{\sup}\lt \frac{\varepsilon}{3}\text{.}\) Then for \(x \in X\) with \(d(x, x_0) \lt \delta\text{,}\) we have the inequality
\begin{equation*}
\begin{split}
|f(x) - f(x_0)| \amp \le |f(x) - g_n(x)| + |g_n(x) - g_n(x_0)| + |g_n(x_0) - f(x_0)| \\
\amp \lt \frac{\varepsilon}{3} + \frac{\varepsilon}{3} + \frac{\varepsilon}{3} = \varepsilon.
\end{split}
\end{equation*}
Therefore, \(F\) is equicontinuous.
Conversely, suppose that \(F\) is bounded and equicontinuous. We first want to show that it is totally bounded. To this end, fix \(r >0\text{.}\)
For any \(p \in X\text{,}\) there exists \(\delta_p \gt 0\) such that for every \(x \in X\) with \(d(x, p) \lt \delta_p\) and for every \(f \in F\text{,}\) we have the inequality
\begin{equation*}
|f(x) - f(p)| \lt \frac{r}{4}.
\end{equation*}
(This is because \(F\) is equicontinuous at \(p\text{.}\)) Now consider the balls \(B_{\delta_p}(p)\) in \(X\) for all \(p \in X\text{.}\) Collectively they form an open cover of \(X\text{.}\) Since \(X\) is compact, there exists a finite subcover. That is, there exist \(p_1, \dotsc, p_N \in X\) such that
\begin{equation*}
X = \bigcup_{n=1}^N B_{\delta_{p_n}}(p_n).
\end{equation*}
Consider the set \(\Phi \subseteq \R^N\) defined as follows:
\begin{equation*}
\Phi = \set{(f(p_1), \dotsc, f(p_N))}{f \in F}.
\end{equation*}
Because \(F\) is bounded, there exists \(R \gt 0\) such that \(\n f_{\sup}\le R\) for all \(f \in F\text{.}\) Hence
\begin{equation*}
\n{(f(p_1), \dotsc, f(p_N))}_{\R^N} \le \sqrt{R^2 + \cdots + R^2} = R \sqrt{N}
\end{equation*}
for every \(f \in F\text{.}\) It follows that \(\Phi\) is a bounded set in \(\R^N\text{.}\) By Proposition 4.12, this implies that \(\Phi\) is totally bounded. Thus there exist \(\phi_1, \dotsc \phi_M \in \Phi\) such that for all \(\phi \in \Phi\text{,}\) there exists \(m \in \{1, \dotsc, M\}\) with \(|\phi - \phi_m| \lt \frac{r}{4}\text{.}\) The same statement may be formulated in terms of \(F\text{:}\) there exist \(f_1, \dotsc, f_M \in F\) such that for all \(f \in F\text{,}\) there exists \(m \in \{1, \dotsc, M\}\) with
\begin{equation}
\sqrt{(f(p_1) - f_m(p_1))^2 + \cdots + (f(p_N) - f_m(p_N))^2} \lt \frac{r}{4}.\tag{4.1}
\end{equation}
Now let \(f \in F\) be arbitrary and choose \(m \in \{1, \dotsc, M\}\) such that (4.1) holds true. Let \(x \in X\) and choose \(n \in \{1, \dotsc, N\}\) with \(d(x, p_n) \lt \delta_{p_n}\text{.}\) Then
\begin{equation*}
\begin{split}
|f(x) - f_m(x)| \amp \le |f(x) - f(p_n)| + |f(p_n) - f_m(p_n)| + |f_m(p_n) - f_m(x)| \\
\amp \lt \frac{r}{4} + \frac{r}{4} + \frac{r}{4} = \frac{3r}{4}.
\end{split}
\end{equation*}
Hence \(\|f - f_m\|_{\sup} \lt r\text{,}\) which shows that \(F\) is totally bounded.
The space \(C_\bdd(X)\) is complete by Theorem 1.50. Theorem 4.27 therefore implies that \(F\) is relatively compact.
Corollary 4.30.
For a compact metric space \((X,d)\text{,}\) a set in \(C_\bdd(X)\) is compact if and only if it is closed, bounded, and equicontinuous.
Proof.
This is a direct consequence of Theorem 4.29 and Theorem 4.5.
Corollary 4.31.
Let \(a \lt b\) and \(\alpha \in (0, 1]\text{.}\) If a set \(S \subset C^{0, \alpha}([a,b])\) is bounded, then it is relatively compact as a subset of \(C^0([a,b])\text{.}\)
Proof.
A bound in \(C^{0, \alpha}([a,b])\) implies equicontinuity, and then the statement follows from Theorem 4.29. The details are requested in Exercise 4.5.2.
Exercises Exercises
1. (PS9) Finite sets are equicontinuous.
Let \((X,d)\) be a metric space and let \(F = \{f_1,\ldots,f_N\}
\subseteq C_\bdd(X)\) be a finite set of continuous functions. Show that \(F\) is equicontinuous.
Hint.
Let \(x_0 \in X\) and \(\varepsilon \gt 0\text{.}\) Use continuity of each \(f_n\) at \(x_0\) to find a \(\delta_n \gt 0\text{.}\) Now choose \(\delta \gt 0\) appropriately in terms of \(\delta_1,\ldots,\delta_n\text{.}\)
Solution.
Let \(x_0 \in X\) and \(\varepsilon \gt 0\text{.}\) For each \(n \in
\{1,\ldots,N\}\text{,}\) \(f_n\) is continuous at \(x_0\text{,}\) and so there exists a \(\delta_n \gt 0\) such that, for all \(x \in X\text{,}\)
\begin{equation}
d(x,x_0) \lt \delta_n \implies \abs{f_n(x)-f_n(x_0)} \lt \varepsilon\text{.}\tag{4.2}
\end{equation}
Set \(\delta = \min\{\delta_1,\ldots,\delta_N\} \gt 0\text{.}\) Then for any \(f = f_n \in F\) and \(x \in X\) with \(d(x,x_0) \lt
\delta \le \delta_n\text{,}\) we can apply (4.2) to get \(\abs{f_n(x)-f_n(x_0)} \lt \varepsilon \text{.}\)
Comment.
In the argument above, we wanted \(d(x,x_0) \lt \delta\) to allow us to apply (4.2). Thus we wanted \(d(x,x_0) \lt \delta\) to imply, for any \(n \in \{1,\ldots,N\}\text{,}\) that \(d(x,x_0) \lt \delta_n\text{.}\) Clearly this holds if \(\delta \le \delta_n\) for every \(n\text{,}\) which we can ensure by setting \(\delta=\min\{\delta_1,\ldots,\delta_N\}\text{.}\) If we used \(\max\{\delta_1,\ldots,\delta_N\}\) instead, as several students tried to, then this wouldn’t work. In retrospect, these students would probably have benefited from writing out this part of their argument in more detail, to see where the choice of \(\delta\) really matters.
2. (PS10) Bounded subsets of \(C^{0,\alpha}\) relatively compact in \(C^0\).
Let \(\alpha \in (0,1]\) and suppose that \(S \subset C^{0,\alpha}([a,b])\) is a bounded (as a subset of \(C^{0,\alpha}([a,b])\)).
(a)
Show that \(S\) is bounded as a subset of \(C^0([a,b])\text{.}\)
Hint.
This is easy. Since \(C^0([a,b])\) is both a metric space and a normed space, \(S\) being bounded can be defined using either the first or the last bullet point in Definition 1.15.
Solution.
Since \(C^0([a,b])\) and \(C^{0,\alpha}([a,b])\) are normed spaces, we use the definition of bounded from the last bullet point in Definition 1.15. Let \(f \in S\text{.}\) Then
\begin{align*}
\n{f}_{C^{0,\alpha}([a,b])}
\amp =
\n{f}_{C^0([a,b])} + [f]_{C^{0,\alpha}([a,b])}\\
\amp \ge \n{f}_{C^0([a,b])}\text{.}
\end{align*}
Taking a supremum, we conclude that
\begin{gather*}
\sup_{f \in S} \n{f}_{C^0([a,b])} \le
\sup_{f \in S} \n{f}_{C^{0,\alpha}([a,b])} \lt \infty
\end{gather*}
where the second supremum is finite by the assumption that \(S\) is bounded in \(C^{0,\alpha}([a,b])\text{.}\)
(b)
Show that \(S\) is equicontinuous.
Hint 1.
Make sure that your proof that \(S\) is equicontinuous cannot be mistaken for a ‘proof’ that every \(f \in S\) is continuous.
Hint 2.
Let \(f \in S\) and \(x,x_0 \in [a,b]\) with \(x \ne x_0\text{.}\) Now estimate \(\abs{f(x)-f(x_0)}\) in terms of \([f]_{C^{0,\alpha}([a,b])}\text{.}\) Show that, since \(S\) is bounded in \(C^{0,\alpha}([a,b])\text{,}\) this implies equicontinuity.
Solution.
Since \(S\) is bounded in \(C^{0,\alpha}([a,b])\text{,}\) there exists \(M \ge 0\) such that \(\n f_{C^{0,\alpha}([a,b])} \le M\) for all \(f \in S\text{.}\)
For any \(x,x_0 \in [a,b]\) and \(f \in S\text{,}\) we can estimate
\begin{align}
\abs{f(x_0) - f(x)}
\amp \le [f]_{C^{0,\alpha}([a,b])} \abs{x - x_0}^\alpha \notag\\
\amp \le M \abs{x -x_0}^\alpha\text{.}\tag{✶}
\end{align}
To see that this implies equicontinuity, fix \(x_0 \in [a,b]\) and \(\varepsilon \gt
0\text{,}\) and set
\begin{equation*}
\delta = \Big(\frac \varepsilon M\Big)^{1/\alpha}\text{.}
\end{equation*}
Then for any \(x \in [a,b]\) with \(\abs{x-x_0} \lt \delta\text{,}\) and for any \(f \in S\text{,}\) (✶) implies
\begin{equation*}
\abs{f(x_0) - f(x)} \le M \abs{x -x_0}^\alpha \lt \varepsilon\text{.}
\end{equation*}
Comment.
When showing that a set \(S\) is equicontinuous, your main goal is convince your reader that your choice of \(\delta\) does not depend on which \(f \in S\) you chose. Otherwise you are simply showing that every function \(f \in S\) is continuous, which is a much weaker statement; see the discussion below Definition 4.28.
(c)
Conclude that \(S\) is relatively compact as a subset of \(C^0([a,b])\text{.}\) What does this mean about sequences of functions \(\seq fn\) in \(S\text{?}\)
Solution.
Relative compactness now follows at once from the previous two parts and Arzelà–Ascoli. By Exercise 4.4.1, this means that any sequence \(\seq fn\) in \(S\) has a subsequence \(\subseq fnk\) which converges in \(C^0([a,b])\text{.}\) That is, there exists \(f_0 \in C^0([a,b])\) such that \(\n{f_{n_k}-f_0}_{C^0([a,b])} \to 0\) as \(k \to \infty\text{.}\)
3. (PS10) A set which is not equicontinuous.
Show that the set \(F\) from Exercise 4.3.5 is not equicontinuous
(a)
directly using Definition 4.28
Hint 1.
If you are at all uncertain about your ability to correctly negate the statement ‘\(F\) is equicontinuous’ on the fly, then work out the negation very carefully on scratch paper beforehand.
Hint 2.
For instance, you can take \(x_0=0\) and \(\varepsilon = 1/2\text{.}\) It may help to sketch a picture.
Solution.
Following the hint we take \(x_0=0\) and \(\varepsilon = 1/2\text{.}\) For any \(\delta \gt
0\text{,}\) we can choose \(n \in \N\) large enoughthat
1
Clearly we can do this since the left hand side is positive and tends to zero as \(n \to \infty\text{.}\)
\begin{equation*}
x = \frac{\pi}{2^{n+1}} \in (0,\delta).
\end{equation*}
For this \(x\) we then have
\begin{equation*}
\abs{f_n(x)-f_n(0)} = \abs{1-0} = 1 \gt \tfrac 12 = \varepsilon \text{.}
\end{equation*}
Thus \(F\) is not equicontinuous at \(x_0\text{.}\)
(b)
using Exercise 4.3.5 and the Arzelà–Ascoli theorem.
Solution.
We know from Exercise 4.3.5 that \(F\) is closed and bounded, but not compact. This then implies that \(F\) is not relatively compact either, as \(\overline F = F\text{.}\) Applying Theorem 4.29, the only possibility is that \(F\) is not equicontinuous.
Comment.
Since we know so much about this set \(F\) from Exercise 4.3.5, and we have so many results in Chapter 4, there are many different ways to argument here. For instance, one can use Theorem 4.27.
4. (PS10) Compactness and indefinite integrals.
Define a mapping \(\Phi \maps C^0([a,b]) \to C^0([a,b])\) by
\begin{equation*}
\Phi(f)(x) = \int_a^x f(t)\, dt\text{.}
\end{equation*}
Suppose that \(F \subset C^0([a,b])\) is bounded. Show that its image \(\Phi(F)\) under \(\Phi\) is relatively compact.
Hint 1.
Remember that when we say that a subset of a metric space is bounded, we always mean this in the sense of Definition 1.15.
Hint 2.
Show that \(\Phi(F)\) is bounded in \(C^{0,1}([a,b])\) and use Corollary 4.31. Another possibility is to show that \(\Phi(F)\) is bounded in \(C^1([a,b])\) and then apply Exercise 3.2.3 and Corollary 4.31.
Solution.
By Corollary 4.31, it suffices to show that \(\Phi(F)\) is bounded in \(C^{0,1}([a,b])\text{.}\) As \(F\) is bounded in \(C^0([a,b])\text{,}\) there exists a constant \(M \gt 0\) such that \(\n
f_{C^0} \le M\) for all \(f \in F\text{.}\) Now let \(g \in \Phi(F)\) be arbitrary, and pick \(f \in F\) such that \(\Phi(f)=g\text{.}\) Then for any \(x \in [a,b]\) we can estimate
\begin{align*}
\abs{g(x)}
\amp = \left| \int_a^x f(t)\, dt \right|\\
\amp \le \int_a^x \abs{f(t)}\, dt \\
\amp \le \int_a^x M\, dt \\
\amp = (x-a) M\\
\amp \le (b-a) M\text{.}
\end{align*}
Thus \(\n g_{C^0([a,b])} \le (b-a)M\text{.}\) To estimate \([g]_{C^{0,1}([a,b])}\text{,}\) pick \(x,y \in [a,b]\) with \(x \ne y\text{,}\) and assume without loss of generality that \(x \lt y\text{.}\) Then
\begin{align*}
\abs{g(y)-g(x)}
\amp = \left|
\int_a^y f(t)\, dt
-
\int_a^x f(t)\, dt
\right|\\
\amp = \left|
\int_x^y f(t)\, dt
\right|\\
\amp \le \int_x^y \abs{f(t)}\, dt \\
\amp \le \int_x^y M\, dt \\
\amp = \abs{x-y} M\text{.}
\end{align*}
Dividing by \(\abs{x-y}\) and taking a supremum, we obtain \([g]_{C^{0,1}([a,b])} \le M\text{.}\) Putting everything together, we have that
\begin{equation*}
\n g_{C^{0,1}([a,b])}
= \n g_{C^0([a,b])} + [g]_{C^{0,1}([a,b])}
\le (b-a+1)M
\end{equation*}
and hence, since \(g \in \Phi(F)\) was arbitrary, that \(\Phi(F)\) is bounded (with diameter \(\le 2(b-a+1)M\)).
Comment 1.
Note that in Corollary 4.31 we need \(S\) to be bounded as a subset of \(C^{0,\alpha}([a,b])\text{,}\) i.e. in the sense of Definition 1.15. This means
- \(S \subset C^{0,\alpha}([a,b])\text{;}\) and
- \(\sup_{f \in S} \n {f}_{C^{0,\alpha}([a,b])} \lt \infty\text{.}\)
Comment 2.
Note carefully the difference between the statement “for any \(f \in S\text{,}\) \(\n f_{C^{0,1}} \lt \infty\)” and the statement “\(\sup_{f \in S} \n
f_{C^{0,1}} \lt \infty\)”. Getting these two statements confused on an exam (or writing in a way where I cannot tell which you mean) is extremely dangerous. For exactly the same issue in a simpler context, see the last part of Exercise B.2 on Problem Sheet 1.
