A limit, if it exists as a real number, is exact and unique and not an approximation. But books and intuitive explanations constantly talk about getting “arbitrarily close”. But being arbitrarily close still does not mean being exactly at the point, right? It is just arbitrarily close ¯\_(ツ)_/¯
So why are we allowed to use the value of a limit in calculations such as the derivative formula? And why is it that the limit is always an exact value and not just an approximation?
We will try to convince even the biggest skeptic (me) that the limit is indeed an exact number and not just an approximation and answer these questions.
To do this, it makes sense to look at the derivative formula, as it is relatively intuitive and offers useful insight. For the sake of simplicity, we will remain within the real-number domain and use polynomials as examples.
For all the mathematicians and rigorous folks, the following sections might sometimes not be completely accurate in favor of intuitive clarity. If, however, you spot a critical error, feel free to reach out!
Limit Notation
Stop, before we dive straight into derivatives we should make something clear. What the hell does \(\lim_{x \to a} f(x)\) mean? This expression is read as follows: “The limit of f(x) as x approaches a”. To make it a bit less abstract, let’s say we have a function \(f(x)=x^2\) and we want to find its limit at \(x=3\). This leads to the following expression: \(\lim_{x \to 3} x^2\) which reads as “The limit of x^2 as x approaches 3”. The key here is that this expression does not simply set \(x=3\) and substitute it into \(f(x)\). Instead, we examine values approaching \(x=3\) from both sides, as shown in the following table:
| Approaching from the left, \(x<3\) | \(f(x)=x^2\) | Approaching from the right, \(x>3\) | \(f(x)=x^2\) |
|---|---|---|---|
| 2.9 | 8.41 | 3.1 | 9.61 |
| 2.99 | 8.9401 | 3.01 | 9.0601 |
| 2.999 | 8.994001 | 3.001 | 9.006001 |
| 2.9999 | 8.99940001 | 3.0001 | 9.00060001 |
| 2.99999 | 8.9999400001 | 3.00001 | 9.0000600001 |
Table (1). As \(x\) approaches \(3\) from the left and from the right, \(f(x)=x^2\) approaches \(9\).
Table (1) strongly suggests that the function values approach 9 from both sides. But none of the displayed values is exactly equal to \(9\). Additionally, we did not use \(x=3\), instead we used values slightly bigger or smaller than \(3\). Because there are infinitely many numbers arbitrarily close to \(3\), we could continue extending the table indefinitely. This is what is meant by “approaching”. We approach \(x=3\) from both sides and the limit captures the value that \(f(x)\) approaches as \(x\) approaches \(3\). In this case, the limit is indeed 9. This also means that the limit and the actual function value at the point do not have to be the same thing. For example, we can define a function that behaves like \(f(x)=x\) everywhere around \(x=3\), but set the value at \(x=3\) to 10:
$$ f(x)= \begin{cases} x, & x\neq 3 \\ 10, & x=3 \end{cases} $$
For now, it suffices to understand that the notation \(\lim_{x \to a} f(x)\), specifically the \(x\to a\) part, means that we examine the behavior of \(f(x)\) for values of \(x\) arbitrarily close to \(a\), without requiring \(x=a\).
Derivative
Before we dive into the formalism of the derivative, let’s provide some intuition. Assuming we have a well-defined, differentiable function, the derivative provides the slope at a given point, also known as the instantaneous rate of change.
The idea behind the formula of the derivative is that we take two points \(x_1\) and \(x_2\) on the x-axis and calculate the difference of their corresponding function values \(f(x_2) and f(x_1)\) over the interval \([x_1,x_2]\).
We can do so by taking the difference of the values \(y_1=f(x_1)\) and \(y_2=f(x_2)\) and dividing by \(x_2-x_1\). We can write this as follows:
$$ \begin{aligned} \text{(1)}\quad \frac{f(x_2)-f(x_1)}{x_2-x_1} = \frac{y_2-y_1}{x_2-x_1} \ = \frac{\Delta y}{\Delta x} \end{aligned} $$
Equation (1) provides the average slope over an interval \([x_1,x_2]\). In case you are unfamiliar or scared of \(\Delta x\) and \(\Delta y\), they are just a compact way to write the differences between two points. The slope of equation (1) is also referred to as the slope of the secant line which is simply the line we get when drawing a line through \((x_1,y_1)\) and \((x_2,y_2)\).
While this sounds super interesting and cool, we are still computing the slope between two different points, but the derivative provides the slope at a single point. So how does this work? How can a single point even have a slope?
In order to show this, let’s first dissect the formula for the derivative piece by piece. The definition of the derivative is the following:
$$ \text{(2)}\quad f’(x)=\lim_{h \to 0} \text{ }\frac{f(x+h)-f(x)}{h} $$
Now, let’s make this easier to digest and use \(f(x)=x^2\). Substituting \(x^2\) we get the following:
$$ \begin{aligned} f’(x) &= \lim_{h \to 0} \text{ }\frac{(x+h)^2 - x^2}{h} \\ &= \lim_{h \to 0} \text{ }\frac{x^2 + 2xh + h^2 - x^2}{h} \\ &= \lim_{h \to 0} \text{ }\frac{2xh + h^2}{h} \\ &= \lim_{h \to 0} \text{ }(2x + h) \\ &= 2x \end{aligned} $$
After evaluating the expression we end up with \(f’(x)=2x\) which is also the derivative as we know. Previously, we discussed secant lines. The derivative now gives us the unique value approached by their slopes as the two points move closer together. Geometrically, this value is the slope of the tangent line. A secant line passes through two distinct points on the curve, whereas the tangent line through the chosen point has the limiting slope of nearby secant lines. Thus, the derivative gives us the exact instantaneous slope rather than an average slope over an arbitrarily small interval.
But stop. The derivative formula seems to contradict our intuitive explanation. We approach \(0\), but we never actually plug \(h=0\) into the original expression. So where did the \(h\) go? Why did it simply disappear if we said it never equals \(0\)? Well, before we can truly answer this question we must stay strong and dive just a tiny bit deeper.
The big ol’ \(\epsilon\)-\(\delta\)
The \(\epsilon\)-\(\delta\)-definition looks like this:
$$ \text{(3)}\quad \lim_{x \to a} f(x) = L \quad \Longleftrightarrow \quad \forall \epsilon > 0\ \exists \delta > 0 \forall x: 0 < |x-a| < \delta \Rightarrow |f(x)-L| < \epsilon $$
Yikes, that’s heavy. Let’s walk through it step by step. First, we have an equivalence, meaning that each side implies the other. Let’s focus on the right-hand side:
$$ \forall \epsilon > 0\ \exists \delta > 0 \forall x: 0 < |x-a| < \delta \Rightarrow |f(x)-L| < \epsilon $$
This is a conditional statement and reads as follows:
“For all epsilon greater than zero, there exists a delta greater than 0, such that IF the distance between \(x\) and \(a\) is strictly greater than 0 and less than delta THEN the distance between \(f(x)\) and \(L\) is less than epsilon.”
Even when written out in words, it still sounds rather heavy, so let’s use a picture. We all know the saying “A picture is worth a thousand words” ✨
Figure (1). \(\epsilon\)-\(\delta\)-Definition visualized. Source (modified): Math StackExchange
In Figure (1) there are two important parts. First, there are the horizontal lines around the limit \(L\), spanning the red tube, which are denoted with \(L+\epsilon\) and \(L-\epsilon\). They define the accepted error, or maximum deviation, around the limit \(L\) that we accept for our function values \(f(x)\).
The second part shows a blue tube around \(a\) which is bounded by \(a+\delta\) and \(a-\delta\). This interval defines a neighborhood around \(a\) in which all points \(a_i\in (a-\delta,a)\cup(a,a+\delta)\) must satisfy \(|f(a_i)-L| < \epsilon\).
We can see that if we tighten the accepted error (red tube) we also have to tighten the blue tube because otherwise there may be inputs \(a_i\in (a-\delta,a)\cup(a,a+\delta)\) where \(|f(a_i)-L| \geq \epsilon\) and thus the condition would therefore fail.
In short, all values \(f(x)\) must fall within this error range \(\epsilon\) for the interval \((a-\delta,a)\cup(a,a+\delta)\) on the x-axis.
Putting it all together
Okay, we have now talked about derivatives and the \(\epsilon\)-\(\delta\)-definition. How should this convince even the biggest skeptic that limits are indeed exact? The ✨magic✨ is buried inside the \(\epsilon\)-\(\delta\)-definition and can be understood particularly well through derivatives.
When going back to derivatives, we learned that the derivative \(f’(x)\) is the slope of the tangent at point \(x\). Furthermore, it is the unique value that the secant slopes converge to as the distance between the two points of the secants approaches \(0.\) Looking at Figure (2), every secant slope is still an approximation of the tangent slope because \(h\neq0\), but the secant slopes are forced toward the exact tangent slope which in turn is the limit.
And here comes the important part: In formula (2) we asked why \(h\) suddenly disappeared. The important point is that \(h\) approaches \(0\), but we never substitute \(h=0\) into the original fraction. The original expression is even undefined at \(h=0\).
$$ \begin{aligned} &= \lim_{h \to 0} \text{ }\frac{2xh + h^2}{h} \\ &= \lim_{h \to 0} \text{ }(2x + h) \\ \end{aligned} $$
We were able to make this cancellation because \(h\neq0\) for all values under consideration. In other words, we simplified the expression only for \(h\neq0\). After that we ended up with \(\lim_{h \to 0} \text{ }(2x + h)\). When we then take the limit we essentially make the distance \(h\) between the two points of the secant line approach \(0\) and thus the slope of the secant lines approaches the slope of the tangent line. The limit allows us to quantify the unique value that all secant slopes converge to in a mathematically sound way, without having to rely on the function value at \(h=0\) which might be undefined or different from the limit. This further illustrates that the limit is a separate object from the value at \(h=0\).
The exactness of the limit can be illustrated with the following example: If we did not take the limit, we would end up with \(2x+h\) which is an approximation since \(h\) would be non-zero and thus correspond to a secant slope and not the tangent slope. For example, if \(h=0.0001\), we obtain a secant slope that is very close to the tangent slope but still not exactly equal to it. Each secant slope still has a small error, no matter how small we make \(h\) because there is no smallest positive nonzero value for \(h\). Therefore, the limit \(L\) is not the last element in a sequence of approximations. Instead it is the unique value for which the approximation error can be made smaller than any positive error margin \(\epsilon\).
We can illustrate this behavior further in Figure (2) where we can make the distance \(h\) smaller and smaller but for nonzero \(h\), we would still obtain a secant line and not a tangent. However, we are interested in the slope at the exact point, which is given by the slope of the tangent at that point. While we numerically observe that the number we are looking for seems to be \(2x\), we need a mathematical framework to rigorously define this concept. Hence we take the limit, as we cannot just put \(h=0\) and magically let \(h\) disappear.
The limit makes it formally possible to find the exact slope at that point without ever substituting \(h=0\) into the original fraction. This is also crucial to handle functions that are undefined at the point we approach or where their limit is different from the function value like in the piecewise function defined earlier. This is a crucial point to understand, as it highlights the distinction between the limit and the actual function value at a specific point.
Hence the limit is the value such that for every error margin \(\epsilon>0\), the \(\epsilon\)-\(\delta\) condition guarantees that \(f(x)\) remains within \(\epsilon\) of the same fixed value \(L\), provided that \(x\) is sufficiently close to \(a\). The condition \(0 < |x-a| < \delta\) describes exactly this idea, where we look for inputs arbitrarily close to \(a\) but not equal to \(a\) itself.
Another important note for the \(\epsilon\)-\(\delta\) definition is that it is used to check what it means for \(L\) to be the limit and to check if a chosen \(L\) is valid. It is primarily a definition and proof tool rather than a practical recipe for calculating limits. Additionally, the \(\epsilon\)-\(\delta\) definition can be used to prove that a limit is unique, which is a fundamental property of limits. If two different values were to satisfy the \(\epsilon\)-\(\delta\) condition for the same limit, it would lead to a contradiction, thus confirming that the limit must be unique.
This emphasizes that the limit is a separate object from the value at \(h=0\). Figure (3) demonstrates the same distinction using a piecewise function. Now we see that \(\lim_{x\to 3} f(x)\) is equal to \(3\) but \(f(3)=10\). Therefore the limit when approaching \(a\) can be different from \(f(a)\) and hence the limit and the function value are two separate objects.
Since \(\lim_{x\to 3} f(x)=3\), the nearby behavior at \(x=3\) suggests that \(f(3)\) would be \(3\). However, we see that this is not the case and \(f(3)=10\). Because \(\lim_{x\to 3} f(x)=3\) (our expected behavior at \(x=3\)) differs from the actual function value \(f(3)\) we can go even further and deduce that the function is not continuous 💡.
But what does continuity mean? That’s another story :)
All in all, it might help to think about limits as a tool to describe the value that \(f(x)\) approaches as \(x\to a\), even when it is not defined at \(a\). The nearby function values are approximations to the limit, but the limit itself is not an approximation, thus allowing us to analyze and understand its behavior in a rigorous way. Keeping in mind that the limit is a separate object from the function value at that point, we can see that the limit provides a powerful tool for understanding the behavior of functions in a precise and rigorous way.