Rate of Change: Average vs Instantaneous

In the previous chapter, we looked at functions as ways to map inputs to outputs and explored limits to understand function behavior near specific points. Now, we shift our focus to how function outputs change as their inputs change. Think about driving a car: your position changes over time. Sometimes you speed up, sometimes you slow down. Calculus gives us tools to describe these changes precisely.

We often talk about change in terms of rates. For instance, speed is the rate of change of distance with respect to time. But there are different ways to measure this rate.

Average Rate of Change: The Big Picture

Imagine you're on a road trip. You start at mile marker 10 at 1:00 PM and reach mile marker 130 at 3:00 PM. What was your average speed?

You traveled a distance of $130 - 10 = 120$ miles. The time taken was $3:00 \text{ PM} - 1:00 \text{ PM} = 2$ hours.

Your average speed was:

$$\text{Average Speed} = \frac{\text{Total Distance}}{\text{Total Time}} = \frac{120 \text{ miles}}{2 \text{ hours}} = 60 \text{ miles per hour}$$

This is the average rate of change of your position over that 2-hour interval. It gives you an overall sense of how quickly your position changed during the trip, but it doesn't tell you if you were driving exactly 60 mph the entire time. You might have stopped for gas or sped up on an open highway.

Mathematically, if we have a function $f(x)$, the average rate of change between two points, say $x_1$ and $x_2$, is the change in the function's output ($y$ value) divided by the change in the input ($x$ value). We often use the Greek letter delta ($\Delta$) to represent "change in". So, $\Delta y$ means the change in $y$, and $\Delta x$ means the change in $x$.

The formula is:

$$\text{Average Rate of Change} = \frac{\Delta y}{\Delta x} = \frac{f(x_2) - f(x_1)}{x_2 - x_1}$$

Geometrically, this formula calculates the slope of the line connecting the two points $(x_1, f(x_1))$ and $(x_2, f(x_2))$ on the graph of the function. This line is called a secant line.

Let's look at a simple function, $f(x) = x^2$. What's the average rate of change between $x_1 = 1$ and $x_2 = 3$?

First, find the corresponding $y$ values: $f(1) = 1^2 = 1$ $f(3) = 3^2 = 9$

Now, apply the formula:

$$\text{Average Rate of Change} = \frac{f(3) - f(1)}{3 - 1} = \frac{9 - 1}{2} = \frac{8}{2} = 4$$

So, on average, the function $f(x) = x^2$ increased by 4 units in $y$ for every 1 unit increase in $x$ between $x=1$ and $x=3$.

The red line connects the points (1, 1) and (3, 9) on the graph of $f(x) = x^2$. Its slope represents the average rate of change between $x=1$ and $x=3$.

Instantaneous Rate of Change: The Moment-by-Moment View

The average rate of change is useful, but often we need to know how fast something is changing right now, at a specific instant. If you look at your car's speedometer, it tells you your speed at that precise moment, not your average speed over the last hour. This is the instantaneous rate of change.

How can we find the rate of change at a single point, say at $x = x_1$? Our average rate of change formula $\frac{f(x_2) - f(x_1)}{x_2 - x_1}$ runs into a problem. If we try to make the interval smaller and smaller by bringing $x_2$ closer to $x_1$, eventually $x_2$ would equal $x_1$. This makes the denominator $x_2 - x_1$ zero, and we can't divide by zero.

This is where the concept of limits from Chapter 1 comes into play. Instead of setting $x_2$ equal to $x_1$, we ask: what value does the average rate of change approach as $x_2$ gets arbitrarily close to $x_1$?

Think about the secant line on the graph. As we move the point $(x_2, f(x_2))$ closer and closer to $(x_1, f(x_1))$, the secant line pivots. The line it approaches in the limit is called the tangent line at the point $(x_1, f(x_1))$. This tangent line touches the curve at that single point and represents the direction of the curve at that exact spot.

The slope of this tangent line is the instantaneous rate of change of the function at $x = x_1$.

As the second point (orange, yellow, green, cyan markers) gets closer to the first point (red marker at x=1), the slope of the secant line approaches the slope of the purple tangent line. The slope of this tangent line represents the instantaneous rate of change at x=1. (Click Play to animate).

This instantaneous rate of change is exactly what we mean by the derivative of a function at a point. It tells us how sensitive the function's output is to tiny changes in its input right at that location.

Understanding this distinction is significant. In machine learning, we often want to know how a small change in a model's parameter (like a weight in a neural network) will affect the model's error right now. We aren't usually interested in the average change over a large adjustment; we want the instantaneous change to guide us on how to make the next small adjustment to improve the model. This requires understanding the instantaneous rate of change, which leads us directly to the definition and calculation of derivatives in the next sections.