Calculating Partial Derivatives

Okay, let's dive into the mechanics of actually calculating partial derivatives. The concept introduced earlier, treating other variables as constants, is the main idea you need to remember. If you're comfortable calculating derivatives for single-variable functions like $f(x)$, you already have most of the skills needed.

The Process: Freeze and Differentiate

When you need to find the partial derivative of a function with respect to a specific variable, follow these steps:

1. Identify the Target Variable: Determine which variable you are differentiating with respect to. For example, if you're calculating $\frac{\partial f}{\partial x}$, your target variable is $x$.
2. Treat Other Variables as Constants: Mentally (or literally, if it helps) replace all other variables in the function with fixed constants. Think of them as numbers like 5, -2, or $\pi$.
3. Apply Standard Differentiation Rules: Now, differentiate the function using the familiar rules (like the Power Rule, Constant Rule, Sum Rule) only with respect to your target variable. Remember that the derivative of any constant term is zero.

Let's work through a couple of examples to make this concrete.

Example 1: A Simple Polynomial

Consider the function: $f(x, y) = x^2 + y^3 + 4$

Let's find the partial derivatives with respect to $x$ and $y$.

Calculating $\frac{\partial f}{\partial x}$ (Partial derivative with respect to $x$):

1. Target Variable: $x$.

2. Treat Others as Constants: Treat $y$ as a constant. This means $y^3$ is also treated as a constant. The number $4$ is already a constant.

3. Differentiate:

   • The derivative of $x^2$ with respect to $x$ is $2x$.
   • The derivative of $y^3$ (treated as a constant) with respect to $x$ is $0$.
   • The derivative of $4$ (a constant) with respect to $x$ is $0$.

   Putting it together using the Sum Rule: $\frac{\partial f}{\partial x} = 2x + 0 + 0 = 2x$

Calculating $\frac{\partial f}{\partial y}$ (Partial derivative with respect to $y$):

1. Target Variable: $y$.

2. Treat Others as Constants: Treat $x$ as a constant. This means $x^2$ is also treated as a constant. The number $4$ is a constant.

3. Differentiate:

   • The derivative of $x^2$ (treated as a constant) with respect to $y$ is $0$.
   • The derivative of $y^3$ with respect to $y$ is $3y^2$.
   • The derivative of $4$ (a constant) with respect to $y$ is $0$.

   Putting it together: $\frac{\partial f}{\partial y} = 0 + 3y^2 + 0 = 3y^2$

Notice how the process isolates the effect of changing only one variable. When we calculated $\frac{\partial f}{\partial x}$, the $y^3$ term vanished because, from the perspective of $x$, $y$ isn't changing.

Example 2: Variables Multiplied Together

Let's look at a function structure often seen when dealing with model parameters, like weights ($w$) and biases ($b$):

$g(w, b) = w^2 b + 5w - 2b + 7$

Calculating $\frac{\partial g}{\partial w}$ (Partial derivative with respect to $w$):

1. Target Variable: $w$.

2. Treat Others as Constants: Treat $b$ as a constant.

3. Differentiate:

   • Consider the term $w^2 b$. Since $b$ is treated as a constant coefficient, the derivative of $w^2 b$ with respect to $w$ is $(b) \times (2w) = 2wb$. (Think of it like finding the derivative of $5x^2$, which is $5 \times 2x = 10x$; here $b$ plays the role of $5$.)
   • The derivative of $5w$ with respect to $w$ is $5$.
   • The derivative of $-2b$ (treated as a constant) with respect to $w$ is $0$.
   • The derivative of $7$ (a constant) with respect to $w$ is $0$.

   Combining these: $\frac{\partial g}{\partial w} = 2wb + 5 + 0 + 0 = 2wb + 5$

Calculating $\frac{\partial g}{\partial b}$ (Partial derivative with respect to $b$):

1. Target Variable: $b$.

2. Treat Others as Constants: Treat $w$ as a constant. This means $w^2$ and $5w$ are also treated as constants.

3. Differentiate:

   • Consider the term $w^2 b$. Since $w^2$ is treated as a constant coefficient, the derivative of $w^2 b$ with respect to $b$ is $w^2 \times 1 = w^2$. (Think of it like finding the derivative of $ax$ with respect to $x$, which is $a$; here $w^2$ plays the role of $a$ and $b$ plays the role of $x$.)
   • The derivative of $5w$ (treated as a constant) with respect to $b$ is $0$.
   • The derivative of $-2b$ with respect to $b$ is $-2$.
   • The derivative of $7$ (a constant) with respect to $b$ is $0$.

   Combining these: $\frac{\partial g}{\partial b} = w^2 + 0 - 2 + 0 = w^2 - 2$

Key Takeaway

Calculating partial derivatives reuses the differentiation rules you learned for single-variable functions. The essential technique is to temporarily "freeze" all variables except the one you are differentiating with respect to, treating them as constants during the calculation. This allows you to determine how the function's output changes as that single target variable changes, holding everything else steady. This technique is fundamental for understanding how to adjust parameters in machine learning models using methods like gradient descent.