Probability Mass Function (PMF) for Discrete Distributions

In the previous section, we established that probability distributions describe the likelihood of different outcomes for a random variable. We also noted a fundamental difference between variables that can take on distinct, separate values (discrete) and those that can take on any value within a range (continuous). Now, let's focus on how we precisely define the probabilities for discrete random variables.

Imagine you have a random process with a finite or countably infinite number of possible outcomes, like rolling a die or counting the number of clicks on an ad. How do we assign probabilities to each specific outcome? This is where the Probability Mass Function (PMF) comes in.

What is a Probability Mass Function (PMF)?

A Probability Mass Function, often abbreviated as PMF, is a function that gives the probability that a discrete random variable $X$ is exactly equal to some specific value $x$. Think of it as distributing the total probability (which is always 1) among all the possible discrete outcomes, assigning a certain "mass" of probability to each value.

Mathematically, we denote the PMF as $P(X=x)$ or sometimes $p(x)$. This function must satisfy two main conditions:

1. Non-negativity: The probability assigned to any specific value $x$ must be greater than or equal to zero. You can't have a negative chance of something happening. $P(X=x) \ge 0 \quad \text{for all possible values } x$
2. Summation to One: If you sum up the probabilities for all possible values that the discrete random variable $X$ can take, the total sum must equal 1. This means we've accounted for all possible outcomes. $\sum_{x} P(X=x) = 1$ where the sum is taken over all possible values $x$ in the sample space of $X$.

Example: Rolling a Fair Six-Sided Die

Let's use a familiar example: rolling a standard, fair six-sided die. The random variable, let's call it $X$, represents the outcome of the roll. The possible values for $X$ are the integers from 1 to 6: $\{1, 2, 3, 4, 5, 6\}$. This is a discrete set of outcomes.

Since the die is fair, each outcome has an equal chance of occurring. There are 6 possible outcomes, so the probability of rolling any specific number is $1/6$.

The PMF for this random variable $X$ can be written as:

$$P(X=x) = \begin{cases} 1/6 & \text{if } x \in \{1, 2, 3, 4, 5, 6\} \\ 0 & \text{otherwise} \end{cases}$$

Let's check our two conditions:

1. Non-negativity: Is $P(X=x) \ge 0$ for all $x$? Yes, the probability is $1/6$ for the possible outcomes and $0$ otherwise, both of which are non-negative.
2. Summation to One: Does the sum of probabilities equal 1? $\sum_{x} P(X=x) = P(X=1) + P(X=2) + P(X=3) + P(X=4) + P(X=5) + P(X=6)$ $= \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = 6 \times \frac{1}{6} = 1$ Yes, the probabilities sum to 1.

So, this function $P(X=x) = 1/6$ (for $x=1..6$) is indeed the PMF for a fair die roll.

Visualizing the PMF

We can visualize the PMF using a bar chart. The height of each bar represents the probability $P(X=x)$ for each possible outcome $x$.

The probability mass is distributed equally across the six possible outcomes of the die roll. Each outcome has a probability mass of 1/6.

The PMF gives us a clear picture of how probability is distributed across the possible discrete values. It's important to remember that the PMF provides the probability of the random variable being exactly equal to a specific value. This contrasts with continuous distributions, where the probability of hitting any single exact value is zero, and we instead use a Probability Density Function (PDF) to talk about probabilities over intervals, which we will cover shortly.

Understanding the PMF is the first step toward working with specific discrete probability distributions like the Bernoulli and Binomial distributions, which are frequently encountered in modeling binary outcomes or count data in machine learning.