Math for Data Science · MDS-5
Reason about uncertainty the way data scientists do — from coin flips to the Central Limit Theorem, simulated in code.
Probability is the language data scientists use to describe uncertainty, model randomness, and justify inference. This course builds the subject from axioms through the Central Limit Theorem, pairing each idea with a hands-on simulation in Python (NumPy/SciPy) or R. By the end you can model real-world randomness, compute and interpret distributions, and explain why sample averages behave the way they do.
Course Outline
Every lesson is self-contained: a full explanation, worked step-by-step examples (in Python and R/Julia), common mistakes, a try-it-yourself, and an interactive quiz. Jump to a unit:
Before computing any probability you must precisely describe what can happen. This lesson sets up the vocabulary of outcomes, sample spaces, and events.
An experiment is any process with an uncertain result. Each indivisible result is an outcome, and the set of all possible outcomes is the sample space, written Omega (or S). An event is any subset of the sample space — a collection of outcomes we care about, such as 'the roll is even.' The event occurs whenever the realized outcome belongs to that subset. Sample spaces can be finite (a die roll), countably infinite (number of coin flips until a head), or continuous (a waiting time). In data science this framing is the foundation of every probabilistic model: classifiers partition an outcome space into label events, and A/B tests treat each user as an outcome. Getting the sample space right — listing outcomes at the correct granularity and ensuring they are exhaustive and mutually exclusive — is what makes later probability calculations valid.
Worked Example 1
Problem. A fair coin is flipped once. Write the sample space and the event 'heads.'
Answer. Omega = {H, T}; A = {H}; P(A) = 1/2.
Worked Example 2
Problem. Two distinguishable dice are rolled. Describe the sample space and the event 'the sum is 7.'
Answer. |Omega| = 36; P(sum = 7) = 6/36 = 1/6.
Worked Example 3
Problem. Flip a coin until the first head appears. What does the sample space look like, and what is the event 'it takes an even number of flips'?
Answer. Omega is countably infinite; P(even number of flips) = 1/3.
Problem. A family has two children. Treating boy/girl as equally likely and birth order as distinguishable, write the sample space and find the probability of exactly one girl.
Solution. Omega = {BB, BG, GB, GG}, 4 equally likely outcomes. 'Exactly one girl' = {BG, GB}, which has 2 outcomes. P = 2/4 = 1/2.
Probability is not arbitrary — it obeys three rules from which every other formula follows. This lesson states Kolmogorov's axioms and shows what they let you derive.
Kolmogorov's axioms define a probability measure P on a sample space. Axiom 1 (non-negativity): P(A) >= 0 for every event A. Axiom 2 (normalization): P(Omega) = 1. Axiom 3 (countable additivity): for disjoint events A1, A2, ..., P(A1 or A2 or ...) = sum of P(Ai). From these three rules everything else follows: P(empty set) = 0, P(A complement) = 1 - P(A), monotonicity (if A is a subset of B then P(A) <= P(B)), and the bound P(A) <= 1. In data science these axioms guarantee that any model outputting probabilities — softmax scores, calibrated classifiers, Bayesian posteriors — is internally consistent. When a model's predicted class probabilities fail to sum to 1 or go negative, it has violated an axiom, signaling a bug or a need for normalization.
Worked Example 1
Problem. If P(A) = 0.7, what is P(not A)?
Answer. P(not A) = 0.3.
Worked Example 2
Problem. A loaded die gives P(1)=0.1, P(2)=0.1, P(3)=0.2, P(4)=0.2, P(5)=0.2. Find P(6) and P(roll <= 3).
Answer. P(6) = 0.2; P(roll <= 3) = 0.4.
Worked Example 3
Problem. Show that if A is a subset of B then P(A) <= P(B).
Answer. Monotonicity: A subset of B implies P(A) <= P(B).
Problem. A spinner lands in region A with probability 0.45 and region B with 0.35; the rest is region C. Regions are disjoint. Find P(C) and P(A or C).
Solution. P(C) = 1 - 0.45 - 0.35 = 0.20. A and C are disjoint, so P(A or C) = 0.45 + 0.20 = 0.65.
When every outcome is equally likely, probability reduces to counting. This lesson formalizes the favorable-over-total rule and its assumptions.
In the equally likely (classical) model, a finite sample space has every outcome carrying the same probability 1/|Omega|. Then for any event A, P(A) = |A| / |Omega| = (favorable outcomes) / (total outcomes). This reduces probability to combinatorics: count the ways an event can happen and divide by the total. The model is only valid when symmetry justifies equal likelihood — fair dice, well-shuffled cards, random sampling. In data science, the equally likely assumption underlies uniform random sampling, train/test splitting, and permutation tests, where each arrangement of labels is treated as equally probable under the null hypothesis. The core skill is careful counting: deciding whether order matters and whether repetition is allowed determines which counting rule applies.
Worked Example 1
Problem. Draw one card from a standard 52-card deck. What is the probability it is a heart?
Answer. P(heart) = 1/4.
Worked Example 2
Problem. Roll two fair dice. What is the probability the two faces are equal (a 'double')?
Answer. P(double) = 1/6.
Worked Example 3
Problem. From 5 distinct books, 2 are chosen at random. What is the probability a specific book (say, the dictionary) is among them?
Answer. P(dictionary chosen) = 2/5 = 0.4.
Problem. A bag has 4 red and 6 blue marbles. One marble is drawn at random. What is the probability it is red?
Solution. All 10 marbles are equally likely. Favorable (red) = 4, total = 10. P(red) = 4/10 = 2/5 = 0.4.
Counting outcomes systematically requires knowing when order matters. This lesson covers permutations, combinations, and the binomial coefficient n choose k.
Counting rules answer 'how many ways?' The multiplication principle says independent choices multiply. A permutation is an ordered arrangement: the number of ways to order all n distinct objects is n!, and the number of ordered selections of k from n is P(n,k) = n!/(n-k)!. A combination is an unordered selection of k from n, counted by the binomial coefficient C(n,k) = n choose k = n!/(k!(n-k)!), which divides out the k! orderings that a permutation would count separately. Combinations power the binomial distribution, feature-subset selection, and bootstrap/permutation resampling in data science. The decisive question is always: does order matter? If yes, use permutations; if no, use combinations.
Worked Example 1
Problem. How many ways can 4 runners finish a race (no ties)?
Answer. 24 finishing orders.
Worked Example 2
Problem. How many distinct 5-card poker hands can be dealt from 52 cards?
Answer. 2,598,960 hands.
Worked Example 3
Problem. A committee of 3 is chosen from 4 women and 5 men. What is the probability it has exactly 2 women?
Answer. P(exactly 2 women) = 5/14 ≈ 0.357.
Problem. How many ways can you choose a 3-topping pizza from 8 available toppings (no repeats, order irrelevant)?
Solution. Order does not matter, so use C(8,3) = 8!/(3!5!) = (8 x 7 x 6)/(3 x 2 x 1) = 336/6 = 56 ways.
When events overlap, you cannot simply add their probabilities. Inclusion-exclusion corrects for the double-counted overlap.
For two events, P(A or B) = P(A) + P(B) - P(A and B): adding P(A) and P(B) counts the overlap A and B twice, so we subtract it once. This is the inclusion-exclusion principle. For three events it extends to P(A or B or C) = sum of singles - sum of pairwise intersections + the triple intersection, alternating signs. The intersection A and B is the event that both occur; the union A or B is the event that at least one occurs. When events are disjoint, P(A and B) = 0 and inclusion-exclusion collapses to plain addition. In data science this principle appears in computing the probability that at least one of several rare failures occurs, in set-based deduplication, and in reasoning about overlapping filters or query conditions.
Worked Example 1
Problem. P(A) = 0.6, P(B) = 0.5, P(A and B) = 0.3. Find P(A or B).
Answer. P(A or B) = 0.8.
Worked Example 2
Problem. Draw one card from 52. What is the probability it is a king OR a heart?
Answer. P(king or heart) = 16/52 = 4/13.
Worked Example 3
Problem. Of 100 learners, 60 take math, 45 take statistics, and 25 take both. How many take at least one of the two?
Answer. 80 learners; P(at least one) = 0.8.
Problem. In a survey, 70% of people like coffee, 40% like tea, and 30% like both. What fraction likes coffee or tea?
Solution. P(coffee or tea) = 0.70 + 0.40 - 0.30 = 0.80. So 80% like at least one of the two.
Probability rules map directly onto set operations, which Python and R can simulate. This lesson turns the algebra into reproducible code.
Events are sets, and probability rules are set operations: union (or), intersection (and), and complement (not). Two computational approaches verify these rules. The analytic approach represents the sample space and events as Python sets or R vectors and counts directly. The Monte Carlo approach simulates the experiment many times and estimates P(A) as the fraction of trials in which A occurs; by the Law of Large Numbers this estimate converges to the true probability. Simulation is invaluable when a sample space is too large or complex to enumerate, which is the common case in real data science. The key practice is to define an event as a boolean condition on simulated outcomes, then average the boolean array to get an empirical probability.
Worked Example 1
Problem. Using set operations on a die {1..6}, verify P(even or >4).
Answer. P(even or >4) = 2/3.
Worked Example 2
Problem. Estimate P(sum = 7) for two dice by simulation in Python and compare to the exact 1/6.
Answer. Simulated ≈ 0.1669 vs exact 0.1667.
Worked Example 3
Problem. Verify the complement rule P(not A) = 1 - P(A) for 'sum >= 4' on two dice using R.
Answer. P(sum>=4) = 0.9167 = 1 - 0.0833.
Problem. Write a one-line NumPy estimate of P(at least one 6 in two dice rolls) and state the exact value.
Solution. Exact: P(at least one 6) = 1 - (5/6)^2 = 1 - 25/36 = 11/36 ≈ 0.3056. Code: rng=np.random.default_rng(1); d=rng.integers(1,7,(1_000_000,2)); print((d==6).any(1).mean()) gives ≈ 0.3057, matching.
Compute the exact probability that at least two people in a room of n share a birthday using the equally-likely / counting model, then write a Python or R simulation that repeatedly fills a room at random and estimates the same probability. Vary n from 2 to 60 and overlay the exact curve against your simulated estimates. Explain where and why simulation error appears.
Deliverable · A script plus a short write-up showing the exact-vs-simulated curve and the smallest n for which the probability first exceeds 50%.
1. Which of the following is a valid probability axiom?
Answer B. One axiom states P(Omega) = 1. Probabilities can be 0, can exceed 0.5, and are not defined by subtracting an outcome count.
2. A fair six-sided die is rolled. What is the probability of rolling an even number OR a number greater than 4?
Answer B. Even = {2,4,6}, greater than 4 = {5,6}. Their union is {2,4,5,6}, which is 4 of 6 outcomes = 2/3.
3. How many distinct 3-person committees can be formed from 8 people?
Answer B. Order does not matter, so use combinations: 8 choose 3 = 8!/(3!5!) = 56.
4. For events A and B, P(A) = 0.5, P(B) = 0.4, and P(A and B) = 0.2. What is P(A or B)?
Answer A. By inclusion-exclusion, P(A or B) = 0.5 + 0.4 - 0.2 = 0.7.
5. Two events are mutually exclusive when:
Answer C. Mutually exclusive (disjoint) means the events share no outcomes; the second option describes independence, a different idea.
I can describe an experiment with a sample space and identify events as subsets of it.
I can apply the probability axioms and inclusion-exclusion to compute event probabilities.
I can use permutations and combinations to count outcomes in equally likely settings.
Most real questions are conditional: given what we already know, how likely is something? This lesson defines conditioning and the multiplication rule that chains it.
The conditional probability of A given B, written P(A|B), is the probability that A occurs once we know B has occurred. It is defined as P(A|B) = P(A and B) / P(B), valid whenever P(B) > 0. Geometrically, conditioning on B shrinks the sample space to B and renormalizes. Rearranging gives the multiplication rule: P(A and B) = P(A|B) * P(B) = P(B|A) * P(A), which lets us chain dependent events step by step. In data science, conditioning is everywhere: a spam filter computes P(spam | words), a recommender estimates P(click | user, item), and feature engineering often encodes conditional rates. Mastering this definition is the gateway to independence, the law of total probability, and Bayes' theorem.
Worked Example 1
Problem. A card is drawn from 52. Given it is a face card (J, Q, K), what is the probability it is a king?
Answer. P(king | face card) = 1/3.
Worked Example 2
Problem. Roll two fair dice. Given the sum is 8, what is the probability one die shows a 5?
Answer. P(some die = 5 | sum = 8) = 2/5.
Worked Example 3
Problem. Two cards are drawn without replacement from 52. Use the multiplication rule to find P(both aces).
Answer. P(both aces) = 1/221 ≈ 0.0045.
Problem. In a deck of 52, two cards are drawn without replacement. What is the probability the second card is a heart given the first was a heart?
Solution. After removing one heart, 12 hearts remain among 51 cards. P(second heart | first heart) = 12/51 = 4/17 ≈ 0.235.
Sometimes knowing B tells you nothing about A. This lesson defines independence precisely and shows how to test for it.
Two events A and B are independent when the occurrence of one does not change the probability of the other: P(A|B) = P(A), equivalently P(A and B) = P(A) * P(B). The product form is the most useful test because it is symmetric and handles zero-probability edge cases gracefully. Independence is not the same as being disjoint — disjoint events with positive probability are actually dependent, because knowing one occurred guarantees the other did not. Independence can extend to many events (mutual independence requires every subset to factorize). In data science, the naive Bayes classifier assumes features are conditionally independent given the class, and many statistical tests assume independent observations; checking these assumptions guards against badly biased estimates.
Worked Example 1
Problem. Two fair coins are flipped. Are 'first is heads' and 'second is heads' independent?
Answer. Independent: P(A and B) = 1/4 = P(A)P(B).
Worked Example 2
Problem. Draw one card. Let A = 'heart' and B = 'king'. Are A and B independent?
Answer. Independent: 1/52 = (1/4)(1/13).
Worked Example 3
Problem. Roll one die. Let A = 'even' and B = 'roll <= 2'. Are they independent?
Answer. Independent: P(A and B) = 1/6 = P(A)P(B).
Problem. A die is rolled. A = 'roll is a multiple of 3' = {3,6}, B = 'roll is even' = {2,4,6}. Are A and B independent?
Solution. P(A) = 2/6 = 1/3, P(B) = 3/6 = 1/2. A and B = {6}, P = 1/6. P(A)P(B) = (1/3)(1/2) = 1/6 = P(A and B). Yes, independent.
To find an overall probability, average over every way it can happen. This lesson introduces partitions and the law of total probability.
A partition splits the sample space into disjoint events A1, ..., Ak that together cover everything. The law of total probability says that for any event B, P(B) = sum over i of P(B | Ai) * P(Ai). Intuitively, we weight the conditional probability of B within each piece by how likely that piece is, then add. It is the denominator engine of Bayes' theorem and the foundation of mixture models. In data science it formalizes scenarios like 'spam arrives from three sources with different rates' or 'customers belong to segments with different conversion probabilities' — you compute an overall rate by averaging segment rates weighted by segment sizes. The key is choosing a partition that makes each conditional probability easy to state.
Worked Example 1
Problem. Factory: machine A makes 60% of parts (2% defective), machine B makes 40% (5% defective). What fraction of all parts are defective?
Answer. P(defective) = 0.032 (3.2%).
Worked Example 2
Problem. An urn has 3 red, 2 blue balls. Two are drawn without replacement. What is P(second is red)?
Answer. P(second is red) = 3/5 (same as the first, by symmetry).
Worked Example 3
Problem. Emails come from work (50%, spam rate 1%), promos (30%, spam rate 20%), unknown (20%, spam rate 60%). What is the overall spam rate?
Answer. Overall spam rate = 0.185 (18.5%).
Problem. A box has two coins: a fair coin and a coin with P(heads)=0.8. You pick one at random and flip it. What is P(heads)?
Solution. Partition by coin: each chosen with probability 1/2. P(H) = (1/2)(0.5) + (1/2)(0.8) = 0.25 + 0.40 = 0.65.
Bayes' theorem flips a conditional probability around, letting evidence update belief. It is the engine of probabilistic inference.
Bayes' theorem computes P(A|B) from the reverse conditional P(B|A): P(A|B) = P(B|A) * P(A) / P(B). The denominator P(B) is usually expanded with the law of total probability, P(B) = P(B|A)P(A) + P(B|not A)P(not A). The theorem reverses the direction of reasoning — from 'probability of evidence given a hypothesis' to 'probability of the hypothesis given the evidence' — which is exactly what we need when we observe data and want to infer causes. In data science Bayes underlies spam filtering, medical diagnosis, A/B test analysis, and the entire field of Bayesian inference. The recurring lesson is that the answer depends heavily on the prior P(A), not just on how diagnostic the evidence is.
Worked Example 1
Problem. A test is 90% accurate for a disease affecting 1% of people (P(pos|disease)=0.9, P(pos|healthy)=0.1). Given a positive test, what is P(disease)?
Answer. P(disease | positive) ≈ 8.3%.
Worked Example 2
Problem. Two factories: A (70% of output, 1% defect), B (30%, 4% defect). A part is defective. Probability it came from B?
Answer. P(from B | defective) ≈ 0.632.
Worked Example 3
Problem. An email contains the word 'free'. P(free|spam)=0.6, P(free|ham)=0.05, and 20% of email is spam. P(spam|free)?
Answer. P(spam | 'free') = 0.75.
Problem. A factory's alarm sounds for 99% of real fires but also for 2% of non-fires. Fires occur on 0.1% of days. The alarm sounds — what is P(fire)?
Solution. P(alarm) = 0.99*0.001 + 0.02*0.999 = 0.00099 + 0.01998 = 0.02097. P(fire|alarm) = 0.00099/0.02097 ≈ 0.047, about 4.7%.
Bayes' theorem has three named pieces. Understanding prior, likelihood, and posterior turns the formula into a story of belief updating.
In Bayesian language, P(H) is the prior — your belief in hypothesis H before seeing data. P(D|H) is the likelihood — how probable the observed data D is under H. P(H|D) is the posterior — your updated belief after seeing D. Bayes' theorem reads posterior is proportional to likelihood times prior: P(H|D) proportional to P(D|H) P(H), with the evidence P(D) acting as a normalizing constant so the posterior sums to 1. The intuition is that data reweights your hypotheses by how well each predicted what you saw. Strong evidence overwhelms a weak prior; a strong prior resists weak evidence. In data science this is the backbone of Bayesian estimation, online learning, and sequential updating, where today's posterior becomes tomorrow's prior.
Worked Example 1
Problem. Two hypotheses for a coin: fair (H1) or two-headed (H2), priors 0.5 each. You flip one head. Find the posteriors.
Answer. Posterior: fair 1/3, two-headed 2/3.
Worked Example 2
Problem. Continuing, you flip the same coin and get a second head. Update the posterior using the previous posterior as the new prior.
Answer. After two heads: fair 1/5, two-headed 4/5 (same as updating from scratch with two heads).
Worked Example 3
Problem. If instead the second flip is a tail, what is the posterior?
Answer. A tail makes the coin certainly fair: posterior fair = 1.
Problem. Prior that a website visitor will buy is 0.10. A visitor adds an item to cart; P(cart|buyer)=0.8, P(cart|non-buyer)=0.2. Find the posterior P(buyer|cart).
Solution. Unnormalized: buyer 0.8*0.10=0.08; non-buyer 0.2*0.90=0.18. Evidence = 0.26. Posterior P(buyer|cart) = 0.08/0.26 ≈ 0.308.
When a condition is rare, even an accurate test produces mostly false alarms. This lesson explains base-rate neglect and the false-positive paradox.
The base rate is the unconditional prevalence P(disease) of a condition. The false-positive paradox arises because when the base rate is very low, the small fraction of false positives among the large healthy population can outnumber the true positives among the few sick people — so a positive test still implies a low probability of disease. Formally, P(disease|positive) depends on prevalence as much as on test accuracy. Base-rate neglect is the cognitive error of judging P(disease|positive) using only the test's accuracy while ignoring how rare the disease is. In data science this is critical for imbalanced classification: a fraud or rare-disease detector with 99% accuracy can still generate overwhelmingly false alarms, which is why precision, recall, and prevalence-aware metrics matter more than raw accuracy.
Worked Example 1
Problem. Disease prevalence 0.1%. Test: sensitivity 99%, specificity 99%. Find P(disease|positive).
Answer. Only about 9% — false positives dominate despite 99% accuracy.
Worked Example 2
Problem. Same test, but now prevalence is 10%. Find P(disease|positive).
Answer. About 92% — a higher base rate makes the same test far more conclusive.
Worked Example 3
Problem. In a city of 1,000,000 with prevalence 0.1% and the 99%/99% test, how many positives are true vs false?
Answer. 990 true vs 9990 false positives; ~9% of positives are real.
Problem. A fraud detector flags transactions with 95% sensitivity and 98% specificity. Only 0.5% of transactions are fraud. If a transaction is flagged, what is P(fraud)?
Solution. P(flag) = 0.95*0.005 + 0.02*0.995 = 0.00475 + 0.0199 = 0.02465. P(fraud|flag) = 0.00475/0.02465 ≈ 0.193, about 19% — most flags are false alarms.
A disease affects 1% of a population. A test has 99% sensitivity and 95% specificity. Use Bayes' theorem to compute the probability that a person who tests positive actually has the disease. Then write a Python or R simulation that generates a large population, applies the test, and empirically estimates the same posterior. Compare the analytic and simulated answers and explain the base-rate effect.
Deliverable · A notebook or script reporting the analytic posterior, the simulated posterior, and a one-paragraph explanation of why the number is surprisingly low.
1. If P(A) = 0.3, P(B) = 0.5, and A and B are independent, what is P(A and B)?
Answer B. For independent events the joint probability multiplies: 0.3 * 0.5 = 0.15.
2. Bayes' theorem lets you compute which quantity from the others?
Answer A. Bayes reverses the conditioning direction, turning P(A given B) into P(B given A) using the priors and total probability.
3. P(A given B) = 0.4, P(B) = 0.5. What is P(A and B)?
Answer C. By the multiplication rule, P(A and B) = P(A given B) * P(B) = 0.4 * 0.5 = 0.2.
4. In the disease-screening example, the main reason a positive test often means low disease probability is:
Answer B. When prevalence is small, the many healthy people generate enough false positives to swamp the few true positives.
5. Which statement is true if A and B are independent?
Answer A. Independence means conditioning on B does not change A's probability, so P(A given B) = P(A).
I can compute conditional probabilities and decide whether two events are independent.
I can apply the law of total probability and Bayes' theorem to update beliefs from evidence.
I can explain why a positive medical test can still mean a low chance of disease.
Working with raw outcomes is clumsy; attaching numbers to them lets us do arithmetic. This lesson defines a random variable as that numeric bridge.
A random variable X is a function that maps each outcome in the sample space to a real number. It does not 'vary randomly' on its own — the randomness lives in which outcome occurs, and X simply reports a number for that outcome. For two coin flips, X = number of heads sends HH to 2, HT and TH to 1, and TT to 0. Once outcomes carry numbers we can ask P(X = 1), P(X <= 1), compute averages, and compare experiments on a common numeric scale. In data science nearly every quantity we model — a click count, a purchase amount, a sensor reading — is a random variable summarizing a complicated underlying experiment. Choosing what to measure (the function g) is itself a modeling decision that determines which distribution and tools apply later.
Worked Example 1
Problem. Flip two fair coins. Define X = number of heads and find the probability distribution of X.
Answer. P(X=0)=0.25, P(X=1)=0.50, P(X=2)=0.25.
Worked Example 2
Problem. Roll two dice and let S be the sum. Find P(S = 5).
Answer. P(S = 5) = 1/9 ≈ 0.111.
Worked Example 3
Problem. A point lands uniformly in [0,1]. Define X = 1 if the point is in [0, 0.3] else 0. What kind of variable is X and what is P(X=1)?
Answer. X is a discrete indicator with P(X=1)=0.3, P(X=0)=0.7.
Problem. Flip three fair coins and let X = number of heads. Find P(X = 2).
Solution. There are 2^3 = 8 equally likely outcomes. Exactly two heads: HHT, HTH, THH — 3 outcomes. P(X=2) = 3/8 = 0.375.
The first fork in modeling any quantity is whether it is counted or measured. This lesson contrasts discrete and continuous random variables.
A discrete random variable takes values in a countable set — typically integers from counting — and assigns positive probability to individual values via a probability mass function. A continuous random variable takes values across an interval of the real line and is described by a probability density function; any single exact value has probability zero, and probabilities come from areas (integrals) over ranges. The dividing question is whether the quantity is counted (number of defects, clicks, arrivals) or measured on a continuum (time, height, temperature). Some quantities are mixed, but most data-science models pick one. The distinction governs every downstream tool: PMF versus PDF, sums versus integrals, and which SciPy or R function family (e.g. binom vs norm) you reach for. Misclassifying a variable leads to using the wrong probability calculus.
Worked Example 1
Problem. Classify: (a) number of emails per hour, (b) exact wait time for a bus.
Answer. (a) discrete, (b) continuous.
Worked Example 2
Problem. For a continuous variable, why is P(X = 5) = 0, and how do we get a nonzero probability?
Answer. Point probability is 0; use an interval's area for a positive probability.
Worked Example 3
Problem. A variable X is uniform on {1,2,3,4,5}. Is it discrete or continuous, and what is P(X = 3)?
Answer. Discrete; P(X = 3) = 0.2.
Problem. Classify each and name the right function (PMF or PDF): (a) the number of heads in 10 flips, (b) a person's reaction time in seconds.
Solution. (a) Counting outcome, discrete, described by a PMF. (b) Measured on a continuum, continuous, described by a PDF.
A PMF is the full probability table of a discrete variable. This lesson shows how to read, validate, and use it.
The probability mass function of a discrete random variable X is p(x) = P(X = x). A valid PMF has two properties: every value is non-negative, p(x) >= 0, and all values sum to 1 over the support. From the PMF you get the probability of any event by adding the masses of the values in it: P(X in A) = sum over x in A of p(x). The PMF is the complete description of a discrete variable — every expectation, variance, and CDF derives from it. In data science, an estimated PMF is just a normalized histogram of categorical or count data, and many models (naive Bayes, language models) store and manipulate PMFs directly. The core skills are validating that a candidate table is a legitimate PMF and summing the right entries to answer a question.
Worked Example 1
Problem. X has p(0)=0.2, p(1)=0.5, p(2)=0.3. Verify it is a valid PMF and find P(X >= 1).
Answer. Valid PMF; P(X >= 1) = 0.8.
Worked Example 2
Problem. A PMF is p(x) = c*x for x = 1,2,3,4. Find c.
Answer. c = 0.1.
Worked Example 3
Problem. For the binomial(n=3, p=0.5), give the PMF and P(X = 2).
Answer. P(X = 2) = 3/8 = 0.375.
Problem. X takes values 1,2,3 with p(1)=0.5, p(2)=k, p(3)=0.2. Find k and P(X <= 2).
Solution. Sum to 1: 0.5 + k + 0.2 = 1, so k = 0.3. P(X <= 2) = p(1) + p(2) = 0.5 + 0.3 = 0.8.
For continuous variables, probability comes from area, not point values. This lesson explains the density function and how to integrate it.
A continuous random variable is described by a probability density function f(x), a non-negative curve whose total area equals 1. Unlike a PMF, f(x) is not a probability — it can exceed 1 — it is a density, and probability comes from area: P(a <= X <= b) = integral from a to b of f(x) dx. Because a single point has no width, P(X = c) = 0, so for continuous variables the inequalities <= and < give the same probability. The height f(x) tells you relative likelihood: where the density is larger, values cluster more. In data science, densities underlie kernel density estimation, likelihoods in maximum-likelihood fitting, and the smooth curves we overlay on histograms. The essential skill is integrating the density (analytically or numerically) over the interval of interest.
Worked Example 1
Problem. X has density f(x) = 2x on [0,1]. Verify it integrates to 1 and find P(X <= 0.5).
Answer. Valid density; P(X <= 0.5) = 0.25.
Worked Example 2
Problem. For the uniform density on [0,4], find P(1 <= X <= 3).
Answer. P(1 <= X <= 3) = 0.5.
Worked Example 3
Problem. Explain why a density value f(2) = 1.5 is allowed, then for f(x)=2x on [0,1] state P(X = 0.5).
Answer. Densities can exceed 1; P(X = 0.5) = 0.
Problem. X has density f(x) = 3x^2 on [0,1]. Find P(X <= 0.5).
Solution. Integral of 3x^2 from 0 to 0.5 = [x^3] from 0 to 0.5 = 0.125. So P(X <= 0.5) = 0.125.
The CDF is one function that works for any random variable, discrete or continuous. This lesson defines it and shows how to extract probabilities.
The cumulative distribution function is F(x) = P(X <= x). It is defined for every random variable, which makes it the universal description of a distribution. A CDF is non-decreasing, approaches 0 as x goes to minus infinity and 1 as x goes to plus infinity, and is right-continuous. For a discrete variable F is a step function that jumps by p(x) at each value; for a continuous variable F is smooth and its derivative is the density. From the CDF you compute interval probabilities by subtraction: P(a < X <= b) = F(b) - F(a). The inverse of the CDF gives quantiles, which underlie percentiles, the inverse-transform sampling method, and confidence intervals. In data science the empirical CDF — the fraction of data at most x — is a fundamental nonparametric summary of a sample.
Worked Example 1
Problem. X is uniform on [0,10]. Find F(3) and P(2 < X <= 6).
Answer. F(3) = 0.3; P(2 < X <= 6) = 0.4.
Worked Example 2
Problem. A discrete X has p(0)=0.2, p(1)=0.5, p(2)=0.3. Give F(x) at x = 0, 1, 2 and find P(X <= 1).
Answer. F(0)=0.2, F(1)=0.7, F(2)=1.0; P(X <= 1) = 0.7.
Worked Example 3
Problem. For a standard normal, use the CDF to find P(-1 < Z <= 1).
Answer. P(-1 < Z <= 1) ≈ 0.6826.
Problem. For an exponential variable with rate 1, F(x) = 1 - e^(-x). Find P(X <= 2).
Solution. F(2) = 1 - e^(-2) = 1 - 0.1353 = 0.8647. So P(X <= 2) ≈ 0.865.
Applying a function to a random variable creates a new one with its own distribution. This lesson shows how transformations reshape distributions.
If X is a random variable and g is a function, then Y = g(X) is a new random variable whose distribution is induced by X's. For a discrete X, the PMF of Y collects the masses of all x mapping to each y. For a continuous, monotonic g, the change-of-variables formula gives the density of Y as f_X(x) / |g'(x)|, evaluated at x = g^{-1}(y); the derivative factor accounts for how g stretches or compresses the scale. Linear transformations Y = aX + b shift and rescale without changing shape, which is exactly how standardization (z = (X - mu)/sigma) and de-standardization work. In data science, transformations such as log, square root, and Box-Cox are used to stabilize variance or make skewed data more normal; understanding how they change the distribution is essential to interpreting the results.
Worked Example 1
Problem. X is uniform on {1,2,3} and Y = X^2. Find the PMF of Y.
Answer. Y in {1,4,9}, each with probability 1/3.
Worked Example 2
Problem. X ~ Normal(mu=10, sigma=2). Describe the distribution of Y = (X - 10)/2.
Answer. Y ~ Normal(0, 1), the standard normal.
Worked Example 3
Problem. X is uniform on [0,1] and Y = -ln(X). Find the density of Y.
Answer. Y ~ Exponential(rate 1), density e^{-y} for y >= 0.
Problem. X is uniform on [0,1]. Find the density of Y = 2X + 1 and its range.
Solution. As X ranges over [0,1], Y ranges over [1,3]. Since Y = 2X+1 is linear with slope 2, f_Y(y) = f_X(x)/|2| = 1/2 on [1,3]. So Y is uniform on [1,3].
Pick one discrete and one continuous random variable. In Python or R, draw many samples from each, then construct an empirical CDF and overlay it on the theoretical CDF. For the continuous case, also estimate the PDF with a histogram or kernel density and compare it to the true density. Comment on how sample size affects the match.
Deliverable · A script and two figures (empirical vs. theoretical CDF, histogram vs. PDF) with a short note on convergence as sample size grows.
1. For a continuous random variable, what is P(X = c) for a single point c?
Answer C. Continuous variables assign probability via area under the PDF; a single point has zero width and therefore zero probability.
2. Which property must every CDF satisfy?
Answer A. A CDF F(x) = P(X <= x) never decreases and tends to 0 at minus infinity and 1 at plus infinity. Integrating to 1 is a PDF property.
3. A valid PMF for a discrete variable must have probabilities that:
Answer B. A PMF lists P(X = x) values that are each >= 0 and sum to exactly 1; integration applies to densities, not mass functions.
4. How do you get a probability over an interval from a PDF?
Answer C. For continuous variables, P(a <= X <= b) is the integral of the PDF from a to b — the area under the curve.
5. A random variable is best described as:
Answer B. A random variable assigns a number to every outcome, turning qualitative results into quantities we can analyze.
I can map experiment outcomes to a random variable and choose between discrete and continuous models.
I can read and use PMFs, PDFs, and CDFs to compute probabilities.
I can describe how a transformation changes a random variable's distribution.
Every yes/no event has a one-trial random variable behind it. This lesson introduces the Bernoulli distribution and the indicator trick.
A Bernoulli random variable models a single trial with two outcomes: it equals 1 (success) with probability p and 0 (failure) with probability 1 - p. Its mean is E[X] = p and its variance is Var(X) = p(1 - p), which is largest at p = 0.5. The indicator variable I_A equals 1 when event A occurs and 0 otherwise, so it is a Bernoulli with p = P(A); crucially E[I_A] = P(A), turning probabilities into expectations. Indicators are the atom from which other distributions are built: a binomial is a sum of independent Bernoullis. In data science Bernoulli variables encode binary labels, conversion events, and clicks, and the indicator-equals-probability identity is the trick behind linearity-of-expectation proofs and Monte Carlo probability estimates.
Worked Example 1
Problem. A biased coin has P(heads) = 0.3. Let X = 1 for heads. Find E[X] and Var(X).
Answer. E[X] = 0.3, Var(X) = 0.21.
Worked Example 2
Problem. Roll a die. Let I = 1 if the roll is a 6. Show E[I] = P(roll = 6).
Answer. E[I] = 1/6 = P(roll = 6).
Worked Example 3
Problem. For which p is a Bernoulli's variance maximized, and what is that maximum?
Answer. Maximized at p = 0.5 with variance 0.25.
Problem. A free-throw shooter makes 80% of shots. Let X = 1 for a make. Find E[X] and Var(X).
Solution. X ~ Bernoulli(0.8). E[X] = 0.8. Var(X) = 0.8 * 0.2 = 0.16.
Count the successes in a fixed number of independent trials and you get the binomial. This lesson develops its PMF and moments.
The binomial distribution counts the number of successes X in n independent Bernoulli trials each with success probability p. Its PMF is P(X = k) = C(n,k) p^k (1-p)^{n-k}, where C(n,k) counts which k of the n trials succeed, and the powers give the probability of any specific such arrangement. The mean is E[X] = np and the variance is Var(X) = np(1-p), both following from viewing X as a sum of n independent Bernoullis. The four binomial assumptions are: fixed n, independent trials, constant p, and two outcomes per trial. In data science the binomial models conversions out of n visitors, defective items per batch, and the numerator of an estimated proportion; it is the backbone of proportion tests and A/B testing math.
Worked Example 1
Problem. Flip a fair coin 5 times. Find P(exactly 3 heads).
Answer. P(3 heads) = 10/32 = 0.3125.
Worked Example 2
Problem. A factory's parts are defective with p = 0.1. In a batch of 20, find the expected number of defects and the variance.
Answer. E[X] = 2 defects, Var(X) = 1.8.
Worked Example 3
Problem. A quiz has 4 multiple-choice questions, each with 4 options answered by guessing (p=0.25). Find P(at least 1 correct).
Answer. P(at least 1 correct) ≈ 0.684.
Problem. A basketball player makes 70% of free throws. Out of 10 attempts, find P(exactly 8 made).
Solution. X ~ Binomial(10, 0.7). P(X=8) = C(10,8)(0.7)^8(0.3)^2 = 45 * 0.05765 * 0.09 ≈ 0.2335.
How many tries until the first success? The geometric distribution answers that, with a famous memoryless twist.
The geometric distribution models the number of independent Bernoulli trials X until (and including) the first success, where each trial succeeds with probability p. Its PMF is P(X = k) = (1-p)^{k-1} p for k = 1, 2, 3, ...: k-1 failures followed by a success. The mean is E[X] = 1/p and the variance is (1-p)/p^2. Its defining feature is memorylessness: given that the first k trials all failed, the distribution of additional trials needed is the same as from the start, P(X > m + n | X > m) = P(X > n). In data science the geometric models the number of attempts until an event (a click, a conversion, a system failure) and appears in retry logic, churn timing, and as the discrete analogue of the exponential distribution.
Worked Example 1
Problem. Roll a die until the first 6. Find P(first 6 on the 3rd roll) and the expected number of rolls.
Answer. P(X=3) ≈ 0.116; expected 6 rolls.
Worked Example 2
Problem. A sales rep closes 20% of calls. Find P(first sale within the first 3 calls).
Answer. P(first sale by call 3) = 0.488.
Worked Example 3
Problem. A machine fails on any day with p = 0.05. Given it has run 10 days without failure, what is P(it runs at least 5 more days)?
Answer. ≈ 0.774, independent of the 10 days already survived.
Problem. A gachapon machine gives a rare prize with probability 0.1 per try. What is the expected number of tries to get it, and P(getting it on exactly the 2nd try)?
Solution. X ~ Geometric(0.1). E[X] = 1/0.1 = 10 tries. P(X=2) = (0.9)^1 (0.1) = 0.09.
When rare events arrive at a steady average rate, counts follow the Poisson distribution. This lesson develops it and its rate parameter.
The Poisson distribution models the number of events occurring in a fixed interval of time or space when events happen independently at a constant average rate lambda. Its PMF is P(X = k) = e^{-lambda} lambda^k / k! for k = 0, 1, 2, .... A distinctive property is that the mean and variance are both equal to lambda, which provides a quick diagnostic check on count data. The Poisson arises as the limit of a binomial when n is large and p is small with np = lambda fixed, so it approximates rare-event counts. In data science the Poisson models website hits per minute, defects per unit, support tickets per day, and is the basis of Poisson regression for count outcomes. The key skill is identifying lambda for the relevant interval and scaling it when the interval length changes.
Worked Example 1
Problem. A call center receives 3 calls per minute on average. Find P(exactly 5 calls in a minute).
Answer. P(5 calls) ≈ 0.101.
Worked Example 2
Problem. Same center (3/min). Find P(no calls in a 2-minute window).
Answer. P(0 in 2 min) = e^{-6} ≈ 0.00248.
Worked Example 3
Problem. Defects occur at 0.5 per meter of cable. Find P(at least 1 defect in a 4-meter cable).
Answer. P(at least 1 defect) ≈ 0.865.
Problem. Emails arrive at an average of 4 per hour. Find P(exactly 2 emails in the next 30 minutes).
Solution. For 30 minutes lambda = 4 * 0.5 = 2. P(X=2) = e^{-2} 2^2 / 2! = 0.1353 * 4 / 2 = 0.2707.
Two more workhorses: equal-chance values and sampling without replacement. This lesson covers the discrete uniform and hypergeometric.
The discrete uniform distribution assigns equal probability 1/n to each of n values, such as the faces of a fair die; its mean is the midpoint of the range. The hypergeometric distribution models the number of successes when drawing n items without replacement from a finite population of N items containing K successes. Its PMF is P(X = k) = C(K,k) C(N-K, n-k) / C(N, n). Because draws are without replacement, trials are dependent and the success probability shifts after each draw, distinguishing it from the binomial. When the population is large relative to the sample, the hypergeometric is well approximated by a binomial. In data science the hypergeometric underlies the math of card and lottery problems, defect sampling from finite lots, and Fisher's exact test for 2x2 contingency tables.
Worked Example 1
Problem. A fair die shows values 1-6. State the distribution and find the mean.
Answer. Discrete uniform; mean 3.5.
Worked Example 2
Problem. From a deck of 52 (4 aces), draw 5 cards. Find P(exactly 2 aces).
Answer. P(exactly 2 aces) ≈ 0.0399.
Worked Example 3
Problem. A box has 10 items, 3 defective. Draw 4 without replacement. Find P(no defectives).
Answer. P(no defectives) = 1/6 ≈ 0.167.
Problem. A lottery has 49 numbers; 6 are winners. You pick 6. Find P(matching exactly 3 winners).
Solution. N=49, K=6, n=6, k=3. P = C(6,3) C(43,3) / C(49,6) = 20 * 12341 / 13983816 = 246820/13983816 ≈ 0.0177.
Knowing the distributions is half the battle; matching one to a scenario is the skill. This lesson gives a decision framework.
Selecting a discrete model starts with the question being asked. Ask: Is it a single yes/no trial? Bernoulli. A count of successes in a fixed number of independent trials with constant p? Binomial. The number of trials until the first success? Geometric. A count of events in a fixed interval at a steady rate, with no fixed n? Poisson. Successes drawn without replacement from a finite population? Hypergeometric. Equal chances among finitely many values? Discrete uniform. Two practical checks help: compare the sample mean and variance (equal suggests Poisson; variance < mean over n suggests binomial), and verify the independence and constant-rate assumptions. In data science this choice determines the likelihood used for fitting, so a mismatch biases parameter estimates and downstream predictions. When in doubt, fit candidates, simulate from each, and compare to the data.
Worked Example 1
Problem. Out of 200 website visitors, 18 signed up. Which model fits the count of sign-ups, and what is the estimated parameter?
Answer. Binomial(n=200, p≈0.09).
Worked Example 2
Problem. A server logs the number of errors each hour over many hours; the mean is 2.0 and the variance is 2.1. Which model is suggested?
Answer. Poisson with lambda ≈ 2 (mean ≈ variance).
Worked Example 3
Problem. You record how many calls a sales rep makes before the first sale, repeated over many days. Which model?
Answer. Geometric, with p ≈ 1/mean(calls-to-first-sale).
Problem. A quality inspector draws 5 widgets without replacement from a crate of 50 (which contains 8 defectives) and counts defectives. Which distribution and parameters?
Solution. Without-replacement draws from a finite population make this hypergeometric: N=50, K=8 defectives, n=5 draws. So X ~ Hypergeometric(N=50, K=8, n=5).
Choose a real or synthetic count dataset (for example, emails per hour or defects per batch). In Python (SciPy) or R, plot a histogram, then fit and overlay both a binomial and a Poisson model. Use simulation to draw samples from your chosen model and compare to the data. Argue which model fits better and why, referencing the mean-equals-variance check for Poisson.
Deliverable · A script with an annotated figure and a short justification of the chosen distribution and its estimated parameters.
1. You flip a fair coin 10 times. The number of heads follows which distribution?
Answer C. A fixed number of independent trials with constant success probability is the binomial setting, here n=10 and p=0.5.
2. For a Poisson distribution with rate lambda, the mean and variance are:
Answer B. A defining feature of the Poisson distribution is that its mean and variance both equal lambda.
3. Which scenario is best modeled by a geometric distribution?
Answer B. The geometric distribution counts the number of independent trials needed to obtain the first success.
4. When should you use the hypergeometric instead of the binomial?
Answer B. The hypergeometric models successes drawn without replacement, so trials are dependent; the binomial assumes replacement/independence.
5. A Bernoulli random variable can take which values?
Answer B. A Bernoulli trial has exactly two outcomes coded as 1 (success) and 0 (failure).
I can identify which discrete distribution fits a counting or trial-based scenario.
I can compute probabilities and parameters for binomial, geometric, and Poisson models.
I can use SciPy or R to evaluate PMFs, CDFs, and simulate discrete data.
The simplest continuous model spreads probability evenly over an interval. This lesson covers the continuous uniform distribution.
The continuous uniform distribution on [a, b] has a constant density f(x) = 1/(b - a) for a <= x <= b and 0 elsewhere, so every equal-length sub-interval is equally likely. Its CDF rises linearly: F(x) = (x - a)/(b - a) on the interval. The mean is the midpoint (a + b)/2 and the variance is (b - a)^2 / 12. Probabilities are just ratios of lengths: P(c <= X <= d) = (d - c)/(b - a). The standard uniform on [0, 1] is especially important because it is the raw material of random number generation: applying the inverse CDF of any distribution to a Uniform(0,1) draw produces a sample from that distribution (inverse-transform sampling). In data science the uniform models 'no preference' priors, randomization in experiments, and is the seed for simulating every other distribution.
Worked Example 1
Problem. X is uniform on [0, 20]. Find P(X <= 5) and the mean.
Answer. P(X <= 5) = 0.25; mean = 10.
Worked Example 2
Problem. A bus arrives uniformly between minutes 0 and 30 after you arrive. Find P(you wait between 10 and 20 minutes).
Answer. P(10 to 20 min) = 1/3 ≈ 0.333.
Worked Example 3
Problem. X is uniform on [2, 8]. Find its variance.
Answer. Var(X) = 3.
Problem. A random number generator returns X uniform on [0, 1]. Find P(0.2 <= X <= 0.7) and the mean.
Solution. Density is 1 on [0,1]. P(0.2 <= X <= 0.7) = 0.7 - 0.2 = 0.5. Mean = (0+1)/2 = 0.5.
How long until the next event? The exponential distribution models waiting times in a steady stream of events.
The exponential distribution models the waiting time until the next event in a Poisson process with rate lambda. Its density is f(x) = lambda e^{-lambda x} for x >= 0, and its CDF is F(x) = 1 - e^{-lambda x}. The mean waiting time is 1/lambda and the variance is 1/lambda^2. Like the geometric, the exponential is memoryless: P(X > s + t | X > s) = P(X > t), so a component that has already lasted s hours has the same remaining-life distribution as a new one. The exponential is the continuous counterpart of the geometric and is tightly linked to the Poisson: if events arrive Poisson with rate lambda, the gaps between them are Exponential(lambda). In data science it models inter-arrival times, time-to-failure, and session durations, and is the simplest survival-analysis baseline.
Worked Example 1
Problem. Customers arrive at rate 2 per hour. Find the mean wait until the next customer and P(wait > 1 hour).
Answer. Mean 0.5 h; P(wait > 1 h) ≈ 0.135.
Worked Example 2
Problem. A bulb's lifetime is exponential with mean 1000 hours. Find P(it lasts less than 500 hours).
Answer. P(X < 500) ≈ 0.393.
Worked Example 3
Problem. The bulb has already lasted 1000 hours. Find P(it lasts at least 500 more hours).
Answer. ≈ 0.607, independent of the 1000 hours already used.
Problem. Calls arrive at a help desk at 5 per hour. Find the probability the next call comes within 6 minutes.
Solution. lambda = 5 per hour; 6 min = 0.1 hour. P(X <= 0.1) = 1 - e^{-5*0.1} = 1 - e^{-0.5} = 1 - 0.6065 = 0.3935.
The bell curve is the most important distribution in statistics. This lesson introduces the normal and its parameters.
The normal (Gaussian) distribution is the symmetric bell-shaped density defined by its mean mu (the center) and standard deviation sigma (the spread): f(x) = (1/(sigma*sqrt(2*pi))) exp(-(x - mu)^2 / (2 sigma^2)). It is unbounded in both directions, symmetric about mu, and follows the 68-95-99.7 empirical rule: about 68%, 95%, and 99.7% of probability lies within 1, 2, and 3 standard deviations of the mean. Its prominence comes from the Central Limit Theorem, which makes sums and averages of many independent variables approximately normal regardless of their original shape. In data science the normal models measurement error, is the assumed noise in linear regression, underlies confidence intervals and z-tests, and is the default prior and likelihood in countless methods. Probabilities are read from its CDF, usually via software or z-tables.
Worked Example 1
Problem. Heights are Normal(mu=170 cm, sigma=10 cm). Find P(height between 160 and 180 cm).
Answer. ≈ 0.68 (68% within one standard deviation).
Worked Example 2
Problem. Test scores are Normal(mu=500, sigma=100). Find P(score > 650).
Answer. P(score > 650) ≈ 0.0668.
Worked Example 3
Problem. For the same scores, find P(400 <= score <= 600).
Answer. P(400 to 600) ≈ 0.683.
Problem. IQ scores are Normal(mu=100, sigma=15). What fraction of people score above 130?
Solution. z = (130 - 100)/15 = 2.0. P(Z > 2) = 1 - 0.9772 = 0.0228, about 2.3% (consistent with the 95% within 2 sigma rule).
Any normal can be converted to the standard normal by a z-score. This lesson shows standardization and why it is useful.
Standardization rescales a value to a z-score: z = (x - mu)/sigma, the number of standard deviations x lies from the mean. If X is Normal(mu, sigma), then Z = (X - mu)/sigma is the standard normal, Normal(0, 1). This lets a single table or function answer probability questions for any normal: convert to z, look up the standard normal CDF, and you are done. Z-scores also make values from different distributions comparable on a common scale and are the basis of outlier detection (|z| large) and feature scaling in machine learning. The inverse operation, x = mu + z*sigma, converts a desired percentile back to the original units. In data science standardization is a routine preprocessing step so that variables with different units contribute comparably to models and distances.
Worked Example 1
Problem. X ~ Normal(mu=50, sigma=8). Convert x = 66 to a z-score and interpret it.
Answer. z = 2.0 (two standard deviations above the mean).
Worked Example 2
Problem. Two learners: Anna scored 85 on a test with mu=78, sigma=5; Ben scored 90 on a test with mu=82, sigma=10. Who did relatively better?
Answer. Anna (z=1.4) outperformed Ben (z=0.8) relatively.
Worked Example 3
Problem. Find the value at the 90th percentile of Normal(mu=100, sigma=15).
Answer. 90th percentile ≈ 119.2.
Problem. Reaction times are Normal(mu=250 ms, sigma=40 ms). A reading of 330 ms has what z-score, and what is P(time > 330)?
Solution. z = (330 - 250)/40 = 2.0. P(Z > 2) = 1 - 0.9772 = 0.0228, about 2.3%.
Beyond the normal, two flexible densities cover skewed positive quantities and bounded proportions. This lesson introduces the gamma and beta.
The gamma distribution is a flexible right-skewed density on positive values with a shape parameter k (or alpha) and a scale theta; it generalizes the exponential (the case k = 1) and models the waiting time until the k-th event in a Poisson process. Its mean is k*theta. The beta distribution lives on [0, 1] with two shape parameters alpha and beta, giving densities that can be uniform, bell-shaped, U-shaped, or skewed; its mean is alpha/(alpha + beta). The beta is the natural model for probabilities and proportions and is the conjugate prior for the binomial, so a Beta prior updated with binomial data yields a Beta posterior — the workhorse of Bayesian A/B testing. In data science gamma models positive, skewed quantities (insurance claims, durations) and beta models rates, click-through probabilities, and uncertainty about a proportion.
Worked Example 1
Problem. A Gamma with shape k=2 and scale theta=3 models a waiting time. Find its mean.
Answer. Mean = 6.
Worked Example 2
Problem. A click-through rate has a Beta(alpha=8, beta=2) belief. Find the mean estimate of the rate.
Answer. Mean estimated rate = 0.8.
Worked Example 3
Problem. Start with a Beta(1,1) (uniform) prior for a conversion rate. After observing 6 conversions in 10 trials, give the posterior.
Answer. Posterior Beta(7, 5), mean ≈ 0.583.
Problem. You hold a Beta(2,2) prior for a coin's heads probability and then observe 7 heads in 10 flips. What is the posterior and its mean?
Solution. Posterior = Beta(2+7, 2+3) = Beta(9, 5). Mean = 9/(9+5) = 9/14 ≈ 0.643.
Software turns distribution theory into draws and percentiles. This lesson covers sampling, the four function families, and quantiles.
Most distributions in SciPy and R expose four operations. The PDF/PMF (scipy .pdf/.pmf, R d*) gives density or mass; the CDF (.cdf, R p*) gives P(X <= x); the quantile/inverse-CDF (.ppf, R q*) maps a probability to a value (a percentile); and the random sampler (.rvs, R r*) draws samples. The quantile function is the inverse of the CDF: ppf(0.5) is the median, ppf(0.9) the 90th percentile. A key idea is inverse-transform sampling: if U is Uniform(0,1), then F^{-1}(U) is a draw from the distribution with CDF F, which is how samplers are built. An empirical quantile from a sample (numpy.quantile or R quantile) estimates these percentiles directly from data. In data science these four operations power simulation, confidence-interval endpoints, risk measures (value-at-risk is a quantile), and reproducible experiments via a fixed seed.
Worked Example 1
Problem. Find the 95th percentile of a standard normal distribution.
Answer. 95th percentile ≈ 1.645.
Worked Example 2
Problem. Use inverse-transform sampling to turn a uniform draw u = 0.5 into an Exponential(rate 1) sample.
Answer. Sample x = -ln(0.5) ≈ 0.693.
Worked Example 3
Problem. Find the interquartile range (IQR) of a Normal(mu=0, sigma=1).
Answer. IQR ≈ 1.349 standard deviations.
Problem. For Normal(mu=100, sigma=15), find the value below which 25% of the distribution falls (the first quartile).
Solution. Q1 = ppf(0.25). Standard normal ppf(0.25) ≈ -0.6745, so x = 100 + (-0.6745)(15) ≈ 89.88. About a quarter of values fall below ~89.9.
In Python or R, draw a large sample from a normal distribution with a chosen mean and standard deviation. Compute z-scores, then verify empirically that about 68%, 95%, and 99.7% of the data fall within 1, 2, and 3 standard deviations of the mean. Also use the CDF/quantile functions to find the value at the 90th percentile and confirm it by counting samples.
Deliverable · A script reporting the empirical 68-95-99.7 percentages, the computed 90th percentile, and a histogram with the fitted normal curve.
1. Approximately what fraction of a normal distribution lies within 2 standard deviations of the mean?
Answer C. The empirical (68-95-99.7) rule says about 95% of values fall within 2 standard deviations of the mean.
2. A z-score of 2.0 means the observation is:
Answer B. A z-score is (x - mu)/sigma, so z = 2.0 places the value two standard deviations above the mean.
3. Which distribution is the natural model for the waiting time until the next event in a Poisson process?
Answer C. The exponential distribution describes the time between independent events occurring at a constant rate, and it is memoryless.
4. For a continuous uniform distribution on [0, 10], what is P(X <= 3)?
Answer B. Density is constant, so the probability is the length of the favorable interval over the total: 3/10 = 0.3.
5. Standardizing a normal variable produces a distribution with:
Answer B. Subtracting the mean and dividing by sigma yields the standard normal Z with mean 0 and standard deviation 1.
I can identify which continuous distribution fits a measurement or waiting-time scenario.
I can standardize values and use the normal distribution to find probabilities and quantiles.
I can use SciPy or R to evaluate PDFs, CDFs, and draw continuous samples.
The expected value is the distribution's center of mass — the long-run average. This lesson defines it for both discrete and continuous variables.
The expected value E[X] is the probability-weighted average of a random variable's values, representing its long-run mean over many repetitions. For a discrete variable, E[X] = sum over x of x * p(x); for a continuous variable, E[X] = integral of x * f(x) dx over the support. Geometrically it is the balance point of the distribution. The expectation need not be an attainable value (a die's expectation is 3.5) and, for some heavy-tailed distributions, may not exist. In data science E[X] is the theoretical target that a sample mean estimates by the Law of Large Numbers; it defines fair prices in decision-making, the baseline prediction that minimizes squared error, and the first moment used in method-of-moments estimation. Computing it correctly is the foundation for variance, covariance, and nearly every downstream summary.
Worked Example 1
Problem. A fair six-sided die is rolled. Find E[X].
Answer. E[X] = 3.5.
Worked Example 2
Problem. A game pays $10 with probability 0.2 and costs $3 otherwise. Find the expected payoff.
Answer. Expected payoff = -$0.40 (a losing game).
Worked Example 3
Problem. X has density f(x) = 2x on [0,1]. Find E[X].
Answer. E[X] = 2/3 ≈ 0.667.
Problem. A lottery ticket wins $100 with probability 0.01 and nothing otherwise. The ticket costs $2. What is the expected net gain?
Solution. Expected winnings = 100(0.01) + 0(0.99) = 1. Net of the $2 cost: E[net] = 1 - 2 = -$1. On average you lose a dollar per ticket.
Expectation passes through sums and constants with no strings attached. This lesson establishes its most powerful property: linearity.
Linearity of expectation says E[aX + bY + c] = a E[X] + b E[Y] + c for any constants and any random variables X and Y — and crucially, it holds whether or not X and Y are independent. This is what makes expectation so useful: hard quantities can be broken into a sum of simple indicator variables whose expectations are easy probabilities, then added. The standard trick is to write a count as a sum of indicators (X = I_1 + ... + I_n) and use E[X] = sum of E[I_i] = sum of P(A_i). In data science, linearity lets us compute expected counts (expected number of matches, collisions, or successes) without wrestling with the joint distribution, and it underlies unbiasedness arguments for estimators. The independence-free nature is the property learners most often underuse.
Worked Example 1
Problem. Roll two dice. Find the expected value of the sum.
Answer. E[sum] = 7.
Worked Example 2
Problem. If E[X] = 4, find E[3X + 5].
Answer. E[3X + 5] = 17.
Worked Example 3
Problem. In a class of 30 learners with random birthdays, find the expected number of learners born on a Monday (1/7 chance each).
Answer. Expected Monday births ≈ 4.29.
Problem. A fair coin is flipped 100 times. Using indicators, find the expected number of heads.
Solution. Let I_i = 1 if flip i is heads, E[I_i] = 0.5. X = sum of 100 indicators, so E[X] = 100 * 0.5 = 50 heads.
The mean tells you the center; variance tells you the spread. This lesson defines variance and its square-root partner, the standard deviation.
Variance measures how far a random variable typically falls from its mean: Var(X) = E[(X - mu)^2], with the convenient computing form Var(X) = E[X^2] - (E[X])^2. The standard deviation sigma = sqrt(Var(X)) restores the original units, making spread interpretable. Key algebra: Var(aX + b) = a^2 Var(X) (shifts do not change spread; scaling squares), and for independent variables Var(X + Y) = Var(X) + Var(Y). Unlike expectation, variance is not linear and the additivity of variances requires independence (or adding twice the covariance otherwise). In data science variance quantifies risk and noise, drives the bias-variance tradeoff, sets confidence-interval widths through the standard error sigma/sqrt(n), and is the quantity minimized or controlled in many estimators and portfolios.
Worked Example 1
Problem. X takes values 0 and 10 each with probability 0.5. Find Var(X) and the standard deviation.
Answer. Var(X) = 25, SD = 5.
Worked Example 2
Problem. A fair die has Var(X) = 35/12. Find Var(2X + 3).
Answer. Var(2X + 3) = 35/3 ≈ 11.67.
Worked Example 3
Problem. X and Y are independent with Var(X)=4 and Var(Y)=9. Find Var(X + Y) and Var(X - Y).
Answer. Var(X+Y) = Var(X-Y) = 13.
Problem. A Bernoulli variable has p = 0.5. Find its variance and standard deviation.
Solution. Var = p(1-p) = 0.5 * 0.5 = 0.25. SD = sqrt(0.25) = 0.5.
To find the expected value of a function of X, you do not need the distribution of that function. LOTUS lets you compute it directly.
The law of the unconscious statistician (LOTUS) states that for a function g, E[g(X)] = sum over x of g(x) p(x) for discrete X, or integral of g(x) f(x) dx for continuous X. The point is that you can compute the expectation of g(X) using the distribution of X alone, without first deriving the (often messy) distribution of Y = g(X). This is how variance itself is computed: Var(X) = E[(X - mu)^2] applies LOTUS with g(x) = (x - mu)^2. A critical consequence is that for nonlinear g, E[g(X)] is generally not g(E[X]) — Jensen's inequality says E[g(X)] >= g(E[X]) for convex g. In data science LOTUS computes expected utilities, expected loss/risk, and moments, and the gap between E[g(X)] and g(E[X]) explains many averaging fallacies.
Worked Example 1
Problem. X is a fair die. Find E[X^2] using LOTUS.
Answer. E[X^2] = 91/6 ≈ 15.17.
Worked Example 2
Problem. Using the previous result, find Var(X) for a fair die via LOTUS.
Answer. Var(X) = 35/12 ≈ 2.917.
Worked Example 3
Problem. X is uniform on [0,1]. Find E[e^X] and compare to e^{E[X]}.
Answer. E[e^X] = e - 1 ≈ 1.718 > e^{0.5} ≈ 1.649.
Problem. A fair coin pays X = 1 for heads, 0 for tails. Find E[X^2] and confirm it equals Var plus mean-squared for this Bernoulli.
Solution. E[X^2] = 1^2(0.5) + 0^2(0.5) = 0.5. Since E[X] = 0.5, Var = E[X^2] - (E[X])^2 = 0.5 - 0.25 = 0.25, matching p(1-p).
When two variables move together, covariance and correlation quantify it. This lesson defines both and explains why correlation is the standardized version.
Covariance measures how two variables vary together: Cov(X, Y) = E[(X - muX)(Y - muY)] = E[XY] - E[X]E[Y]. It is positive when the variables tend to rise together, negative when one rises as the other falls, and zero when there is no linear co-movement. But covariance is scale-dependent, so its magnitude is hard to interpret. Correlation fixes this by standardizing: rho = Cov(X, Y) / (sigmaX * sigmaY), a unit-free number in [-1, 1] where +/-1 means a perfect linear relationship and 0 means none. A crucial caveat: independence implies zero correlation, but zero correlation does not imply independence — correlation captures only linear association. In data science correlation drives feature selection, multicollinearity diagnostics, and is the building block of the covariance matrices used in PCA and portfolio optimization.
Worked Example 1
Problem. X and Y have E[X]=2, E[Y]=3, E[XY]=8. Find Cov(X,Y).
Answer. Cov(X, Y) = 2.
Worked Example 2
Problem. Continuing, if sigmaX = 1 and sigmaY = 4, find the correlation.
Answer. rho = 0.5.
Worked Example 3
Problem. Let X be symmetric about 0 (e.g. uniform on {-1,0,1}) and Y = X^2. Show Cov(X,Y) = 0 despite strong dependence.
Answer. Cov(X, Y) = 0 although X and Y are dependent.
Problem. Two assets have returns with Cov = 0.012, sigmaX = 0.1, sigmaY = 0.2. Find the correlation and interpret it.
Solution. rho = 0.012 / (0.1 * 0.2) = 0.012 / 0.02 = 0.6. A fairly strong positive linear relationship: the assets tend to move up and down together.
Moments summarize a distribution's shape, and one clever function encodes them all. This lesson introduces moments and the moment-generating function.
The n-th moment of X about the origin is E[X^n]; the first moment is the mean and the second central moment E[(X - mu)^2] is the variance. Higher central moments describe shape: the third (standardized) is skewness (asymmetry) and the fourth is kurtosis (tail heaviness). The moment-generating function (MGF) is M(t) = E[e^{tX}], and when it exists near 0 it 'generates' the moments: the n-th derivative of M evaluated at t = 0 equals E[X^n]. The MGF also uniquely determines a distribution and turns sums of independent variables into products of MGFs, which is one clean route to proving results like the Central Limit Theorem. In data science moments power method-of-moments estimation, skewness/kurtosis diagnostics for non-normality, and the theory behind many limit results.
Worked Example 1
Problem. A fair die has E[X] = 3.5 and E[X^2] = 91/6. State its first two moments and the variance.
Answer. Moments 3.5 and 15.17; variance 2.917.
Worked Example 2
Problem. For an Exponential(rate lambda), the MGF is M(t) = lambda/(lambda - t) for t < lambda. Use it to find E[X].
Answer. E[X] = 1/lambda.
Worked Example 3
Problem. Using the same MGF, find E[X^2] and Var(X) for the exponential.
Answer. E[X^2] = 2/lambda^2; Var(X) = 1/lambda^2.
Problem. The MGF of a Bernoulli(p) is M(t) = (1 - p) + p e^t. Find E[X] using M'(0).
Solution. M'(t) = p e^t, so M'(0) = p e^0 = p. Thus E[X] = p, the known Bernoulli mean.
In Python or R, simulate two correlated random variables (for example via a chosen covariance structure). Estimate their means, variances, covariance, and correlation from the samples and compare to the theoretical values. Then demonstrate linearity of expectation by checking that the sample mean of aX + bY matches a*mean(X) + b*mean(Y), even when X and Y are dependent.
Deliverable · A script reporting estimated vs. theoretical moments and correlation, plus a scatter plot illustrating the dependence.
1. Linearity of expectation, E[X + Y] = E[X] + E[Y], requires that X and Y be:
Answer C. Linearity of expectation holds for any X and Y regardless of dependence; independence is only needed for variances and products.
2. For a fair six-sided die, the expected value is:
Answer B. E[X] = (1+2+3+4+5+6)/6 = 21/6 = 3.5.
3. If Var(X) = 9, what is the standard deviation of X?
Answer C. Standard deviation is the square root of variance: sqrt(9) = 3.
4. A correlation of -1 between X and Y indicates:
Answer B. Correlation ranges from -1 to 1; a value of -1 means the points lie exactly on a downward-sloping line.
5. Covariance differs from correlation mainly because:
Answer B. Correlation normalizes covariance by the two standard deviations, removing units and bounding it to [-1, 1].
I can compute expected value and variance for discrete and continuous random variables.
I can apply linearity of expectation even when variables are dependent.
I can compute and interpret covariance and correlation between two variables.
Real problems involve several variables at once. This lesson sets up joint, marginal, and conditional distributions for two variables.
The joint distribution describes the probabilities of combinations of two (or more) random variables: a joint PMF p(x, y) = P(X = x, Y = y) for discrete variables, or a joint density f(x, y) for continuous ones, with the total summing or integrating to 1. From the joint you recover a marginal by summing or integrating out the other variable: p_X(x) = sum over y of p(x, y). The conditional distribution of Y given X = x renormalizes a slice of the joint: p(y | x) = p(x, y) / p_X(x). These three views — joint, marginal, conditional — are linked by p(x, y) = p(y | x) p_X(x). In data science the joint distribution is the full probabilistic model of several features, marginals describe single features, and conditionals are exactly what predictive models estimate, such as P(label | features).
Worked Example 1
Problem. A joint PMF over (X,Y) in {0,1}x{0,1}: p(0,0)=0.1, p(0,1)=0.2, p(1,0)=0.3, p(1,1)=0.4. Find the marginal of X.
Answer. p_X(0) = 0.3, p_X(1) = 0.7.
Worked Example 2
Problem. Using the same joint, find the conditional P(Y = 1 | X = 1).
Answer. P(Y=1 | X=1) ≈ 0.571.
Worked Example 3
Problem. Are X and Y independent in this joint distribution?
Answer. Not independent: 0.40 != 0.42.
Problem. For the joint above, find the marginal of Y and P(X = 0 | Y = 0).
Solution. p_Y(0) = p(0,0)+p(1,0) = 0.1+0.3 = 0.4. P(X=0 | Y=0) = p(0,0)/p_Y(0) = 0.1/0.4 = 0.25.
Independence of variables generalizes independence of events. This lesson states the factorization criterion and its consequences.
Two random variables X and Y are independent when their joint distribution factorizes into the product of their marginals: p(x, y) = p_X(x) p_Y(y) for all x, y (or f(x,y) = f_X(x) f_Y(y) for densities). Equivalently, the conditional distribution of one does not depend on the value of the other. Independence has powerful consequences: E[XY] = E[X]E[Y], Cov(X, Y) = 0, and Var(X + Y) = Var(X) + Var(Y). Note the one-way street: independence implies zero correlation, but zero correlation does not guarantee independence. In data science, independence (often the weaker conditional independence) is the simplifying assumption behind naive Bayes, the i.i.d. assumption justifying most estimators and the CLT, and the factorization that makes likelihoods tractable. Knowing when it holds — and the cost of assuming it wrongly — is central to sound modeling.
Worked Example 1
Problem. Two dice are rolled independently. Verify P(X=2, Y=5) factorizes.
Answer. P(X=2, Y=5) = 1/36 = P(X=2)P(Y=5).
Worked Example 2
Problem. X and Y are independent with E[X]=3, E[Y]=4, Var(X)=2, Var(Y)=5. Find E[XY] and Var(X+Y).
Answer. E[XY] = 12; Var(X+Y) = 7.
Worked Example 3
Problem. Give a case where Cov(X,Y) = 0 but X and Y are dependent.
Answer. X uniform on {-1,0,1}, Y=X^2: Cov=0 yet dependent.
Problem. X and Y are independent with Var(X) = 4, Var(Y) = 9. Find Var(2X - Y).
Solution. Var(2X - Y) = 2^2 Var(X) + (-1)^2 Var(Y) = 4*4 + 9 = 16 + 9 = 25 (no covariance term since independent).
Adding random variables combines their distributions through convolution. This lesson explains how sums behave in mean, variance, and shape.
When you add independent random variables, the means and variances simply add: E[X + Y] = E[X] + E[Y] (always) and Var(X + Y) = Var(X) + Var(Y) (when independent). The distribution of the sum, however, is the convolution of the individual distributions: P(X + Y = z) = sum over k of p_X(k) p_Y(z - k) in the discrete case, or the corresponding integral for densities. Some families are closed under addition: a sum of independent normals is normal, a sum of independent Poissons is Poisson with added rates, and a sum of n independent Bernoulli(p) is Binomial(n, p). In data science, sums and convolutions arise when aggregating independent contributions — total demand, accumulated noise, combined counts — and the closure properties let us model these aggregates exactly, while the CLT explains why many sums look normal.
Worked Example 1
Problem. X ~ Normal(2, 3) and Y ~ Normal(5, 4) are independent (means, variances). Find the distribution of X + Y.
Answer. X + Y ~ Normal(7, 7).
Worked Example 2
Problem. X ~ Poisson(3) and Y ~ Poisson(5) independent. Find the distribution of X + Y.
Answer. X + Y ~ Poisson(8).
Worked Example 3
Problem. Two independent dice X and Y. Use convolution to find P(X + Y = 4).
Answer. P(X + Y = 4) = 3/36 = 1/12.
Problem. X ~ Binomial(10, 0.5) and Y ~ Binomial(15, 0.5) are independent. What is the distribution of X + Y?
Solution. Independent binomials with the same p add their trial counts: X + Y ~ Binomial(10 + 15, 0.5) = Binomial(25, 0.5).
Why does averaging more data help? The Law of Large Numbers guarantees the sample mean settles on the truth.
The Law of Large Numbers (LLN) states that as the sample size n grows, the sample mean of independent, identically distributed observations converges to the true expected value mu. Intuitively, random highs and lows cancel out as you average more data, so the running average stabilizes near mu. This is the formal justification for estimating an unknown mean (or any expectation, since probabilities are expectations of indicators) by simulation or by collecting data: Monte Carlo estimates work precisely because of the LLN. It is important to separate the LLN from the CLT: the LLN says where the sample mean goes (to mu), while the CLT describes the shape of its fluctuations around mu. The LLN says nothing about individual outcomes — there is no 'gambler's fallacy' correction; only the average converges.
Worked Example 1
Problem. A fair die is rolled repeatedly. What does the LLN say the running average approaches?
Answer. The running average approaches 3.5.
Worked Example 2
Problem. Explain why a Monte Carlo estimate of P(sum of two dice = 7) works.
Answer. The simulated fraction converges to 1/6 by the LLN.
Worked Example 3
Problem. After 10 heads in a row, is the next flip more likely to be tails (to 'balance out')?
Answer. No — P(tails) stays 0.5; the LLN is about long-run proportions, not corrections.
Problem. You simulate flips of a fair coin and track the proportion of heads. Roughly what value does it approach as the number of flips grows very large?
Solution. The proportion of heads converges to the true probability E[I_heads] = 0.5 by the Law of Large Numbers. Individual streaks do not change this long-run limit.
The most celebrated result in probability: averages of almost anything become normal. This lesson states and interprets the CLT.
The Central Limit Theorem (CLT) says that the sum (or mean) of a large number of independent, identically distributed variables is approximately normal, regardless of the original distribution's shape, provided it has finite variance. Concretely, if X1, ..., Xn have mean mu and standard deviation sigma, then the sample mean is approximately Normal(mu, sigma^2/n), and the standardized quantity (mean - mu)/(sigma/sqrt(n)) approaches the standard normal as n grows. This is why the bell curve appears everywhere: many real quantities are sums of many small independent effects. A common rule of thumb is that n around 30 gives a good approximation for moderately skewed data, though heavy skew needs larger n. In data science the CLT justifies normal-based confidence intervals, z- and t-tests, and the standard error sigma/sqrt(n) for the mean, even when the underlying data are not normal.
Worked Example 1
Problem. A population has mu = 50, sigma = 10. For samples of size n = 100, what is the approximate distribution of the sample mean?
Answer. Sample mean ≈ Normal(mean 50, SD 1).
Worked Example 2
Problem. Using the above, find the approximate probability that the sample mean exceeds 52.
Answer. P(sample mean > 52) ≈ 0.0228.
Worked Example 3
Problem. Why does averaging 100 exponential waiting times look bell-shaped even though the exponential is heavily skewed?
Answer. The CLT makes the mean of 100 exponentials approximately normal.
Problem. Dice rolls have mu = 3.5 and sigma ≈ 1.708. For the mean of n = 50 rolls, give the approximate standard error and the distribution of the sample mean.
Solution. Standard error = sigma/sqrt(n) = 1.708/sqrt(50) ≈ 0.2415. By the CLT the sample mean is approximately Normal(3.5, 0.2415^2).
A statistic computed from a sample is itself random. This lesson introduces sampling distributions and the standard error that measures their spread.
A sampling distribution is the distribution of a statistic (such as the sample mean or proportion) across all possible samples of a given size. Because the statistic varies from sample to sample, it has its own distribution with a center and a spread. For the sample mean, the center is the population mean mu and the spread is the standard error, SE = sigma/sqrt(n) — note this is the standard deviation of the statistic, not of the raw data. The standard error shrinks as n grows, which is why larger samples give more precise estimates, and its 1/sqrt(n) rate means quadrupling the sample size halves the error. Combined with the CLT, the sampling distribution of the mean is approximately normal, which is what makes confidence intervals (estimate +/- z*SE) and hypothesis tests work. In data science the standard error quantifies estimation uncertainty for any reported statistic.
Worked Example 1
Problem. A population has sigma = 20. Find the standard error of the mean for samples of size n = 25 and n = 100.
Answer. SE = 4 (n=25) and SE = 2 (n=100).
Worked Example 2
Problem. A sample of 64 has mean 70 from a population with sigma = 16. Build an approximate 95% confidence interval for mu.
Answer. Approx 95% CI for mu = (66.08, 73.92).
Worked Example 3
Problem. A poll of n = 400 finds 60% support. Find the standard error of the sample proportion.
Answer. SE of the proportion ≈ 0.0245.
Problem. A sample of n = 36 from a population with sigma = 12 has mean 50. Find the standard error and an approximate 95% confidence interval for mu.
Solution. SE = 12/sqrt(36) = 12/6 = 2. 95% CI = 50 +/- 1.96*2 = 50 +/- 3.92 = (46.08, 53.92).
In Python or R, pick a clearly non-normal distribution (for example exponential or uniform). Repeatedly draw samples of size n, compute each sample mean, and plot the histogram of those means for several values of n (such as 1, 5, 30, 100). Overlay the normal curve predicted by the CLT and show that the standard error shrinks like sigma/sqrt(n). Also illustrate the LLN by plotting a running average converging to the true mean.
Deliverable · A script with the sequence of sample-mean histograms, a running-average plot, and a short explanation of how LLN and CLT differ.
1. The Central Limit Theorem states that the distribution of the sample mean becomes approximately normal as:
Answer B. The CLT's power is that for large n the sample mean is approximately normal even when the underlying distribution is not.
2. The Law of Large Numbers says that as sample size grows, the sample mean:
Answer B. The LLN guarantees convergence of the sample mean to the expected value; the CLT describes the shape of its fluctuations.
3. For a sample mean from a population with standard deviation sigma, the standard error is:
Answer C. The standard error of the mean is sigma/sqrt(n), so it shrinks as the sample size increases.
4. A marginal distribution is obtained from a joint distribution by:
Answer B. Marginalizing removes a variable by summing (discrete) or integrating (continuous) over all its values.
5. Two random variables X and Y are independent when their joint distribution:
Answer A. Independence means the joint factorizes as the product of marginals; zero covariance is necessary but not sufficient.
I can work with joint, marginal, and conditional distributions for two random variables.
I can explain the Law of Large Numbers and the Central Limit Theorem and when each applies.
I can demonstrate via simulation that sample means become normal as sample size grows.
Where this leads
Part of the open-source Crunch Academy tech-education path. Pair it with the data-science and ML tracks when you're ready.
All Math for Data Science courses Crunch Academy home