Math for Data Science · MDS-3
Limits, derivatives, integrals, series, and numerical methods — the change-and-accumulation toolkit that powers optimization and modeling in data science.
A computational, data-science-flavored tour of single-variable calculus. You will build intuition for limits, derivatives, and integrals, then connect them to optimization, approximation, and modeling problems you will meet in machine learning. Every unit pairs paper-and-pencil reasoning with SymPy/NumPy and Julia so the symbolic and numerical pictures reinforce each other.
Course Outline
Every lesson is self-contained: a full explanation, worked step-by-step examples (in Python and R/Julia), common mistakes, a try-it-yourself, and an interactive quiz. Jump to a unit:
A limit describes the value a function heads toward as its input creeps toward a target — even if the function is undefined exactly there.
The limit lim x→a f(x) = L means we can force f(x) as close to L as we like by taking x sufficiently close to a (but not equal to a). The key insight is that the limit ignores f(a) itself: the function may be undefined or take a different value at a, yet still have a limit. Formally (ε–δ), for every ε > 0 there is a δ > 0 so that 0 < |x − a| < δ implies |f(x) − L| < ε. Limits matter because they are the foundation of both the derivative (a slope as run shrinks to zero) and the integral (a sum as widths shrink to zero). Intuitively, you probe the function on both sides of a and ask whether the outputs agree on a single target.
Worked Example 1
Problem. Evaluate lim x→3 (2x + 1).
Answer. 7
Worked Example 2
Problem. Evaluate lim x→1 (x² − 1)/(x − 1).
Answer. 2
Worked Example 3
Problem. Estimate lim x→0 sin(x)/x numerically.
Answer. 1
Problem. Evaluate lim x→2 (x² − 4)/(x − 2).
Solution. Substitution gives 0/0. Factor: (x − 2)(x + 2)/(x − 2) = x + 2 for x ≠ 2. Limit = 2 + 2 = 4.
Most limits yield to a small toolbox: substitution, factoring, rationalizing, and numerical tables when algebra stalls.
To compute a limit, first try direct substitution; if f is continuous at a, the answer is f(a). If you hit an indeterminate form like 0/0, manipulate the expression: factor and cancel common roots, rationalize a numerator or denominator that contains radicals, or combine fractions. These moves remove the offending factor that caused the 0/0 without changing the limit (since they agree with the original for x ≠ a). When no algebraic trick is obvious, a numerical table — evaluating f at inputs marching toward a from both sides — reveals the trend. Numerics give intuition and catch errors, but they can mislead near subtraction of nearly-equal numbers, so confirm with algebra or symbolic tools when possible.
Worked Example 1
Problem. Evaluate lim x→4 (√x − 2)/(x − 4).
Answer. 1/4
Worked Example 2
Problem. Evaluate lim x→0 (1/(x+1) − 1)/x.
Answer. −1
Worked Example 3
Problem. Evaluate lim x→3 (x² − 9)/(x² − x − 6).
Answer. 6/5
Problem. Evaluate lim x→1 (x³ − 1)/(x − 1).
Solution. Factor x³ − 1 = (x − 1)(x² + x + 1). Cancel (x − 1): x² + x + 1. Substitute x = 1: 1 + 1 + 1 = 3.
One-sided limits look at approach from only the left or right; limits at infinity describe end behavior as x grows without bound.
The left-hand limit lim x→a⁻ f(x) uses only x < a; the right-hand limit lim x→a⁺ f(x) uses only x > a. The two-sided limit exists exactly when both one-sided limits exist and are equal. One-sided limits are essential at jumps, piecewise boundaries, and endpoints. Limits at infinity, lim x→±∞ f(x), describe end behavior: for a rational function, divide numerator and denominator by the highest power of x in the denominator; lower-order terms vanish, leaving the ratio of leading behaviors. If the numerator's degree is larger, the limit is ±∞; equal degrees give the ratio of leading coefficients; smaller numerator degree gives 0. A finite limit at infinity signals a horizontal asymptote.
Worked Example 1
Problem. Find lim x→0⁺ |x|/x and lim x→0⁻ |x|/x.
Answer. Right = 1, Left = −1; two-sided DNE
Worked Example 2
Problem. Find lim x→∞ (3x² + 1)/(x² − 5).
Answer. 3 (horizontal asymptote y = 3)
Worked Example 3
Problem. Find lim x→∞ (2x + 1)/(x² + 3).
Answer. 0
Problem. Find lim x→∞ (5x³ − 2x)/(2x³ + 7).
Solution. Divide by x³: (5 − 2/x²)/(2 + 7/x³). As x→∞ the small terms vanish, giving 5/2.
A function is continuous at a point when there is no break: the limit exists and matches the function value.
f is continuous at a if three conditions hold: f(a) is defined, lim x→a f(x) exists, and the two are equal. If any fails, there is a discontinuity, classified by how it fails. A removable discontinuity is a hole: the two-sided limit exists but f(a) is missing or wrong — patchable by redefining one point. A jump discontinuity has finite left and right limits that disagree, typical of piecewise functions. An infinite (essential) discontinuity occurs when the function blows up, as at a vertical asymptote. Continuity is what lets you compute limits by substitution and underpins major theorems (Intermediate Value, Extreme Value). In data work, continuous models behave predictably; discontinuities flag regime changes or undefined regions.
Worked Example 1
Problem. Classify the discontinuity of f(x) = (x² − 1)/(x − 1) at x = 1.
Answer. Removable discontinuity at x = 1
Worked Example 2
Problem. Classify f(x) = x/|x| at x = 0.
Answer. Jump discontinuity at x = 0
Worked Example 3
Problem. For what value of c is f(x) = {x² if x ≤ 2; cx + 1 if x > 2} continuous at x = 2?
Answer. c = 3/2
Problem. Is f(x) = (x² − 4)/(x − 2) continuous at x = 2? If not, classify and repair it.
Solution. f(2) is 0/0 (undefined). Factor: x + 2, limit = 4. The limit exists but f(2) is undefined → removable discontinuity. Repair by defining f(2) = 4.
The squeeze theorem traps a hard limit between two friendly ones; indeterminate forms need work before they resolve.
The squeeze (sandwich) theorem says: if g(x) ≤ f(x) ≤ h(x) near a and lim x→a g(x) = lim x→a h(x) = L, then lim x→a f(x) = L. It is the standard way to handle limits where an oscillating factor is bounded, such as x²·sin(1/x). Indeterminate forms — 0/0, ∞/∞, 0·∞, ∞ − ∞, 1^∞, 0⁰, ∞⁰ — are expressions whose limit is not decided by form alone; the true value depends on the rates involved. Resolve them by algebra (factoring, rationalizing), the squeeze theorem, known limits like sin(x)/x → 1, or L'Hôpital's rule (differentiate top and bottom for 0/0 and ∞/∞). The point: 'indeterminate' means undecided, not impossible.
Worked Example 1
Problem. Evaluate lim x→0 x²·sin(1/x).
Answer. 0
Worked Example 2
Problem. Evaluate lim x→0 (1 − cos x)/x² using L'Hôpital.
Answer. 1/2
Worked Example 3
Problem. Evaluate lim x→∞ x·sin(1/x).
Answer. 1
Problem. Evaluate lim x→0 x·cos(1/x).
Solution. Since −1 ≤ cos(1/x) ≤ 1, we have −|x| ≤ x·cos(1/x) ≤ |x|. Both bounds → 0 as x→0, so by the squeeze theorem the limit is 0.
Numerical limit estimates are powerful but limited by finite floating-point precision; tiny steps can make accuracy worse, not better.
Computers store reals as 64-bit floats with about 15–16 significant digits and a machine epsilon near 2.2×10⁻¹⁶. When you estimate a limit by plugging in x ever closer to a, two errors compete. Mathematically, smaller steps should improve the estimate (truncation error shrinks). But evaluating expressions like (f(a+h) − f(a)) for very small h subtracts nearly-equal numbers — catastrophic cancellation — so relative round-off error explodes. The total error is U-shaped in h: it falls, bottoms out at an optimal h, then rises. Practical lessons: avoid subtracting nearly-equal floats, prefer algebraically simplified forms, use symbolic tools (SymPy) for exact limits, and treat a numerical table as supporting evidence rather than proof.
Worked Example 1
Problem. Estimate lim h→0 (e^h − 1)/h (the derivative of e^x at 0, exact value 1) at h = 1e-1, 1e-8, 1e-16.
Answer. Best near h≈1e-8; fails at h=1e-16 due to round-off
Worked Example 2
Problem. Why does evaluating (1 − cos h)/h² near h = 1e-8 lose accuracy, and what is the fix?
Answer. Use 1 − cos h = 2 sin²(h/2); limit is 1/2
Problem. Using a table, estimate lim h→0 (sin h)/h with h = 0.1, 0.01, 0.001, and state the limit.
Solution. sin(0.1)/0.1 ≈ 0.998334; sin(0.01)/0.01 ≈ 0.999983; sin(0.001)/0.001 ≈ 0.99999983. Values climb toward 1, so the limit is 1. (Going far smaller, e.g. 1e-12, would start showing round-off noise.)
Pick three functions with interesting behavior (e.g., sin(x)/x at 0, (x²-1)/(x-1) at 1, and 1/x as x→∞). For each, estimate the limit numerically by tabulating f(x) at inputs marching toward the target, then confirm with sympy.limit. Note any case where the numerical table misleads you due to rounding.
Deliverable · A Python notebook with a numeric-table function, the three SymPy limit confirmations, and a short markdown note on where numerics and symbolics disagreed and why.
1. What does lim x→2 (x² - 4)/(x - 2) equal?
Answer C. Factor the numerator: (x-2)(x+2)/(x-2) = x+2 for x≠2, which approaches 2+2 = 4.
2. A function has lim x→a⁻ f(x) = 3 and lim x→a⁺ f(x) = 5. What is true at x = a?
Answer C. A two-sided limit exists only when the left and right limits are equal; here they differ, so the limit does not exist (a jump discontinuity).
3. Which scenario describes a removable discontinuity at x = a?
Answer A. A removable discontinuity is a single-point gap (or mismatch) that can be patched by redefining f(a) to equal the existing limit.
4. What is lim x→∞ (3x² + 1)/(x² - 5)?
Answer B. Dividing top and bottom by x², the lower-order terms vanish and the ratio of leading coefficients 3/1 = 3 remains.
5. By the squeeze theorem, what is lim x→0 x²·sin(1/x)?
Answer B. Since -x² ≤ x²·sin(1/x) ≤ x² and both bounds go to 0 as x→0, the squeezed function also goes to 0.
I can evaluate a limit algebraically, numerically, and graphically and say when it does not exist.
I can classify a function as continuous or identify the kind of discontinuity at a point.
I can explain why numerical limit estimates degrade due to floating-point error.
The derivative measures instantaneous rate of change — the slope of the tangent line — defined as the limit of secant-line slopes.
The derivative f'(x) is defined as lim h→0 [f(x+h) − f(x)]/h. Geometrically, the quotient [f(x+h) − f(x)]/h is the slope of the secant line through (x, f(x)) and (x+h, f(x+h)); as h shrinks toward 0, this secant rotates into the tangent line, whose slope is the instantaneous rate of change at x. The derivative exists only when this limit exists, which requires f to be continuous there and free of corners or vertical tangents. The derivative matters everywhere in data science: it is the slope a gradient-descent step follows, the sensitivity of a model output to an input, and the building block of every optimization method. Computing it from the definition trains the intuition behind every shortcut rule.
Worked Example 1
Problem. Use the limit definition to find f'(x) for f(x) = x².
Answer. f'(x) = 2x
Worked Example 2
Problem. Find f'(x) for f(x) = 1/x using the definition.
Answer. f'(x) = −1/x²
Worked Example 3
Problem. Find the slope of the tangent line to f(x) = √x at x = 4.
Answer. Slope = 1/4
Problem. Use the limit definition to find f'(x) for f(x) = 3x² − x.
Solution. Difference quotient: [3(x+h)² − (x+h) − (3x² − x)]/h = [3x² + 6xh + 3h² − x − h − 3x² + x]/h = (6xh + 3h² − h)/h = 6x + 3h − 1. Limit as h→0: 6x − 1.
Four algebraic rules let you differentiate most everyday functions without returning to the limit definition.
The power rule says d/dx[xⁿ] = n·xⁿ⁻¹ for any real n. The sum rule lets you differentiate term by term: (f ± g)' = f' ± g', and constants pull out: (c·f)' = c·f'. The product rule handles a product of two functions: (fg)' = f'g + fg' — note you cannot simply multiply derivatives. The quotient rule handles a ratio: (f/g)' = (f'g − fg')/g², valid where g ≠ 0; a memory aid is 'low d-high minus high d-low, over low squared.' These rules are derived from the limit definition but become mechanical with practice. Together they cover polynomials, rational functions, and products/quotients of any differentiable pieces — the workhorse toolkit for everything that follows.
Worked Example 1
Problem. Differentiate f(x) = 4x³ − 2x + 7.
Answer. f'(x) = 12x² − 2
Worked Example 2
Problem. Differentiate f(x) = x²·(x + 3) using the product rule.
Answer. f'(x) = 3x² + 6x
Worked Example 3
Problem. Differentiate f(x) = (x² + 1)/(x − 1) using the quotient rule.
Answer. f'(x) = (x² − 2x − 1)/(x − 1)²
Problem. Differentiate f(x) = (3x − 1)/(x² + 2).
Solution. u = 3x − 1, v = x² + 2; u' = 3, v' = 2x. Quotient rule: (3(x² + 2) − (3x − 1)(2x))/(x² + 2)² = (3x² + 6 − 6x² + 2x)/(x² + 2)² = (−3x² + 2x + 6)/(x² + 2)².
Beyond polynomials, a short table of derivatives covers the transcendental functions that appear throughout modeling.
Key derivatives to memorize: d/dx[sin x] = cos x, d/dx[cos x] = −sin x, d/dx[tan x] = sec²x. For exponentials, d/dx[eˣ] = eˣ (the function that equals its own derivative) and d/dx[aˣ] = aˣ·ln a. For logarithms, d/dx[ln x] = 1/x and d/dx[log_a x] = 1/(x·ln a). These follow from the limit definition combined with the special limits sin(h)/h → 1 and (eʰ − 1)/h → 1. They matter because exponentials model growth/decay, logs appear in log-likelihoods and entropy, and trig functions model periodic data. Combined with the product, quotient, and (next) chain rules, this table differentiates nearly every closed-form function you will meet.
Worked Example 1
Problem. Differentiate f(x) = 3sin x − 2eˣ.
Answer. f'(x) = 3cos x − 2eˣ
Worked Example 2
Problem. Differentiate f(x) = x²·ln x using the product rule.
Answer. f'(x) = 2x·ln x + x
Worked Example 3
Problem. Differentiate f(x) = (sin x)/(eˣ).
Answer. f'(x) = (cos x − sin x)/eˣ
Problem. Differentiate f(x) = eˣ·cos x.
Solution. Product rule with u = eˣ, v = cos x; u' = eˣ, v' = −sin x. Result: eˣ·cos x + eˣ·(−sin x) = eˣ(cos x − sin x).
The chain rule differentiates a composition by multiplying the outer derivative (at the inner function) by the inner derivative.
For a composite f(g(x)), the chain rule states d/dx[f(g(x))] = f'(g(x))·g'(x): differentiate the outer function, leaving the inside alone, then multiply by the derivative of the inside. In Leibniz notation, if y = f(u) and u = g(x), then dy/dx = (dy/du)·(du/dx) — the rates multiply. Recognizing 'outer' and 'inner' is the key skill: in sin(x²), the outer is sin, the inner is x². The chain rule is arguably the most important rule in all of calculus for data science: backpropagation in neural networks is the chain rule applied layer by layer, propagating gradients from the loss back through every weight. Master peeling functions from the outside in.
Worked Example 1
Problem. Differentiate f(x) = (3x + 1)⁵.
Answer. f'(x) = 15(3x + 1)⁴
Worked Example 2
Problem. Differentiate f(x) = sin(x²).
Answer. f'(x) = 2x·cos(x²)
Worked Example 3
Problem. Differentiate f(x) = e^(sin x)·(x² + 1) (chain + product).
Answer. f'(x) = e^(sin x)[(x² + 1)cos x + 2x]
Problem. Differentiate f(x) = √(x² + 1).
Solution. Rewrite as (x² + 1)^(1/2). Outer derivative: (1/2)(x² + 1)^(−1/2); inner derivative: 2x. Chain rule: (1/2)(x² + 1)^(−1/2)·2x = x/√(x² + 1).
When y is tangled with x in an equation rather than solved explicitly, differentiate both sides and solve for dy/dx.
Implicit differentiation handles relationships like x² + y² = 25 where y is not isolated. Treat y as an unknown function of x and differentiate both sides with respect to x, applying the chain rule to every y term: d/dx[y²] = 2y·(dy/dx). This produces an equation containing dy/dx, which you then solve algebraically. The technique is essential for curves that are not functions (circles, ellipses), for related-rates problems, and for differentiating inverse functions. It also yields the powerful logarithmic-differentiation trick for messy products and powers. The core idea: every time you differentiate a y, the chain rule attaches a dy/dx factor, because y depends on x.
Worked Example 1
Problem. Find dy/dx for x² + y² = 25.
Answer. dy/dx = −x/y
Worked Example 2
Problem. Find dy/dx for x²y + y³ = 4.
Answer. dy/dx = −2xy/(x² + 3y²)
Worked Example 3
Problem. Find the slope of the tangent to x² + y² = 25 at the point (3, 4).
Answer. Slope = −3/4
Problem. Find dy/dx for xy = 1.
Solution. Differentiate using the product rule: y + x·(dy/dx) = 0. Solve: dy/dx = −y/x. (Consistent with y = 1/x giving y' = −1/x² = −y/x.)
Differentiating repeatedly gives higher-order derivatives; the second derivative measures concavity and acceleration.
The second derivative f''(x) is the derivative of f'(x); the third is f'''(x), and in general f⁽ⁿ⁾ is the result of differentiating n times. Notation includes f''(x), d²y/dx², and y''. Interpretations: if f(t) is position, f'(t) is velocity and f''(t) is acceleration. The sign of f'' tells concavity — f'' > 0 means concave up (curving like a cup), f'' < 0 means concave down — which drives the second-derivative test for extrema and locates inflection points where f'' changes sign. Higher derivatives also build Taylor series, where the nth derivative at a point supplies the nth coefficient. Each differentiation typically lowers a polynomial's degree by one until it hits zero.
Worked Example 1
Problem. Find f''(x) for f(x) = x⁴ − 3x² + 5.
Answer. f''(x) = 12x² − 6
Worked Example 2
Problem. Find the second derivative of f(x) = sin x and state a pattern.
Answer. f''(x) = −sin x
Worked Example 3
Problem. For f(x) = e^(2x), find f''(x).
Answer. f''(x) = 4e^(2x)
Problem. Find f'''(x) for f(x) = 2x³ + x².
Solution. f'(x) = 6x² + 2x; f''(x) = 12x + 2; f'''(x) = 12.
Code can differentiate three ways — symbolic, numerical (finite differences), and automatic differentiation — each with trade-offs.
Symbolic differentiation (SymPy) manipulates expressions algebraically to return an exact formula; it is precise and human-readable but can suffer 'expression swell' on large compositions. Numerical differentiation approximates with finite differences like [f(x+h) − f(x−h)]/(2h); it is simple and works on black-box functions but introduces truncation and round-off error and must choose h carefully. Automatic differentiation (autodiff), used by PyTorch, JAX, and TensorFlow, applies the chain rule to the elementary operations recorded during a function's evaluation, yielding derivatives accurate to machine precision with no formula blow-up. Autodiff is the engine behind backpropagation: it differentiates programs, not just formulas, efficiently. Knowing which tool fits — exact study (symbolic), quick probes (numerical), or training models (autodiff) — is a core computational skill.
Worked Example 1
Problem. Differentiate f(x) = sin(x²) symbolically, then evaluate f'(1).
Answer. f'(x) = 2x·cos(x²); f'(1) ≈ 1.0806
Worked Example 2
Problem. Estimate f'(1) for f(x) = sin(x²) with a centered finite difference, h = 1e-5.
Answer. ≈ 1.0806 (agrees with symbolic to ~5 digits)
Worked Example 3
Problem. Explain why autodiff gives the exact f'(1) for f(x) = sin(x²) while finite differences only approximate it.
Answer. Autodiff = exact chain rule at machine precision; finite differences carry approximation error
Problem. For f(x) = e^(x²), find f'(x) symbolically and evaluate at x = 0.5, then sketch how you would check it numerically.
Solution. Chain rule: f'(x) = 2x·e^(x²). At x = 0.5: 2(0.5)·e^(0.25) = 1·e^(0.25) ≈ 1.2840. Numerical check: (f(0.5+h) − f(0.5−h))/(2h) with h = 1e-5 yields ≈ 1.2840, confirming the symbolic result.
Hand-derive the derivative of f(x) = (3x² + 1)·e^(sin x), showing every product- and chain-rule step. Then verify with sympy.diff, and finally compute the derivative at x = 1 using a tiny finite-difference approximation. Compare the three answers.
Deliverable · A document or notebook showing the hand derivation, the SymPy result, the finite-difference value at x=1, and a one-paragraph note on which method you would trust in a production ML pipeline and why.
1. What is the derivative of f(x) = x³?
Answer A. By the power rule, d/dx[xⁿ] = n·xⁿ⁻¹, so the derivative of x³ is 3x².
2. Using the chain rule, what is d/dx[sin(x²)]?
Answer B. The outer derivative cos(x²) times the inner derivative 2x gives 2x·cos(x²).
3. What is the derivative of f(x) = e^x?
Answer B. The exponential e^x is its own derivative — the defining property of the natural exponential.
4. Applying the product rule to f(x) = x²·ln(x), what do you get?
Answer C. (x²)'·ln(x) + x²·(ln x)' = 2x·ln(x) + x²·(1/x) = 2x·ln(x) + x.
5. For x² + y² = 25, implicit differentiation gives dy/dx = ?
Answer A. Differentiate: 2x + 2y·(dy/dx) = 0, so dy/dx = -2x/(2y) = -x/y.
I can differentiate polynomial, trig, exponential, and composite functions by hand using the standard rules.
I can apply the chain rule and implicit differentiation to nested and implicit relationships.
I can reproduce a derivative with SymPy and contrast symbolic differentiation with autodiff.
Critical points are the candidate locations for maxima and minima; the first-derivative test classifies them by how the slope's sign changes.
A critical point of f is a value c in the domain where f'(c) = 0 or f'(c) is undefined. These are the only candidates for local extrema (Fermat's theorem: a local max or min in an open interval forces f'(c) = 0 when the derivative exists). The first-derivative test classifies each candidate by the sign of f' on either side: if f' changes from positive to negative at c, f has a local maximum there; from negative to positive gives a local minimum; no sign change means neither. This matters because optimization — minimizing a loss, maximizing a likelihood — reduces to locating and classifying critical points. The procedure is mechanical: differentiate, solve f'(x) = 0, find where f' is undefined, then test the sign of f' across each candidate.
Worked Example 1
Problem. Find and classify the critical points of f(x) = x³ − 3x.
Answer. Local max at x = −1 (value 2), local min at x = 1 (value −2)
Worked Example 2
Problem. Find the critical points of f(x) = x^(2/3).
Answer. Critical point at x = 0; local minimum (a cusp)
Worked Example 3
Problem. Classify the critical points of f(x) = x⁴ − 4x³.
Answer. x = 0 is neither; x = 3 is a local minimum
Problem. Find and classify the critical points of f(x) = 2x³ − 9x² + 12x.
Solution. f'(x) = 6x² − 18x + 12 = 6(x² − 3x + 2) = 6(x − 1)(x − 2). Critical points at x = 1 and x = 2. Signs: x<1 gives f'>0; 1<x<2 gives f'<0; x>2 gives f'>0. So x = 1 is a local max (f(1) = 5) and x = 2 is a local min (f(2) = 4).
The second derivative reveals how a curve bends and gives a quick test to classify critical points.
Concavity describes the bending of a graph. Where f''(x) > 0 the curve is concave up (holds water, slopes increasing); where f''(x) < 0 it is concave down. An inflection point is where concavity changes — typically where f''(x) = 0 or is undefined AND the sign of f'' actually flips. The second-derivative test classifies a critical point c where f'(c) = 0: if f''(c) > 0 the curve is concave up there, so c is a local minimum; if f''(c) < 0, a local maximum; if f''(c) = 0 the test is inconclusive and you fall back to the first-derivative test. This matters in optimization: convex regions (f'' > 0) guarantee a minimizer behaves well, which is exactly why convexity is prized in machine learning.
Worked Example 1
Problem. Use the second-derivative test on f(x) = x³ − 3x at its critical points.
Answer. Local max at x = −1, local min at x = 1
Worked Example 2
Problem. Find the inflection point(s) of f(x) = x³ − 6x² + 9x.
Answer. Inflection point at (2, 2)
Worked Example 3
Problem. Show why the second-derivative test is inconclusive for f(x) = x⁴ at x = 0, then classify it.
Answer. Local minimum at x = 0 (second-derivative test inconclusive)
Problem. For f(x) = x⁴ − 4x², find the intervals of concavity and any inflection points.
Solution. f'(x) = 4x³ − 8x; f''(x) = 12x² − 8. Set f''=0: x² = 2/3, x = ±√(2/3) ≈ ±0.816. f'' > 0 for |x| > √(2/3) (concave up) and f'' < 0 for |x| < √(2/3) (concave down). Both are inflection points since concavity changes sign there.
Optimization turns a word problem into a single-variable function to maximize or minimize over its feasible domain.
Applied optimization follows a recipe: (1) identify the quantity to optimize (the objective) and the constraint(s); (2) use the constraint to express the objective as a function of one variable; (3) determine the feasible domain; (4) find critical points by setting the derivative to zero (plus checking endpoints); (5) classify them and compare to find the global optimum on the domain. The Extreme Value Theorem guarantees a continuous function on a closed interval attains both a max and a min — at a critical point or an endpoint. This is the core of modeling: minimizing cost, maximizing area or profit, fitting a least-squares line, or minimizing a loss. The hardest step is usually translation: building the right one-variable objective from the description.
Worked Example 1
Problem. A farmer has 100 m of fence for a rectangular pen. What dimensions maximize the area?
Answer. 25 m × 25 m (a square), area 625 m²
Worked Example 2
Problem. Minimize the surface area of an open-top box with square base and volume 32 cm³.
Answer. Base 4×4 cm, height 2 cm; minimal surface area 48 cm²
Worked Example 3
Problem. Find the point on the line y = 2x + 1 closest to the origin.
Answer. Closest point (−2/5, 1/5)
Problem. A rectangle is inscribed with its base on the x-axis and top corners on the parabola y = 12 − x². Maximize its area.
Solution. By symmetry the corners are at (±x, 12 − x²), width 2x, height 12 − x². Area A(x) = 2x(12 − x²) = 24x − 2x³. A'(x) = 24 − 6x² = 0 ⇒ x² = 4 ⇒ x = 2. A''(x) = −12x < 0 ⇒ max. Dimensions: width 4, height 8, area 32.
Related-rates problems use the chain rule to connect how fast linked quantities change over time.
When two or more quantities are related by an equation and all change with time, differentiating the relationship with respect to t links their rates. The recipe: (1) write an equation relating the quantities; (2) differentiate both sides with respect to time t, applying the chain rule so each variable gains a d/dt factor (e.g., d/dt[x²] = 2x·dx/dt); (3) substitute the known instantaneous values and rates; (4) solve for the unknown rate. The key conceptual move is that variables are functions of time, so every derivative carries a rate like dx/dt. Related rates appear wherever quantities co-vary: filling tanks, expanding shadows, approaching objects — and conceptually mirror how gradients propagate through linked variables.
Worked Example 1
Problem. A balloon's radius grows at dr/dt = 2 cm/s. How fast is its volume changing when r = 5 cm?
Answer. dV/dt = 200π ≈ 628.3 cm³/s
Worked Example 2
Problem. A 5 m ladder leans on a wall; its base slides away at 1 m/s. How fast does the top slide down when the base is 3 m from the wall?
Answer. The top slides down at 3/4 m/s
Worked Example 3
Problem. Water fills a cone (apex down, radius 3 m at top, height 6 m) at 2 m³/min. How fast is the depth rising when the water is 3 m deep?
Answer. dh/dt = 8/(9π) ≈ 0.283 m/min
Problem. A circle's area grows at 8 cm²/s. How fast is the radius increasing when r = 4 cm?
Solution. A = πr², so dA/dt = 2πr·(dr/dt). Substitute dA/dt = 8 and r = 4: 8 = 2π(4)·(dr/dt) = 8π·(dr/dt). Solve: dr/dt = 1/π ≈ 0.318 cm/s.
Near a point, a smooth function looks like its tangent line; linearization uses that line to estimate nearby values.
The linear approximation (linearization) of f near x = a is L(x) = f(a) + f'(a)(x − a) — the tangent line at a. For x close to a, f(x) ≈ L(x), because a differentiable function is locally well-modeled by its slope. The differential dy = f'(x)·dx captures the approximate change in y for a small change dx in x; it estimates error propagation and small increments. Linearization is the first-order Taylor approximation and underlies countless numerical methods: Newton's method, gradient descent steps, and sensitivity/error analysis all replace a function locally by its tangent. The approximation is excellent for small |x − a| and degrades as you move away, with error governed by the curvature (second derivative).
Worked Example 1
Problem. Use linearization to approximate √4.1.
Answer. √4.1 ≈ 2.025
Worked Example 2
Problem. Approximate (1.02)⁵ using linearization.
Answer. (1.02)⁵ ≈ 1.10
Worked Example 3
Problem. A cube's side is measured as 10 cm with a possible error of ±0.1 cm. Use differentials to estimate the error in the computed volume.
Answer. dV ≈ ±30 cm³ (about 3% relative error)
Problem. Use linearization to estimate ∛8.1 (cube root of 8.1).
Solution. Let f(x) = x^(1/3), a = 8 (∛8 = 2). f'(x) = (1/3)x^(−2/3), f'(8) = (1/3)(1/4) = 1/12. L(x) = 2 + (1/12)(x − 8). At x = 8.1: L = 2 + (1/12)(0.1) = 2 + 0.00833 ≈ 2.0083. (True value ≈ 2.00829.)
Gradient descent minimizes a function by repeatedly stepping in the direction opposite its derivative — the engine of machine-learning training.
Gradient descent is an iterative optimization method: starting from a guess x₀, it updates xₙ₊₁ = xₙ − η·f'(xₙ), where η (the learning rate) controls step size. Because f'(x) points uphill (direction of steepest increase), subtracting it moves toward lower values, so f tends to decrease each step. The method converges toward a critical point — ideally a minimum — when η is small enough; too-large η overshoots and may diverge, while too-small η crawls. In machine learning the function f is a loss measuring model error, and the derivative (gradient in many dimensions) is computed by automatic differentiation/backpropagation. Every neural network is trained by this loop. Single-variable gradient descent is the clearest place to build the intuition before generalizing to many parameters.
Worked Example 1
Problem. Minimize f(x) = x² starting at x₀ = 10 with learning rate η = 0.1. Show two steps.
Answer. x₁ = 8, x₂ = 6.4 (converging to 0)
Worked Example 2
Problem. For f(x) = x², find the η that converges fastest and the η that diverges.
Answer. Fastest at η = 0.5; diverges for η ≥ 1
Worked Example 3
Problem. One gradient-descent step on f(x) = x⁴ − 3x² + x from x₀ = 1 with η = 0.05.
Answer. x₁ = 1.05
Problem. Minimize f(x) = (x − 3)² starting at x₀ = 0 with η = 0.25. Compute x₁ and x₂.
Solution. f'(x) = 2(x − 3). Step 1: f'(0) = −6, x₁ = 0 − 0.25(−6) = 1.5. Step 2: f'(1.5) = 2(1.5 − 3) = −3, x₂ = 1.5 − 0.25(−3) = 1.5 + 0.75 = 2.25. The iterates approach the minimum at x = 3.
Choose a single-variable cost function such as f(x) = x⁴ - 3x² + x. Find its critical points analytically with SymPy and classify them with the second derivative. Then implement a small gradient-descent loop in NumPy, run it from two different starting points, and observe which minimum each run reaches.
Deliverable · A notebook showing the symbolic critical-point analysis, a gradient-descent function with a tunable learning rate, a plot of the function with the descent paths overlaid, and a note on how the start point and learning rate affected the outcome.
1. At a critical point where f'(c) = 0 and f''(c) > 0, the point is a:
Answer B. A positive second derivative means the curve is concave up, so a critical point there is a local minimum (a valley).
2. An inflection point of f occurs where:
Answer C. An inflection point is where concavity changes, which corresponds to the second derivative changing sign.
3. To maximize the area of a rectangle with fixed perimeter P, you should set up the problem by:
Answer B. Use the constraint to write area in terms of a single variable, then differentiate and set to zero; the optimum is a square.
4. In gradient descent, the update x ← x - η·f'(x) moves x in which direction?
Answer B. Subtracting the (scaled) derivative steps opposite the direction of steepest increase, so f tends to decrease.
5. The linear approximation of f near x = a is given by:
Answer A. Linearization uses the tangent line: L(x) = f(a) + f'(a)(x - a).
I can locate and classify extrema using the first- and second-derivative tests.
I can set up and solve an optimization problem from a word description.
I can connect the derivative to one step of gradient descent and explain the role of the learning rate.
An antiderivative reverses differentiation; the indefinite integral collects all antiderivatives of a function.
F is an antiderivative of f if F'(x) = f(x). Because the derivative of a constant is zero, antiderivatives come in families differing by a constant, so the indefinite integral is written ∫f(x) dx = F(x) + C, where C is the constant of integration. Basic rules run the derivative rules backward: ∫xⁿ dx = xⁿ⁺¹/(n+1) + C for n ≠ −1, ∫(1/x) dx = ln|x| + C, ∫eˣ dx = eˣ + C, ∫cos x dx = sin x + C, ∫sin x dx = −cos x + C. Linearity lets you integrate term by term and pull out constants. Antiderivatives matter because the Fundamental Theorem turns them into exact areas, and they reconstruct a quantity from its rate of change — recovering position from velocity, or total accumulation from a density.
Worked Example 1
Problem. Find ∫(3x² + 4x − 5) dx.
Answer. x³ + 2x² − 5x + C
Worked Example 2
Problem. Find ∫(1/x + eˣ) dx.
Answer. ln|x| + eˣ + C
Worked Example 3
Problem. Find the antiderivative F of f(x) = 6x² satisfying F(1) = 5.
Answer. F(x) = 2x³ + 3
Problem. Find ∫(4x³ − 2sin x) dx.
Solution. ∫4x³ dx = x⁴; ∫−2sin x dx = −2(−cos x) = 2cos x. So the answer is x⁴ + 2cos x + C.
A definite integral is the limit of Riemann sums — adding up the areas of ever-thinner rectangles under a curve.
To define the definite integral ∫ₐᵇ f(x) dx, partition [a, b] into n subintervals of width Δx = (b − a)/n, pick a sample point in each, and sum the rectangle areas: Σ f(xᵢ)·Δx (a Riemann sum). Left, right, and midpoint sums differ by their sample choice. As n → ∞ (Δx → 0), these sums converge to a single number — the definite integral — interpreted as the signed area between f and the x-axis (area below the axis counts negative). This construction is why integrals model accumulation: total distance from a speed curve, total probability under a density, total work from a force. Riemann sums are also the practical recipe for numerical integration when no antiderivative exists.
Worked Example 1
Problem. Approximate ∫₀² x² dx with a right Riemann sum using n = 4 rectangles.
Answer. ≈ 3.75
Worked Example 2
Problem. Use the limit of right Riemann sums to find ∫₀¹ x dx exactly.
Answer. 1/2
Worked Example 3
Problem. Interpret ∫₋₁¹ x dx as signed area and evaluate.
Answer. 0
Problem. Approximate ∫₀² x² dx with a left Riemann sum using n = 4 rectangles.
Solution. Δx = 0.5; left endpoints x = 0, 0.5, 1.0, 1.5; heights 0, 0.25, 1.0, 2.25. Sum × Δx = (0 + 0.25 + 1.0 + 2.25)(0.5) = (3.5)(0.5) = 1.75 (an underestimate; exact 8/3 ≈ 2.667).
The Fundamental Theorem links the two halves of calculus: differentiation and integration are inverse operations.
FTC Part 1: if g(x) = ∫ₐˣ f(t) dt for continuous f, then g'(x) = f(x) — differentiating an accumulation function recovers the integrand. FTC Part 2 (the evaluation theorem): if F is any antiderivative of f, then ∫ₐᵇ f(x) dx = F(b) − F(a). Part 2 is the workhorse: instead of computing a limit of Riemann sums, you find an antiderivative and subtract its endpoint values. Together the parts show that integration and differentiation undo each other, unifying area and rate of change. This is one of the great results in mathematics: it converts hard limit-of-sum problems into routine antidifferentiation, and it explains why accumulation functions are smooth, with slope equal to the thing being accumulated.
Worked Example 1
Problem. Evaluate ∫₁³ 2x dx using FTC Part 2.
Answer. 8
Worked Example 2
Problem. Find g'(x) if g(x) = ∫₂ˣ (t³ + 1) dt.
Answer. g'(x) = x³ + 1
Worked Example 3
Problem. Evaluate ∫₀^π sin x dx.
Answer. 2
Problem. Evaluate ∫₁⁴ (2x + 1) dx using FTC Part 2.
Solution. Antiderivative F(x) = x² + x. Then F(4) − F(1) = (16 + 4) − (1 + 1) = 20 − 2 = 18.
u-substitution reverses the chain rule: replace an inner function with u to simplify the integral.
When an integrand contains a function and (a multiple of) its derivative, set u = g(x) so that du = g'(x) dx, converting ∫f(g(x))·g'(x) dx into the simpler ∫f(u) du. After integrating in u, substitute back to x. For definite integrals, either back-substitute and use the original limits, or — more cleanly — change the limits to u-values g(a) and g(b) and evaluate directly in u. The skill is pattern recognition: spotting an inner function whose derivative (up to a constant) is also present. This single technique unlocks a huge class of integrals — exponentials of polynomials, powers of linear expressions, trig of compound arguments — and is the reverse mirror image of the chain rule that dominated differentiation.
Worked Example 1
Problem. Evaluate ∫ 2x·(x² + 1)⁵ dx.
Answer. (x² + 1)⁶/6 + C
Worked Example 2
Problem. Evaluate ∫ x·e^(x²) dx.
Answer. (1/2)e^(x²) + C
Worked Example 3
Problem. Evaluate the definite integral ∫₀¹ 3x²·(x³ + 2)⁴ dx by changing limits.
Answer. 211/5 = 42.2
Problem. Evaluate ∫ cos(x)·e^(sin x) dx.
Solution. Let u = sin x, du = cos x dx. The integral becomes ∫ e^u du = e^u + C. Back-substitute: e^(sin x) + C.
Integration by parts reverses the product rule, trading a hard integral for an easier one.
Integration by parts comes from the product rule: ∫u dv = uv − ∫v du. The art is choosing u (to differentiate) and dv (to integrate) so that the new integral ∫v du is simpler than the original. A helpful priority for picking u is LIATE — Logarithmic, Inverse-trig, Algebraic, Trigonometric, Exponential — choose u from earliest in this list. The technique shines for products of unlike functions: polynomial × exponential, polynomial × trig, or a lone logarithm. Sometimes you apply it repeatedly (reducing a polynomial's degree each time) or cyclically (for ∫eˣ sin x dx, the original integral reappears and you solve algebraically). It is the integral counterpart of the product rule and a staple for probability and signal-processing integrals.
Worked Example 1
Problem. Evaluate ∫ x·eˣ dx.
Answer. x·eˣ − eˣ + C = eˣ(x − 1) + C
Worked Example 2
Problem. Evaluate ∫ ln x dx.
Answer. x·ln x − x + C
Worked Example 3
Problem. Evaluate ∫ x·cos x dx.
Answer. x·sin x + cos x + C
Problem. Evaluate ∫ x·sin x dx.
Solution. Let u = x (du = dx), dv = sin x dx (v = −cos x). Then ∫x sin x dx = −x cos x − ∫(−cos x) dx = −x cos x + ∫cos x dx = −x cos x + sin x + C.
Definite integrals compute areas between curves, the average value of a function, and total accumulation from a rate.
Three core applications flow from the definite integral. Area between two curves f (upper) and g (lower) on [a, b] is ∫ₐᵇ [f(x) − g(x)] dx. The average value of f on [a, b] is (1/(b − a))∫ₐᵇ f(x) dx — the constant height of a rectangle with the same area, formalized by the Mean Value Theorem for integrals. Accumulation interprets ∫ₐᵇ r(t) dt as the total change of a quantity whose rate is r(t): integrate a velocity to get displacement, a flow rate to get volume, or a probability density to get a probability. In data science these reappear as expected values (E[X] = ∫x·p(x) dx), cumulative distribution functions, and total counts from rates — making integration the language of accumulation and averaging.
Worked Example 1
Problem. Find the area between y = x² and y = x from x = 0 to x = 1.
Answer. 1/6
Worked Example 2
Problem. Find the average value of f(x) = x² on [0, 3].
Answer. 3
Worked Example 3
Problem. A particle's velocity is v(t) = 3t² (m/s). Find the total displacement from t = 0 to t = 2.
Answer. 8 m
Problem. Find the average value of f(x) = sin x on [0, π].
Solution. Average = (1/(π − 0))∫₀^π sin x dx = (1/π)[−cos x]₀^π = (1/π)(−cos π + cos 0) = (1/π)(1 + 1) = 2/π ≈ 0.637.
Code integrates two ways — symbolically (exact formula) or by numerical quadrature (approximation) — and many integrals demand the numerical route.
Symbolic integration (SymPy's integrate) returns an exact antiderivative or definite value when one exists in closed form. But many integrands, like e^(−x²), have no elementary antiderivative, so we turn to numerical quadrature: algorithms that estimate ∫ₐᵇ f dx from sampled values. The trapezoidal rule sums trapezoids; Simpson's rule fits parabolas for higher accuracy; adaptive routines like SciPy's quad refine where the function varies fastest and report an error estimate. NumPy's trapz and SciPy's quad are the practical workhorses. The trade-off mirrors differentiation: symbolic is exact but can fail or balloon, while numerical always returns a number with controllable error. Knowing when each applies — and validating numerics against a known case — is an essential computational skill.
Worked Example 1
Problem. Compute ∫₀¹ x² dx symbolically and confirm the value.
Answer. 1/3
Worked Example 2
Problem. Estimate ∫₀² e^(−x²) dx, which has no elementary antiderivative, by numerical quadrature.
Answer. ≈ 0.8821
Worked Example 3
Problem. Approximate ∫₀¹ x² dx with the trapezoidal rule using n = 4 and compare to the exact 1/3.
Answer. ≈ 0.34375 (vs exact 0.3333)
Problem. Use NumPy's trapezoidal rule to approximate ∫₀^π sin x dx with n = 100 subintervals and compare to the exact value.
Solution. The exact value is 2. With x = np.linspace(0, np.pi, 101) and np.trapz(np.sin(x), x), the result is ≈ 1.99984 — within about 1.6×10⁻⁴ of 2, and the error falls as n increases.
Take f(x) = e^(-x²) on [0, 2], a function with no elementary antiderivative. Approximate its definite integral with a left Riemann sum for n = 10, 100, and 1000 rectangles in NumPy, then compare against scipy.integrate.quad (or sympy.integrate). Plot how the error shrinks as n grows.
Deliverable · A notebook with a Riemann-sum function, a convergence table or log-log error plot versus n, and a short reflection on why this integral needs a numerical method rather than a closed form.
1. What is ∫ 2x dx?
Answer A. An antiderivative of 2x is x², since d/dx[x²] = 2x; add the constant of integration C.
2. By the Fundamental Theorem of Calculus (Part 2), ∫₁³ 2x dx equals:
Answer B. An antiderivative is x², so evaluate 3² - 1² = 9 - 1 = 8.
3. Which technique best handles ∫ x·cos(x) dx?
Answer C. A product of a polynomial and a trig function calls for integration by parts (let u = x, dv = cos x dx).
4. A left Riemann sum with finitely many rectangles generally gives:
Answer B. Riemann sums approximate the integral; refining the partition (more rectangles) converges to the exact value.
5. For ∫ 2x·(x² + 1)⁵ dx, the natural substitution is:
Answer B. With u = x² + 1, du = 2x dx, turning the integral into ∫u⁵ du — exactly the chain rule run backward.
I can compute definite and indefinite integrals using substitution and integration by parts.
I can state and apply both parts of the Fundamental Theorem of Calculus.
I can approximate a definite integral with a Riemann sum in code and compare it to an exact result.
A sequence is an ordered list of numbers; its limit describes the value the terms settle toward as the index grows.
A sequence {aₙ} assigns a number to each index n = 1, 2, 3, …, often by a formula like aₙ = 1/n or a recurrence. The sequence converges to L if its terms get arbitrarily close to L for all large n: lim n→∞ aₙ = L. Formally, for every ε > 0 there is an N so that |aₙ − L| < ε whenever n > N. If no such finite L exists, the sequence diverges (it may grow without bound or oscillate). Useful tools include limit laws, the fact that a bounded monotonic sequence converges, and treating aₙ = f(n) so continuous-function limits (and L'Hôpital) apply. Sequences matter because series are built from them, iterative algorithms produce sequences of estimates, and convergence is the precise notion behind 'the method settles down.'
Worked Example 1
Problem. Find lim n→∞ (3n + 1)/(n + 2).
Answer. 3
Worked Example 2
Problem. Does aₙ = (−1)ⁿ converge?
Answer. Diverges (no limit)
Worked Example 3
Problem. Find lim n→∞ ln(n)/n.
Answer. 0
Problem. Find lim n→∞ (2n² − n)/(5n² + 3).
Solution. Divide top and bottom by n²: (2 − 1/n)/(5 + 3/n²). As n→∞ the small terms vanish, giving 2/5.
An infinite series sums a sequence's terms; it converges when its partial sums approach a finite limit.
An infinite series Σ aₙ is the sum a₁ + a₂ + a₃ + …. Its meaning is the limit of the partial sums Sₙ = a₁ + … + aₙ: the series converges to S if Sₙ → S, and diverges otherwise. The most important closed-form case is the geometric series Σ arⁿ (n from 0), which converges to a/(1 − r) when |r| < 1 and diverges when |r| ≥ 1. A necessary condition for convergence is the divergence test: if aₙ does not approach 0, the series diverges — but aₙ → 0 alone does NOT guarantee convergence (the harmonic series Σ 1/n diverges despite 1/n → 0). Series matter because Taylor expansions, probability generating functions, and many algorithmic costs are sums of infinitely many terms.
Worked Example 1
Problem. Evaluate the geometric series Σ (1/2)ⁿ for n = 0, 1, 2, ….
Answer. 2
Worked Example 2
Problem. Use the divergence test on Σ n/(n + 1).
Answer. Diverges
Worked Example 3
Problem. Evaluate Σ 3·(1/4)ⁿ for n = 0, 1, 2, ….
Answer. 4
Problem. Evaluate Σ (2/3)ⁿ for n = 0, 1, 2, ….
Solution. Geometric with a = 1 and r = 2/3 (|r| < 1). Sum = 1/(1 − 2/3) = 1/(1/3) = 3.
When no closed form exists, convergence tests decide whether a series converges by examining its terms' behavior.
Several tests diagnose convergence. The ratio test computes L = lim |aₙ₊₁/aₙ|: if L < 1 the series converges absolutely, if L > 1 it diverges, and L = 1 is inconclusive — it is ideal for factorials and exponentials. The comparison test bounds a positive series by a known one: if 0 ≤ aₙ ≤ bₙ and Σbₙ converges, so does Σaₙ (and the reverse for divergence); the limit-comparison variant compares the limiting ratio of terms. The integral test applies when aₙ = f(n) for a positive, decreasing f: Σaₙ and ∫₁^∞ f(x) dx converge or diverge together — this proves the p-series Σ 1/nᵖ converges exactly when p > 1. Picking the right test is a pattern-matching skill central to analyzing series.
Worked Example 1
Problem. Use the ratio test on Σ 1/n!.
Answer. Converges (L = 0)
Worked Example 2
Problem. Use the integral test on the p-series Σ 1/n² .
Answer. Converges
Worked Example 3
Problem. Use comparison to test Σ 1/(n² + 1).
Answer. Converges (by comparison with Σ 1/n²)
Problem. Use the ratio test to decide whether Σ 2ⁿ/n! converges.
Solution. aₙ = 2ⁿ/n!; aₙ₊₁/aₙ = (2ⁿ⁺¹/(n+1)!)·(n!/2ⁿ) = 2/(n+1). L = lim n→∞ 2/(n+1) = 0 < 1. By the ratio test, the series converges (its sum is e² − 1).
A power series is a polynomial of infinite degree in (x − a); it converges only on an interval centered at a.
A power series Σ cₙ(x − a)ⁿ is built from coefficients cₙ and is centered at a. For each x it is just a number series, so convergence depends on x. The set of x where it converges is always an interval centered at a, described by a radius of convergence R: the series converges for |x − a| < R, diverges for |x − a| > R, and the endpoints must be checked separately. You find R with the ratio test, computing lim |cₙ₊₁(x − a)ⁿ⁺¹ / cₙ(x − a)ⁿ| < 1 and solving for |x − a|. Within its interval a power series behaves like a polynomial — you can differentiate and integrate term by term — which is exactly why Taylor series (the next lesson) are so powerful for approximating and manipulating functions.
Worked Example 1
Problem. Find the interval of convergence of Σ xⁿ (n from 0).
Answer. (−1, 1), radius R = 1
Worked Example 2
Problem. Find the radius of convergence of Σ xⁿ/n! (n from 0).
Answer. Converges for all x, R = ∞
Worked Example 3
Problem. Find the interval of convergence of Σ (x − 2)ⁿ/n (n from 1).
Answer. [1, 3), radius R = 1
Problem. Find the radius and interval of convergence of Σ xⁿ/2ⁿ (n from 0).
Solution. Ratio: |xⁿ⁺¹/2ⁿ⁺¹ · 2ⁿ/xⁿ| = |x|/2 → |x|/2. Converges when |x|/2 < 1 ⇒ |x| < 2, so R = 2, base interval (−2, 2). Endpoints x = 2 gives Σ1 (diverges) and x = −2 gives Σ(−1)ⁿ (diverges). Interval: (−2, 2).
A Taylor series rebuilds a smooth function as an infinite polynomial from its derivatives at a point; centered at 0 it is a Maclaurin series.
The Taylor series of f about a is Σ f⁽ⁿ⁾(a)/n! · (x − a)ⁿ — each coefficient is the nth derivative at a divided by n!. Centered at a = 0 it is the Maclaurin series. The idea: match the function's value and all its derivatives at the center, so the polynomial hugs the curve ever more tightly as you add terms. Key Maclaurin series to know: eˣ = Σ xⁿ/n!; sin x = x − x³/3! + x⁵/5! − …; cos x = 1 − x²/2! + x⁴/4! − …; 1/(1 − x) = Σ xⁿ for |x| < 1. These are foundational in data science: they linearize and quadratically approximate functions (the basis of Newton's method and second-order optimization), underlie how libraries compute transcendental functions, and explain error in numerical methods.
Worked Example 1
Problem. Find the Maclaurin series of eˣ.
Answer. eˣ = Σ xⁿ/n! = 1 + x + x²/2 + x³/6 + …
Worked Example 2
Problem. Find the Maclaurin series of cos x up to the x⁴ term.
Answer. cos x ≈ 1 − x²/2 + x⁴/24
Worked Example 3
Problem. Find the Taylor series of ln(x) about a = 1 up to the (x − 1)³ term.
Answer. ln x ≈ (x − 1) − (x − 1)²/2 + (x − 1)³/3
Problem. Find the Maclaurin series of sin x up to the x⁵ term.
Solution. Derivatives at 0 cycle sin→cos→−sin→−cos: values 0, 1, 0, −1, 0, 1. Coefficients: 0, 1, 0, −1/3!, 0, 1/5!. Keeping odd powers: sin x ≈ x − x³/6 + x⁵/120.
Truncating a Taylor series after finitely many terms gives a polynomial approximation; the leftover is the truncation error.
Using only the first n + 1 terms of a Taylor series gives the degree-n Taylor polynomial Pₙ(x); the difference Rₙ(x) = f(x) − Pₙ(x) is the remainder, or truncation error. Taylor's theorem with the Lagrange remainder bounds it: Rₙ(x) = f⁽ⁿ⁺¹⁾(ξ)/(n+1)! · (x − a)ⁿ⁺¹ for some ξ between a and x. Two levers shrink the error: more terms (larger n shrinks via the factorial) and staying near the center (small |x − a| shrinks via the power). For alternating series like sin and cos, the simpler alternating-series bound says the error is at most the first omitted term. This is the backbone of numerical accuracy analysis: it tells you how many terms a library needs, and why local approximations (linearization, Newton's method) work so well near the center.
Worked Example 1
Problem. Approximate e^(0.1) with the Maclaurin polynomial 1 + x + x²/2 and estimate the error.
Answer. ≈ 1.105, error ≈ 0.00017
Worked Example 2
Problem. Use the alternating-series bound to estimate the error in approximating sin(0.2) by x − x³/6.
Answer. Error ≤ ~2.7×10⁻⁶
Worked Example 3
Problem. How many Maclaurin terms of eˣ are needed so the error at x = 1 is below 10⁻³?
Answer. n = 6 (terms through x⁶/6!)
Problem. Approximate cos(0.3) using 1 − x²/2 and bound the error with the next term.
Solution. P(0.3) = 1 − (0.09)/2 = 1 − 0.045 = 0.955. The first omitted term is x⁴/4! = (0.3)⁴/24 = 0.0081/24 ≈ 0.000338, so the error is at most about 3.4×10⁻⁴. (True cos(0.3) ≈ 0.955336, error ≈ 3.4×10⁻⁴.)
Using SymPy, generate the Maclaurin series of e^x, sin(x), and cos(x) up to several orders. Then write a NumPy function that evaluates each truncated series and plot the approximation against the true function on an interval, showing how adding terms improves the fit. Measure the maximum error for orders 1 through 7.
Deliverable · A notebook with the SymPy series expansions, a truncated-series evaluator, overlay plots of approximation vs. true function, and a table of maximum error by truncation order.
1. The geometric series Σ (1/2)ⁿ for n = 0,1,2,… converges to:
Answer B. With a = 1 and r = 1/2, the sum is a/(1-r) = 1/(1 - 1/2) = 2.
2. The ratio test concludes convergence when:
Answer A. If the limiting ratio of successive terms is below 1, the series converges absolutely; above 1 it diverges; equal to 1 is inconclusive.
3. The Maclaurin series of e^x begins:
Answer B. Because every derivative of e^x is 1 at 0, the series is Σ xⁿ/n! = 1 + x + x²/2! + x³/3! + ….
4. A divergent series is one whose partial sums:
Answer C. Divergence means the sequence of partial sums fails to settle on a finite value.
5. Increasing the order of a Taylor approximation near the center generally:
Answer B. Adding higher-order terms captures more of the function's behavior, reducing truncation error near the expansion point.
I can determine whether a basic series converges using an appropriate test.
I can construct the Taylor or Maclaurin series of a common function to a given order.
I can use a truncated Taylor series to approximate a function and quantify the truncation error in code.
A differential equation relates a function to its own derivatives, encoding a rule for how a quantity changes.
A differential equation (DE) is an equation involving an unknown function and its derivatives, such as dy/dx = ky. Its order is the highest derivative present (first-order uses only y'). A solution is a function that satisfies the equation; the general solution carries arbitrary constants, and an initial condition y(x₀) = y₀ selects one particular solution. DEs are the natural language of change: they say how fast something grows, cools, or moves rather than giving the value directly, and solving recovers the function from that rule. They power modeling everywhere — population dynamics, radioactive decay, circuits, epidemics, and the continuous-time view of optimization (gradient flow). Verifying a proposed solution is simple: differentiate it and check that it satisfies the equation and any initial condition.
Worked Example 1
Problem. Verify that y = e^(3x) is a solution of dy/dx = 3y.
Answer. Yes, y = e^(3x) is a solution
Worked Example 2
Problem. State the order of (d²y/dx²) + 5(dy/dx) − y = 0 and how many constants its general solution has.
Answer. Second order; two arbitrary constants
Worked Example 3
Problem. Given the general solution y = C·e^(2x), find the particular solution with y(0) = 5.
Answer. y = 5e^(2x)
Problem. Verify that y = sin x is a solution of y'' + y = 0.
Solution. y' = cos x, y'' = −sin x. Then y'' + y = −sin x + sin x = 0. The equation is satisfied, so y = sin x is a solution.
A separable equation can be split so each variable lives on its own side, then both sides are integrated.
A first-order ODE is separable if it can be written dy/dx = g(x)·h(y). The solution method separates variables: move all y's (and dy) to one side and all x's (and dx) to the other, giving (1/h(y)) dy = g(x) dx, then integrate both sides. The two constants of integration combine into one arbitrary constant, which an initial condition pins down. Watch for equilibrium solutions where h(y) = 0 (constant solutions you divided away). Separable equations are the first solvable family and cover many real models: exponential growth/decay (dy/dt = ky), Newton's law of cooling, and the logistic equation. The technique is essentially the Fundamental Theorem applied to each side after the algebra of separation.
Worked Example 1
Problem. Solve dy/dx = xy.
Answer. y = A·e^(x²/2)
Worked Example 2
Problem. Solve dy/dx = y² with y(0) = 1.
Answer. y = 1/(1 − x)
Worked Example 3
Problem. Solve dy/dx = (2x)/(y) with y(0) = 3.
Answer. y = √(2x² + 9)
Problem. Solve dy/dx = 3x²·y with y(0) = 2.
Solution. Separate: (1/y) dy = 3x² dx. Integrate: ln|y| = x³ + C, so y = A·e^(x³). Apply y(0) = 2: 2 = A·e⁰ = A. Thus y = 2e^(x³).
A slope field plots the slope dy/dx at a grid of points, revealing how solutions flow even without a formula.
For a first-order ODE dy/dx = f(x, y), the slope field (direction field) draws a short line segment with slope f(x, y) at each point of a grid. Any solution curve must be tangent to these segments, so following the flow sketches solutions through any starting point — a purely qualitative tool that needs no closed-form solution. Slope fields reveal equilibria (where f = 0, giving horizontal segments and constant solutions), their stability (do nearby solutions approach or leave them?), and asymptotic behavior. This is invaluable when a DE is hard or impossible to solve symbolically: you still read off long-term trends. Reading a slope field — spotting where solutions rise, fall, level off, or approach a carrying capacity — builds the intuition that growth/decay and logistic models then make precise.
Worked Example 1
Problem. Describe the slope field of dy/dx = y and the behavior of solutions.
Answer. Exponential growth away from the unstable equilibrium y = 0
Worked Example 2
Problem. Find the equilibrium solutions of dy/dt = y(1 − y) and classify their stability.
Answer. y = 0 unstable, y = 1 stable
Worked Example 3
Problem. For dy/dx = x, where are the slope-field segments horizontal, and what shape are the solutions?
Answer. Horizontal along x = 0; solutions are parabolas y = x²/2 + C
Problem. Find and classify the equilibria of dy/dt = 2 − y.
Solution. Set 2 − y = 0 ⇒ y = 2 is the only equilibrium. For y < 2, dy/dt > 0 (rising toward 2); for y > 2, dy/dt < 0 (falling toward 2). Solutions approach y = 2 from both sides, so it is a stable equilibrium.
When a quantity changes at a rate proportional to its size, it grows or decays exponentially.
The model dy/dt = ky says the rate of change is proportional to the current amount. Solving (it is separable) gives y(t) = y₀·e^(kt), where y₀ = y(0). If k > 0 the quantity grows exponentially (populations, compound interest, early epidemics); if k < 0 it decays exponentially (radioactive material, drug clearance, cooling toward zero). Two handy constants: the doubling time t₂ = ln 2 / k for growth, and the half-life t½ = ln 2 / |k| for decay. The model is the workhorse of first-order kinetics and the continuous analog of geometric sequences. It is also a benchmark: real systems often grow exponentially only at first, then saturate — which motivates the logistic model next.
Worked Example 1
Problem. A bacteria culture grows by dy/dt = 0.5y with y(0) = 100. Find y(t) and y(4).
Answer. y(t) = 100e^(0.5t); y(4) ≈ 739
Worked Example 2
Problem. A radioactive sample has a half-life of 10 years. Find its decay constant k and the fraction left after 25 years.
Answer. k ≈ −0.0693/yr; ≈ 17.7% remains after 25 years
Worked Example 3
Problem. A population doubles every 3 hours. Find its growth rate k and the factor of increase after 9 hours.
Answer. k ≈ 0.231/hr; the population increases 8×
Problem. Caffeine in the body decays with k = −0.17/hr. Starting at 200 mg, how much remains after 6 hours?
Solution. y(t) = 200·e^(−0.17t). At t = 6: exponent = −1.02, e^(−1.02) ≈ 0.3606. So y(6) ≈ 200·0.3606 ≈ 72.1 mg.
The logistic model adds a carrying capacity to exponential growth, producing an S-shaped curve that levels off.
The logistic equation dy/dt = r·y·(1 − y/K) refines exponential growth: r is the intrinsic growth rate and K the carrying capacity. When y is small, (1 − y/K) ≈ 1 and growth is nearly exponential; as y approaches K, the bracket shrinks toward 0 and growth halts, so y levels off at K. It is separable (via partial fractions) with solution y(t) = K / (1 + A·e^(−rt)), where A = (K − y₀)/y₀. The graph is the S-shaped (sigmoid) curve, with an inflection point at y = K/2 where growth is fastest. Equilibria are y = 0 (unstable) and y = K (stable). The logistic model describes constrained populations, technology adoption, and — as the logistic/sigmoid function — the activation in logistic regression and neural networks.
Worked Example 1
Problem. For dy/dt = y(1 − y/100), identify the equilibria and the value of y where growth is fastest.
Answer. Equilibria y = 0 (unstable), y = 100 (stable); fastest growth at y = 50
Worked Example 2
Problem. Write the logistic solution for r = 0.5, K = 1000, y(0) = 100.
Answer. y(t) = 1000/(1 + 9e^(−0.5t))
Worked Example 3
Problem. Using the solution above, find the limiting population and y(0) as a check.
Answer. Limit 1000 (= K); y(0) = 100 checks out
Problem. A fish population follows dy/dt = 0.4·y(1 − y/500) with y(0) = 50. Write y(t) and give the long-run population.
Solution. A = (500 − 50)/50 = 9, r = 0.4, K = 500. So y(t) = 500/(1 + 9e^(−0.4t)). As t → ∞, the exponential vanishes and y → 500, the carrying capacity.
Code solves ODEs two ways: symbolically with dsolve for closed forms, and numerically (Euler/solvers) when none exists.
SymPy's dsolve returns an exact solution when the ODE has one, applying initial conditions with ics. But most real ODEs lack closed forms, so we integrate numerically. Euler's method is the simplest scheme: from dy/dt = f(t, y), step forward by yₙ₊₁ = yₙ + h·f(tₙ, yₙ) with step size h. It is first-order accurate — halving h roughly halves the error — and can drift or go unstable if h is too large. Production code uses higher-order, adaptive solvers like SciPy's solve_ivp (Runge–Kutta), which control error automatically. The workflow mirrors integration: try symbolic first for insight, then validate or replace it with a numerical solver, always checking that smaller steps converge to the same curve.
Worked Example 1
Problem. Solve dy/dt = y with y(0) = 1 symbolically and state the exact solution.
Answer. y(t) = e^t
Worked Example 2
Problem. Take one Euler step for dy/dt = y, y(0) = 1, with h = 0.1, and compare to the exact value.
Answer. y₁ = 1.1 (exact ≈ 1.1052)
Worked Example 3
Problem. Explain why Euler's method accumulates error and how to reduce it.
Answer. Error from the constant-slope assumption; shrink h or use RK4/solve_ivp
Problem. Take two Euler steps for dy/dt = −2y, y(0) = 1, with h = 0.1, and compare to the exact e^(−0.2).
Solution. Step 1: y₁ = 1 + 0.1·(−2·1) = 0.8. Step 2: y₂ = 0.8 + 0.1·(−2·0.8) = 0.8 − 0.16 = 0.64. Exact y(0.2) = e^(−0.4) ≈ 0.6703, so Euler gives 0.64 — close, with error shrinking as h decreases.
Take the logistic ODE dy/dt = r·y·(1 - y/K) with chosen r, K, and y(0). Solve it symbolically with sympy.dsolve, then approximate the same solution with a hand-written Euler's method loop in NumPy. Plot both curves on one axis and study how the Euler step size affects accuracy.
Deliverable · A notebook with the symbolic solution, an Euler-method implementation, an overlay plot of symbolic vs. numerical curves for two step sizes, and a note on the accuracy/cost trade-off.
1. Which equation is a first-order separable differential equation?
Answer A. dy/dx = x·y separates as (1/y) dy = x dx; the others cannot be split into a pure-x times pure-y form (or are higher order).
2. The solution of dy/dt = k·y with y(0) = y₀ is:
Answer B. Exponential growth/decay: separating and integrating gives y = y₀·e^(kt).
3. In the logistic model dy/dt = r·y·(1 - y/K), the constant K represents:
Answer C. K is the carrying capacity — the population level at which growth stops because the bracket (1 - y/K) reaches zero.
4. A slope field is most useful for:
Answer B. Slope fields show the direction solutions follow at each point, revealing qualitative behavior even without a closed form.
5. Euler's method approximates the next value with:
Answer A. Euler steps forward along the current slope: yₙ₊₁ = yₙ + h·f(xₙ, yₙ), where h is the step size.
I can identify and solve a separable first-order differential equation.
I can interpret a slope field and match it to a growth, decay, or logistic model.
I can solve an ODE both symbolically (SymPy) and numerically (Euler/solver) and compare the results.
Many calculus problems have no closed-form answer, so numerical methods approximate them with arithmetic to controllable accuracy.
Symbolic calculus gives exact formulas, but the real world quickly outruns it: integrals like ∫e^(−x²) dx have no elementary antiderivative, most nonlinear equations f(x) = 0 have no algebraic root formula, and real models often come as data or black-box functions with no formula at all. Numerical methods step in: they approximate derivatives, integrals, and roots using finite arithmetic on sampled values, trading exactness for a numeric answer with a controllable error. The cost is two kinds of error — truncation (from approximating a limit with a finite step) and round-off (from finite floating-point precision) — which you manage by choosing step sizes and algorithms wisely. This is the computational core of data science: optimizers, simulators, and ML training are all numerical methods underneath, so understanding their accuracy and limits is essential.
Worked Example 1
Problem. Name a definite integral with no elementary antiderivative and explain how to evaluate it.
Answer. Use numerical quadrature; ∫₀¹ e^(−x²) dx ≈ 0.7468
Worked Example 2
Problem. Explain why x = cos(x) cannot be solved algebraically and how to find the root.
Answer. Numerically, x ≈ 0.7391
Worked Example 3
Problem. You have sensor data sampled every second but no formula for the signal. How do you get its rate of change and total?
Answer. Finite differences for the rate; trapezoidal rule for the total
Problem. Give one example each of a derivative, an integral, and an equation that you would solve numerically rather than symbolically, with a one-line reason.
Solution. Derivative: the slope of a sampled stock-price series (no formula, only data) — use finite differences. Integral: ∫₀² e^(−x²) dx (no elementary antiderivative) — use quadrature. Equation: x − cos x = 0 (transcendental, no algebraic root) — use Newton's method or bisection.
Finite differences estimate derivatives from function values at nearby points; the centered formula is the accurate workhorse.
Finite differences approximate f'(x) using sampled values. The forward difference [f(x+h) − f(x)]/h is first-order accurate (error ∝ h). The centered difference [f(x+h) − f(x−h)]/(2h) is second-order accurate (error ∝ h²) because the leading error terms cancel by symmetry, so it is usually preferred for the same h. A second derivative uses [f(x+h) − 2f(x) + f(x−h)]/h². These formulas come straight from Taylor expansions: substituting the series and canceling shows the order of accuracy. The catch is the trade-off between truncation error (shrinks with smaller h) and round-off from subtracting nearly equal numbers (grows with smaller h), giving a U-shaped total error and an optimal h. Finite differences are the simplest way to differentiate black-box or data-defined functions.
Worked Example 1
Problem. Estimate f'(1) for f(x) = x² with a forward difference, h = 0.01 (exact f'(1) = 2).
Answer. ≈ 2.01 (exact 2)
Worked Example 2
Problem. Estimate f'(1) for f(x) = x² with a centered difference, h = 0.01.
Answer. = 2.00 (exact)
Worked Example 3
Problem. Estimate f''(0) for f(x) = cos x using the second-difference formula with h = 0.1 (exact f''(0) = −1).
Answer. ≈ −0.9992 (exact −1)
Problem. Estimate f'(2) for f(x) = x³ with a centered difference, h = 0.001 (exact f'(2) = 12).
Solution. Centered: [f(2.001) − f(1.999)]/(2·0.001). f(2.001) = 8.012006001, f(1.999) = 7.988011999. Difference = 0.024000002; divide by 0.002 → 12.000 (matches the exact 12 to the precision shown).
Quadrature rules estimate a definite integral by summing simple shapes — trapezoids or parabolic arcs — over a partition.
Numerical integration (quadrature) approximates ∫ₐᵇ f dx from sampled values. The trapezoidal rule connects adjacent points with straight lines: with n equal subintervals of width h = (b − a)/n, T = h·[½f₀ + f₁ + … + f_{n−1} + ½fₙ], and its error scales like h² (second order). Simpson's rule fits a parabola to each pair of subintervals (n must be even): S = (h/3)·[f₀ + 4f₁ + 2f₂ + 4f₃ + … + 4f_{n−1} + fₙ], with error like h⁴ (fourth order) — far more accurate on smooth integrands for the same number of points, and exact for cubics. Both converge to the true integral as n grows; Simpson's faster. These rules power numerical expectation, area, and accumulation computations whenever an antiderivative is unavailable.
Worked Example 1
Problem. Approximate ∫₀¹ x² dx with the trapezoidal rule, n = 2 (exact 1/3).
Answer. ≈ 0.375
Worked Example 2
Problem. Approximate ∫₀¹ x² dx with Simpson's rule, n = 2.
Answer. = 1/3 (exact)
Worked Example 3
Problem. Approximate ∫₀^π sin x dx with Simpson's rule, n = 2 (exact 2).
Answer. ≈ 2.094 (exact 2)
Problem. Approximate ∫₀² (x² + 1) dx with the trapezoidal rule, n = 2 (exact 14/3 ≈ 4.667).
Solution. h = 1; nodes 0, 1, 2 with f = 1, 2, 5. T = h·[½f₀ + f₁ + ½f₂] = 1·[0.5 + 2 + 2.5] = 5.0. (Slight overestimate of the exact 4.667; Simpson's with n = 2 would give the exact value since the integrand is quadratic.)
Root finders locate where f(x) = 0; bisection is slow but sure, Newton's method is fast but needs a good start.
To solve f(x) = 0 numerically, two classic methods trade safety for speed. Bisection requires a bracket [a, b] with f(a) and f(b) of opposite signs; by the Intermediate Value Theorem a root lies between, and repeatedly halving the interval and keeping the sign-changing half traps it. It is guaranteed to converge but only linearly (one bit of accuracy per step). Newton's method iterates xₙ₊₁ = xₙ − f(xₙ)/f'(xₙ), following the tangent line to its x-intercept; it converges quadratically (digits double each step) near a simple root but needs the derivative and a good initial guess, and can diverge or cycle otherwise. A practical strategy combines them: bracket with bisection, then polish with Newton. Root finding underlies solving equilibrium conditions, calibrating models, and maximum-likelihood estimation.
Worked Example 1
Problem. Do two bisection steps for f(x) = x² − 2 on [1, 2] (root √2 ≈ 1.4142).
Answer. Root in [1.25, 1.5], estimate ≈ 1.375
Worked Example 2
Problem. Do one Newton step for f(x) = x² − 2 from x₀ = 2 (f'(x) = 2x).
Answer. x₁ = 1.5
Worked Example 3
Problem. Show Newton converges fast for f(x) = x² − 2: compute x₂ from x₁ = 1.5.
Answer. x₂ ≈ 1.41667 (error ~0.0024)
Problem. Use one Newton step to improve the root of f(x) = x³ − x − 2 from x₀ = 1.5 (f'(x) = 3x² − 1).
Solution. f(1.5) = 3.375 − 1.5 − 2 = −0.125; f'(1.5) = 6.75 − 1 = 5.75. x₁ = 1.5 − (−0.125)/5.75 = 1.5 + 0.02174 ≈ 1.52174. (The true root is ≈ 1.5214, so one step is already very close.)
Numerical accuracy is a balance: shrinking the step cuts truncation error but eventually amplifies floating-point round-off.
Every numerical method carries two error sources. Truncation error comes from replacing a limit with a finite step — it shrinks predictably as the step h decreases (∝ h for forward differences, ∝ h² centered, ∝ h⁴ Simpson's). Round-off error comes from finite floating-point precision (about 16 digits, machine epsilon ≈ 2.2×10⁻¹⁶); it grows as h shrinks because formulas subtract nearly equal numbers — catastrophic cancellation. The total error is therefore U-shaped in h: it falls, hits a minimum at an optimal h, then rises. For a forward difference the optimum is around h ≈ √ε ≈ 1e-8; for centered, around ε^(1/3) ≈ 1e-5. Practical defenses: prefer higher-order or algebraically stable formulas, avoid subtracting close values, and never assume 'smaller h is always better.' This balance is the heart of reliable numerical computation.
Worked Example 1
Problem. Estimate f'(1) for f(x) = eˣ (exact e ≈ 2.71828) with a forward difference at h = 1e-1, 1e-8, 1e-15.
Answer. Best near h ≈ 1e-8; worse at both extremes
Worked Example 2
Problem. Why does computing (1 − cos h)/h² for tiny h lose accuracy, and what is a fix?
Answer. Cancellation; fix with 1 − cos h = 2 sin²(h/2)
Worked Example 3
Problem. Order these by accuracy for the same small h: forward difference, centered difference, Simpson integration.
Answer. Most to least accurate: Simpson (h⁴) > centered (h²) > forward (h)
Problem. For a forward-difference derivative, the truncation error ~ M·h/2 and round-off ~ 2ε/h (ε ≈ 1e-16). Estimate the h that minimizes total error (take M ≈ 1).
Solution. Total ≈ h/2 + 2ε/h. Differentiate w.r.t. h and set to 0: 1/2 − 2ε/h² = 0 ⇒ h² = 4ε ⇒ h = 2√ε. With ε ≈ 1e-16, √ε = 1e-8, so optimal h ≈ 2e-8 — matching the rule of thumb h ≈ √ε for forward differences.
Combining differentiation, integration, and root-finding routines yields a small but complete numerical calculus toolkit.
A practical toolkit packages the core methods as reusable functions: a centered finite-difference derivative, a trapezoidal (or Simpson) integrator, and bisection plus Newton root-finders. Good engineering principles apply: validate each routine against a known exact case (e.g., d/dx[x²] at 1 is 2; ∫₀¹ x² dx = 1/3; the root of x² − 2 is √2), expose the step size or tolerance as a parameter, and report or check error. Real projects lean on battle-tested libraries — NumPy (np.gradient, np.trapz), SciPy (integrate.quad, integrate.solve_ivp, optimize.brentq/newton) — which add adaptivity and error control. The value of building the toolkit yourself is conceptual: you see exactly how calculus becomes arithmetic, how accuracy depends on step size, and when to trust a number. That understanding is what lets you debug optimizers and simulators in data science.
Worked Example 1
Problem. Validate a centered-difference derivative routine on f(x) = x² at x = 1 (exact 2).
Answer. ≈ 2.0000 (matches exact)
Worked Example 2
Problem. Validate a trapezoidal integrator on ∫₀¹ x² dx (exact 1/3) with n = 1000.
Answer. ≈ 0.3333335 (error ~1.7e-7)
Worked Example 3
Problem. Validate a Newton root-finder on f(x) = x² − 2 starting at x₀ = 2 to find √2.
Answer. ≈ 1.4142136 (matches √2 to ~7 digits)
Problem. Outline a single test cell that validates a derivative, an integrator, and a root-finder against known answers, and state the expected results.
Solution. Test deriv(lambda x: x**2, 1, 1e-5) → expect ≈ 2; test trap(lambda x: x**2, 0, 1, 1000) → expect ≈ 1/3 ≈ 0.33333; test newton(lambda x: x**2 - 2, lambda x: 2*x, 2) → expect ≈ √2 ≈ 1.41421. Assert each result is within a small tolerance (e.g., 1e-4) of the exact value; if all pass, the toolkit is trustworthy on these benchmarks.
Implement four functions in NumPy: a centered finite-difference derivative, a trapezoidal integrator, a bisection root-finder, and a Newton's-method root-finder. Test each on a known example (e.g., f(x) = x² - 2 whose root is √2) and report the error against the exact value. For the derivative, sweep step size h and find where round-off error starts to dominate.
Deliverable · A single module/notebook with the four functions, a test cell comparing each to a known exact answer, and a plot of finite-difference error versus h showing the U-shaped truncation-plus-round-off curve.
1. The centered finite-difference approximation of f'(x) is:
Answer B. The centered difference [f(x+h) - f(x-h)]/(2h) cancels the leading error term and is more accurate than one-sided differences.
2. Compared to the trapezoidal rule, Simpson's rule generally:
Answer B. Simpson's rule fits parabolas instead of straight lines, achieving higher accuracy on smooth integrands for the same partition.
3. Newton's method updates an estimate using:
Answer A. Newton's method follows the tangent line to its x-intercept: xₙ₊₁ = xₙ - f(xₙ)/f'(xₙ).
4. A key requirement for the bisection method to work is that:
Answer B. Bisection relies on the Intermediate Value Theorem: a continuous f with opposite signs at the endpoints must have a root between them.
5. As the finite-difference step size h is made extremely small, the total error eventually:
Answer C. Truncation error shrinks with smaller h, but subtracting nearly equal numbers amplifies floating-point round-off, so total error eventually rises.
I can implement finite-difference derivatives and trapezoidal/Simpson integration from scratch.
I can find a root with bisection and Newton's method and explain their convergence behavior.
I can reason about how step size and floating point limit numerical accuracy.
Where this leads
Part of the open-source Crunch Academy tech-education path. Pair it with the data-science and ML tracks when you're ready.
All Math for Data Science courses Crunch Academy home