Numerical Methods: How They Work in Practice

Build a working mental model of numerical methods by tracking where error comes from, how it grows or shrinks, and which tradeoffs control accuracy and stability. Learn why some algorithms converge fast yet fail suddenly, and how to sanity-check results before you trust them.

Numerical methods can feel like cheating because they replace exact math with approximations, yet still produce answers you can engineer with. The trick is that every approximation carries error you can predict, bound, and sometimes amplify by accident. What makes this practical is also what makes it dangerous. The same method can look perfect on one problem and collapse on a nearby one because of rounding, ill-conditioning, or a bad step size. Once you see where those failures come from, numerical methods stop feeling mysterious and start feeling testable.

Controlled error, not hand-waving

Most numerical methods are built from a local approximation repeated many times. Each repetition introduces truncation error, and each floating-point operation introduces roundoff error. You do not get to remove either one. You only choose how they trade off as you change a step size, a tolerance, or an iteration limit.

A common surprise is that making the step size smaller does not always improve accuracy. Truncation error often shrinks with step size, but roundoff accumulates and can eventually dominate. The result is a U-shaped error curve where there is a sweet spot rather than a monotone improvement.

Take a look at how error can change as step size changes.

Once you have seen that curve once, many later choices become less emotional. If your result gets worse when you tighten a step, you are not cursed. You are likely sliding from truncation-dominated to roundoff-dominated behavior.

Sweet spot
Accuracy is often best at an intermediate step size. Smaller can mean noisier when roundoff and cancellation start to dominate.

Floating-point explains the weird disagreements

Floating-point arithmetic follows the IEEE 754 standard, which means most real numbers are stored as nearby representable numbers. Every arithmetic operation rounds to a representable value. That tiny rounding is usually harmless, until your formula magnifies it.

Three ideas connect most numerical surprises.

• Rounding: every operation nudges the exact answer to a nearby representable value.
• Cancellation: subtracting nearly equal numbers discards significant digits, turning small rounding into large relative error.
• Conditioning: the problem itself may amplify input or intermediate perturbations, even with a perfect algorithm.
• Stability: an algorithm is stable if it does not amplify unavoidable floating-point perturbations more than the problem already does.

See how rounding, cancellation, conditioning, and stability combine into the error you observe.

A practical way to think about it is that conditioning is about the question you asked, while stability is about how you computed the answer. A stable method can still give a poor result on an ill-conditioned problem, and an unstable method can ruin a well-conditioned one.

Root finding methods and their failure modes

Root finding solves $f(x)=0$ by iteratively improving a guess. The tradeoff is simple.

Bisection is slow but predictable. Newton’s method is fast when it works, and brittle when its assumptions fail. The secant method sits between them by estimating derivatives from past points.

Bisection, Newton, secant in one view

Bisection maintains a bracket where $f(a)$ and $f(b)$ have opposite signs, then repeatedly halves the interval. If $f$ is continuous, it converges, period. Newton updates $x_{k+1}=x_k-\frac{f(x_k)}{f'(x_k)}$, which can converge quadratically near a simple root, but it can also diverge, cycle, or jump to a different root. Secant replaces $f'(x)$ with a finite difference slope, reducing derivative cost but adding sensitivity to the last two iterates.

Explore how starting guesses and function shape change convergence or divergence.

If Newton fails, the culprit is often not Newton itself but a mismatch between method assumptions and reality. Derivatives can be tiny or noisy, roots can be multiple, or your initial guess can land where the tangent points away from the root. In practice, many robust solvers blend methods, using bracketing to stay safe and Newton-like steps to accelerate.

Guardrails
When you can bracket a root, keep the bracket. It turns root finding from hope into an invariant you can test each iteration.

Solving $Ax=b$ without burning memory

Linear systems are where numerical methods meet performance constraints. You can often choose between direct methods, which aim to finish in a predictable number of operations, and iterative methods, which aim to use cheap iterations and exploit structure like sparsity.

Gaussian elimination transforms $A$ to an upper triangular form, then back-substitutes. In floating point, elimination without pivoting can be catastrophically unstable even when the exact math is fine. Pivoting swaps rows or columns to avoid dividing by small numbers and to control growth in intermediate values.

Iterative methods, such as conjugate gradient or GMRES, never form all the fill-in that elimination can create. They can be dramatically faster for large sparse systems, but they introduce new tuning knobs like tolerances and preconditioners. They can also stagnate on ill-conditioned problems.

Compare how dense versus sparse and direct versus iterative choices shift cost, memory, and sensitivity.

A useful rule is that sparsity is not just about storage. It is about which operations you can do without creating new nonzeros. Direct methods can destroy sparsity through fill-in, while iterative methods can preserve it but may need preconditioning to converge in a reasonable number of iterations.

Integration, differentiation, and step size reality

Numerical integration and differentiation turn smooth calculus operations into discrete sums and differences. They inherit the same truncation versus roundoff tension as earlier, with an extra twist. Dynamics can introduce stability limits that make small steps mandatory.

Quadrature methods approximate $\int_a^b f(x)\,dx$ by weighted sums of function values. Higher-order rules reduce truncation error on smooth functions, but can perform poorly when the function has sharp features you are not resolving. Finite differences approximate derivatives using nearby points, for example $\frac{f(x+h)-f(x)}{h}$. Small $h$ reduces truncation error but increases cancellation and roundoff.

Stiffness is the classic case where accuracy and stability separate. A method can be accurate in principle, yet unstable unless the step size is tiny. In stiff ordinary differential equations, explicit methods often need impractically small steps, while implicit methods trade per-step cost for stability.

Explore how step size changes accuracy and stability on smooth versus stiff examples.

When results oscillate or blow up as you step forward, do not immediately blame your model. Check whether you are violating a stability condition for your method. When results look plausible but drift, suspect truncation error and step size choice.

Optimization that does not get stuck forever

Optimization methods search for parameters that minimize a function, often written as $\min_x f(x)$. The most common workhorse is gradient descent, which updates $x_{k+1}=x_k-\alpha_k\nabla f(x_k)$ with a step size $\alpha_k$. The step size is the whole method. Too large and you diverge or bounce. Too small and you crawl.

Line search picks $\alpha_k$ to ensure progress, often using conditions like sufficient decrease. Constraints change the geometry. A method can make a good descent step that still violates feasibility, so constrained methods project, use penalty terms, or solve subproblems that respect constraints.

Non-convergence usually has a cause you can name.

• Bad scaling, so gradients point the right way but step sizes behave wildly across coordinates
• Noisy or approximate gradients, common in simulation or stochastic settings
• Nonconvex landscapes, where saddle points and flat regions slow progress
• Hidden constraints, where the true feasible set is smaller than you think

Use a guided set of prompts to map your problem to a method choice and concrete diagnostics.

The fastest improvement you can make in practice is often not switching algorithms. It is plotting or logging the norm of the gradient, the step size, and the objective value each iteration. Those three signals tell you whether you are stuck, diverging, or simply under-stepping.

Reliability checklist you can reuse everywhere

Trust comes from habits that catch the common failure modes early. A lightweight checklist that works across root finding, linear solves, integration, and optimization looks like this.

• Make error estimates explicit, even crude ones, so you know what accuracy claim you are making.
• Use stopping criteria tied to the problem, not just iteration counts. Check residuals like $\|Ax-b\|$ or $|f(x)|$, not only step size.
• Do a parameter sweep on tolerances or step sizes to see whether results stabilize in a range, not just at one setting.
• Run at least one sanity check with a known special case, symmetry, conservation law, or limiting behavior.
• Perturb inputs slightly and see whether outputs change as conditioning would predict.
• Cross-check with a different method when stakes are high, especially when conditioning is unknown.

The mindset is not distrust. It is controlled skepticism backed by cheap experiments. If you can make a result change on purpose by adjusting a knob, you can usually make it stop changing for the right reason too.