Scientific and Numerical Computing

Zusammenfassung

Before computers calculated payrolls or served web pages, they did sums — for ballistics, for bombs, for the structure of matter. Scientific computing is the oldest application of the machine and, in a sense, its reason for existing: the ENIAC, the EDSAC, and von Neumann’s IAS machine were all built to solve equations no human team could finish in time. Out of that work grew numerical analysis — the mathematics of computing real answers with finite, error-prone arithmetic — and a software stack (Fortran, BLAS, LINPACK, LAPACK, MATLAB) that still underlies almost every simulation run today. This is the story of how the world learned to compute the continuous on a discrete machine.

The Original Application

The first electronic computers were numerical instruments. ENIAC (1945) was commissioned to produce artillery firing tables; its very first serious problem, run in late 1945, was a calculation for the hydrogen bomb. John von Neumann, who shaped the stored-program architecture, spent the rest of his life treating the computer chiefly as a tool for applied mathematics — hydrodynamics, shock waves, weather (see Weather and Climate Modeling), and the nonlinear physics of explosions.

The defining trait of scientific computing is that it models the continuous — fluids, fields, orbits, populations — on a machine that can only store finitely many digits and take finitely many steps. Every such computation is therefore an approximation, and the central question is not “what is the answer?” but “how wrong is the answer, and can we bound the error?” That question is the subject of numerical analysis.

Monte Carlo: Computing by Chance

The Manhattan Project produced not only the bomb but one of the most widely used algorithms in science. Stuck on neutron-diffusion problems too tangled for closed-form solution, Stanislaw Ulam proposed in the mid-1940s that they be solved by random sampling — simulating thousands of random neutron histories and averaging the outcomes. John von Neumann saw how to run it on ENIAC, and Nicholas Metropolis named the technique the Monte Carlo method, after the casino where Ulam’s uncle gambled. The name and method appeared in print in a 1949 Metropolis–Ulam paper.

Monte Carlo inverted the usual relationship between computers and randomness: instead of fighting noise, it used randomness as the engine of computation. The 1953 Metropolis algorithm for sampling from probability distributions became a cornerstone of computational physics and chemistry, and its descendants (Markov chain Monte Carlo) power everything from financial risk models to modern Bayesian statistics (see Judea Pearl and the Causal Revolution).

The Tyranny of Floating Point

Real numbers do not fit in a computer. The standard compromise is floating-point arithmetic — a sign, a fixed-width fraction, and an exponent — which trades exactness for range. The trouble is that floating-point arithmetic breaks the ordinary laws of algebra: addition is not associative, subtraction of nearly equal numbers destroys precision (“catastrophic cancellation”), and naïve code can produce confidently wrong answers.

The pioneer James H. Wilkinson at the UK’s National Physical Laboratory turned the analysis of these errors into a rigorous discipline. His technique of backward error analysis — asking not “how far is the computed answer from the true one?” but “for what slightly perturbed problem is our answer the exact solution?” — made it possible to certify numerical algorithms. Wilkinson received the Turing Award in 1970 for his work on numerical linear algebra and rounding-error analysis.

For decades, every computer manufacturer implemented floating point differently, so the same program gave different answers — or crashed — on different machines. William Kahan led the effort that produced the IEEE 754 standard in 1985, defining bit layouts, rounding modes, and special values (NaN, signed infinities) that almost every processor since has implemented. Portable, predictable numerics is so taken for granted today that it is easy to forget it had to be fought for; Kahan received the Turing Award in 1989.

The Software Stack: Fortran, BLAS, LINPACK, LAPACK

Scientific computing built the first great reusable-software ecosystem. Fortran (1957), designed at IBM under John Backus, was created precisely so scientists could write formulas instead of assembly — and it remains in heavy use in high-performance codes seven decades later.

On top of it came a deliberately layered numerical-library stack:

BLAS (Basic Linear Algebra Subprograms, from 1979) — standardized the low-level vector and matrix operations so chip vendors could hand-tune them.
LINPACK (late 1970s) and its successor LAPACK (1992) — solved linear systems, least-squares, and eigenvalue problems by calling BLAS, giving portable performance.
MATLAB — Cleve Moler wrote it in the 1970s as a teaching front-end to LINPACK and EISPACK so students could use linear algebra without writing Fortran; commercialized by MathWorks in 1984, it became the lingua franca of engineering computation.

The LINPACK benchmark, derived from that library, became the yardstick for ranking the world’s fastest machines: the TOP500 list, published twice a year since 1993, measures supercomputers by how fast they solve a dense linear system (see The Supercomputer Era and Seymour Cray and Supercomputing).

Discretizing the Continuous

Most physical laws are partial differential equations — heat flow, fluid motion, electromagnetism, structural stress. Solving them numerically means replacing the continuum with a grid or mesh and the derivatives with differences:

Finite difference methods approximate derivatives by differences between neighboring grid points — the approach of the first weather forecasts.
The finite element method, developed in the 1950s–60s for aircraft and civil-engineering structural analysis, breaks a domain into small elements and is now standard in mechanical and electromagnetic simulation.
Krylov subspace methods (such as the conjugate gradient method, 1952) and other iterative solvers made it feasible to solve the enormous sparse linear systems these discretizations produce.

These techniques turned the computer into a “third pillar” of science alongside theory and experiment: the numerical wind tunnel, the in-silico drug trial, the simulated galaxy. Whole fields — computational fluid dynamics, computational chemistry, lattice quantum chromodynamics — exist only because the equations were finally tractable.

High-Performance Computing

Scientific problems are insatiable: finer grids, longer simulations, more particles. That hunger drove supercomputing from Cray’s vector machines through massively parallel clusters to today’s GPU-accelerated, exascale systems. Programming them spawned its own standards — MPI (message passing, 1994) for distributed memory and OpenMP (1997) for shared memory — and its own hard problems: keeping tens of thousands of processors busy without drowning in communication. The same workloads now share hardware with AI training, and scientific breakthroughs like AlphaFold increasingly fuse numerical simulation with machine learning.

⚠️ Dead End: The Dream of Exact Real Arithmetic

A recurring temptation has been to escape floating-point error entirely — to compute with exact real numbers. Interval arithmetic (carrying rigorous lower and upper bounds), exact real-number computation, and symbolic-only approaches have all been pursued. They work, but at a price: intervals tend to grow pessimistically wide, and exact arithmetic is often orders of magnitude slower. For the overwhelming majority of large-scale science, the field instead doubled down on the messier path — well-analyzed approximate arithmetic, with error bounds proved by hand. More recently, the pendulum swung the other way: machine-learning and graphics workloads showed that for many problems lower precision (16-bit, 8-bit, even smaller) is good enough and far faster, making numerical precision a tunable knob rather than a fixed virtue. The lesson of numerical computing is consistent across seventy years — the goal was never exactness, but controlled inexactness.