Calculus of Variations – Part 1 of 2: Give It a Wiggle
Benjamin Skuse
We have all been in the situation where we cannot get a gadget or device to work. We have tried every button, lever and switch. We have watched every ‘how-to’ YouTube video, and still it will not work. Our last resort? Give it a good old shake. This is the fundamental essence of the calculus of variations: We wiggle a functional (a function of functions) a little bit to be able to identify the stationary values (maxima, minima, saddle points, possibly global or local) of that functional. Or put another way, these stationary values are the values for which, if you wiggle them, the functional does not change (to first order).
Why would we want to calculate the stationary values of a functional? It turns out that a multitude of real-world problems can be stated as minimising or maximising a functional, from understanding soap bubbles to calculating the softest possible spacecraft landing. But to truly understand why the technique is so powerful across a wide variety of practical applications, we need to rewind back over 300 years to the birth of the calculus of variations.
Titans of Mathematics
Before the theory of the calculus of variations had been formulated, British polymath Sir Isaac Newton can lay claim to stating and solving the first problem in the field. In his 1685 opus the Principia, Newton asked: What is the shape of a body that renders its resistance minimal when moving through a fluid at a given constant velocity? This question was and is important in the design of gun shells, for example, where designers want to create a shape with the least air resistance possible.
Newton’s solution involved a complicated geometric technique that few, if any, of Newton’s contemporaries could fathom. Eventually in 1691, Newton was persuaded to explain how he solved the problem, which was just in time to be applied to another problem that really kicked off the calculus of variations: the Brachistochrone.
In 1696, Swiss mathematician Johann Bernoulli posed a problem in Acta Eruditorum, for which he hoped “to gain the gratitude of the whole scientific community by placing before the finest mathematicians of our time a problem which will test their methods and the strength of their intellect”:
Given two points A and B in a vertical plane, what is the curve traced out by a point acted on only by gravity, which starts at A and reaches B in the shortest time.
A more practical way of describing the problem is to ask: Disregarding friction, what is the shape of a wire joining two given points, so that a bead will slide down the wire under gravity from one point to the other in the shortest time?
It turns out that the wire shape is a segment of a cycloid, which is the curve traced out by a point on the rim of a circular wheel rolling in a straight line. In addition to Johann Bernoulli, who had already solved it himself, four solutions were obtained by his brother Jacob Bernoulli, German polymath Gottfried Leibniz, French mathematician Guillaume de l’Hôpital and Newton, the latter of whom finding the test particularly irritating, stated: “I do not love to be dunned [pestered] and teased by foreigners about mathematical things.”
The Euler–Lagrange Equation
Though all of these titans developed cutting-edge methods in their solutions, each solution was found by considering just this particular problem. It took decades to generalise problems of this type. In 1744, Swiss polymath Leonhard Euler found what has now come to be known as the Euler–Lagrange differential equation for a function of the maximising or minimising function and its derivative. Alongside finally posing such problems correctly, he also presented geometrical methods for attacking them, though these methods only gave necessary conditions that a solution had to satisfy, and did not identify whether a solution existed or not.
Corresponding extensively with Euler, it was Italian mathematician and physicist Joseph-Louis Lagrange in 1760 who finally provided the necessary tools and analytic method to attack what would become known as calculus of variations problems. Lagrange invented the method of variations and introduced the concept of the first variation \( \delta S \) of a functional \( S \).
\( \delta S \) is analogous to the first derivative of a function. In other words, it is a small change (variation) in an entire function, which is the input for the functional \( S \). Looking at how a functional changes if you give its input function a little wiggle was central to Lagrange’s thinking, and unlocked the power of the calculus of variations. Lagrange’s innovations also provided the means to express the Euler–Lagrange equation in the form it is written today:
For a given functional:
\[ S[y] = \int_{a}^{b} F(x,y,y’) \,dx, \qquad y(a) = A, \quad y(b) = B, \]
stationary values are given by solutions to the following Euler–Lagrange equation:
\[ \frac{d}{dx}\left( \frac{\partial F}{\partial y’} \right) – \frac{\partial F}{\partial y} = 0 \].
All of a sudden, solutions to very general problems looking for stationary values of a functional could be investigated for the first time in a rapid and methodical way in four steps:
1. Write down a functional \( S[y] \) describing the problem in the form of a definite integral over some function \( F(x, y, y’) \).
2. Identify \( F(x, y, y’) \).
3. Write the Euler–Lagrange equation for \( F \).
4. Solve the resulting differential equation for \( y(x) \): the function making the given functional stationary.
An Example
A very insightful example is the aforementioned Brachistochrone problem. For simplicity, we can ask what the shape of a wire \( y(x) \) joining two points \( (0,0) \) and \( (b,1) \) is, so that a bead of mass \(m \) will slide down the wire under gravity in the shortest time, assuming energy is conserved because we neglect friction.
At any given point along the wire \( P = (x, y(x))\), we know a great deal about the bead, including its speed \(v = \frac{ds}{dt} \), where \(s \) is distance and \( t \) is time; its kinetic energy \( \frac{1}{2}mv^{2}\); and its potential energy \( -mgy \). This allows us, after some algebra, to take step 1 and derive the functional for the time taken for a bead to slide down the wire with shape \( y(x) \), starting from rest at the origin:
\[ T[y] = \frac{1}{\sqrt{2g}} \int_{0}^{b} \sqrt{\frac{1 + (y’)^2}{y}}dx, \qquad y(0) = 0, \quad y(b) = 1.\]
For steps 2 and 3, as the functional is clearly independent of \(x \), \( F(x, y, y’) \) is
\[ F(y,y’) = \frac{1}{\sqrt{2g}}\sqrt{\frac{1 + (y’)^2}{y}}\]
and the Euler–Lagrange equation can be simplified to \( y’\frac{\delta F}{\delta y’} – F = c,\) where \(c\) is a constant, to give after a couple of algebraic steps:
\[\int x = \int\sqrt{\frac{y}{c^2 – y}}dy.\]
Attempting step 4, the left-hand side is equal to \( x\) and the right-hand side can be solved by substituting \(y\) for \( c^2\sin^2\sigma\), finally providing parametric solutions that minimise the functional \(T[y]\):
\[x = \frac{1}{2}c^2(2\sigma – \sin2\sigma), \qquad y = \frac{1}{2}c^2(1 – \cos2\sigma)\].
As it happens, the above solution to the Brachistochrone problem is identical via simple transformation to the equation of the curve traced by a point \(P\) on the rim of a circular wheel as it rolls along a straight line – the cycloid: \(x = r(\theta – \sin\theta), y = r(1 – \cos\theta)\), i.e. the movement of a rolling wheel and the most efficient path of a body traversing a wire under gravity are geometrically identical.
Rolling but Still on Rocky Ground
Beyond the Brachistochrone problem, a vast number of problems in mechanics could be and were soon being reformulated in terms of stationary value problems and solved by the calculus of variations in this way. In fact, it was Euler who made the key breakthrough in this direction, introducing the notion of the principle of least action, otherwise known as Hamilton’s principle, named after Irish mathematician Sir William Rowan Hamilton. In woolly terms, this means nature invariably chooses the most efficient way of completing a task. Through Hamilton’s principle, all mechanics can be reformulated in terms of stationary value problems and solved by the calculus of variations (more on this in Part II).
However, the very foundations of the calculus of variations were still on rocky ground and mathematicians didn’t even realise it. For 18th Century mathematicians, the existence and uniqueness of solutions to the kind of variational problems they posed were obvious, as they could be verified experimentally, and they had not even considered defining the domain of a given functional and of admissible functions.
Almost a century later, German mathematicians Carl Jacobi and Karl Weierstrass set the record straight, discovering sufficient conditions for stationary values of functionals, with the latter rebuilding the calculus of variations from the ground up. Most importantly, Weierstrass removed the worrying paradox in the calculus of variations that a solution has to be assumed to exist in order to derive said solution.
Weierstrass’ solid foundations set the stage for the likes of David Hilbert and Marston Morse to develop the calculus of variations even further in the 20th Century. Morse, in particular, extended the calculus of variations using topological concepts and methods in qualitative studies of variational problems to provide a relationship between the critical points of certain smooth functions on a manifold (a topological space that locally resembles Euclidean space near each point) and the topology of the manifold. In simpler terms, it offered a way to study a manifold by analysing functions defined on that manifold. Morse theory’s ability to decompose high-dimensional shapes into simpler components makes it invaluable in fields requiring algorithms and detailed analysis of complex forms and structures, such as in medical imaging, material science and robotics.
Further generalization of the calculus of variations led to optimal control theory, which studies similar problems but from a dynamic viewpoint, allowing practical constraints to be incorporated naturally in the problem formulation. Soviet mathematician Lev Pontryagin and US applied mathematician Richard Bellman are credited with developing the method in the 1950s. Given its capability to systematically approach decision-making and control processes, optimal control theory has since been wielded in dynamic programming in a huge range of scenarios. This has helped ensure that the calculus of variations continues to find new and important applications across science and society, as will be discussed in Part II.
The post Calculus of Variations – Part 1 of 2: Give It a Wiggle originally appeared on the HLFF SciLogs blog.