1. Defining Chaos: Determinism, Nonlinearity and Sensitive Dependence

The mathematical phenomenon of chaos is studied in sciences as diverse as astronomy, meteorology, population biology, economics and social psychology. While there are few (if any) causal mechanisms such diverse disciplines have in common, the phenomenological behavior of chaos—e.g., sensitivity to the tiniest changes in initial conditions or seemingly random and unpredictable behavior that nevertheless follows precise rules—appears in many of the models in these disciplines. Observing similar chaotic behavior in such diverse fields certainly presents a challenge to our understanding of chaos as a phenomenon.
1.1 A Brief History of Chaos

Arguably, one can say that Aristotle was already aware of something like what we now call sensitive dependence. Writing about methodology and epistemology, he observed that “the least initial deviation from the truth is multiplied later a thousandfold” (Aristotle OTH, 271b8). Nevertheless, thinking about how small disturbances might grow explosively to produce substantial effects on a physical system's behavior became a phenomenon of ever intensifying investigation beginning with a famous paper by Edward Lorenz (1963), where he noted that a particular meteorological model could exhibit exquisitely sensitive dependence on small changes in initial conditions. Ironically, the framework for formulating questions about sensitive dependence had been articulated in 1922 by French mathematician Jacques Hadamard, who argued that any solution exhibiting sensitive dependence was a sign of a mathematical model that incorrectly described its target system.

However, some scientists and mathematicians prior to Lorenz had examined this phenomena though these were basically isolated investigations never producing a recognizable, sustained field of inquiry as happened after the publication of Lorenz's seminal paper. Sensitive dependence on initial conditions (SDIC) for some systems had already been identified by James Clerk Maxwell (1876, p. 13). He described such phenomena as being cases where the “physical axiom” that from like antecedents flow like consequences is violated. For the most part, Maxwell thought this kind of behavior would be found only in systems with a sufficiently large number of variables (possessing a sufficient level of complexity in this numerical sense). Henri Poincaré (1913), on the other hand, later recognized that this same kind of behavior could be realized in systems with a small number of variables (simple systems exhibiting very complicated behavior). Pierre Duhem, relying on work by Hadamard and Poincaré, further articulated the practical consequences of SDIC for the scientist interested in deducing mathematically precise consequences from mathematical models (1982, pp. 138–142).

Poincaré discussed examples that, in hindsight, we can view as raising doubts about taking explosive growth of small effects to be a sufficient condition for defining chaos. First, consider a perfectly symmetric cone perfectly balanced on its tip with only the force of gravity acting on it. In the absence of any impressed forces, the cone would maintain this unstable equilibrium forever. It is unstable because the smallest nudge, from an air molecule, say, will cause the cone to tip over, but it could tip over in any direction due to the slight differences in various perturbations arising from suffering different collisions with different molecules. Here, variations in the slightest causes issue forth in dramatically different effects (a violation of Maxwell's physical axiom). If we were to plot the tipping over of the unstable cone, we would see that from a small ball of starting conditions, a number of different trajectories issuing forth from this ball would quickly diverge from each other.

The concept of nearby trajectories diverging or growing away from each other plays an important role in discussions of chaos. Three useful benchmarks for characterizing divergence are linear, exponential and geometric growth rates. Linear growth can be represented by the simple expression y = ax+b, where a is an arbitrary positive constant and b is an arbitrary constant. A special case of linear growth is illustrated by stacking pennies on a checkerboard (a = 1, b = 0). If we use the rule of placing one penny on the first square, two pennies on the second square, three pennies on the third square, and so forth, we will end up with 64 pennies stacked on the last square. The total number of pennies on the checkerboard will be 2080. Exponential growth can be represented by the expression y = noeax, where no is some initial quantity (say the initial number of pennies to be stacked) and a is an arbitrary positive constant. It is called ‘initial’ because when x = 0 (the ‘initial time’), we get y = n0. Going back to our penny stacking analogy (a = 1), we again start with placing 1 penny on the first square, but now about 2.7 pennies are stacked on the second square, about 7.4 pennies on the third square, and so forth, and we finally end up with about 6.2 × 1027 pennies staked up on the last square! Clearly, exponential growth outpaces linear very rapidly. Finally, we have geometric growth, which can be represented by the expression y = abx, where a and b are arbitrary positive constants. Note that in the case a = e and b = 1, we recover the exponential case.[1]

Many authors consider an important mark of chaos to be trajectories issuing from nearby points diverging from one another exponentially quickly. However, it is also possible for trajectory divergence to be faster than exponential. Take Poincaré's example of a molecule in a gas of N molecules. If this molecule suffered the slightest of deviations from its initial starting point and you compared the molecule's trajectories from these two slightly different starting points, the resulting trajectories would diverge at a geometric rate, to the nth power, due to the n subsequent collisions, each being different than what it would have been had there been no slight change in the initial condition.

A third example discussed by Poincaré is of a man walking on a street on his way to his business. He starts out at a particular time. Meanwhile unknown to him, there is a tiler working on the roof of a building on the same street. The tiler accidentally drops a tile, killing the business man. Had the business man started out at a slightly earlier or later time, the outcome of his trajectory would have been vastly different!
1.2 Defining Chaos

Many intuitively think that the example of the business man is qualitatively different from Poincaré's other two examples and has nothing to do with chaos at all. However, the cone unstably balanced on its tip that begins to fall also is not a chaotic system as it has no other identifying features usually picked out as belonging to chaotic dynamics, such as nonlinear behavior (see below). Furthermore, it only has one unstable point—the tip—whereas chaos usually requires instability at nearly all points in a region (see below). To be able to identify systems as chaotic or not, we need a definition or a list of distinguishing characteristics. But coming up with a workable, broadly applicable definition of chaos has been problematic.
1.2.1 Dynamical Systems and Determinism

To begin, chaos is typically understood as a mathematical property of a dynamical system. A dynamical system is a deterministic mathematical model, where time can be either a continuous or a discrete variable. Such models may be studied as mathematical objects or may be used to describe a target system (some kind of physical, biological or economic system, say). I will return to the question of using mathematical models to represent real-world systems throughout this article.

For our purposes, we will consider a mathematical model to be deterministic if it exhibits unique evolution:

(Unique Evolution)
A given state of a model is always followed by the same history of state transitions.

A simple example of a dynamical system would be the equations describing the motion of a pendulum. The equations of a dynamical system are often referred to as dynamical or evolution equations describing the change in time of variables taken to adequately describe the target system (e.g., the velocity as a function of time for a pendulum). A complete specification of the initial state of such equations is referred to as the initial conditions for the model, while a characterization of the boundaries for the model domain are known as the boundary conditions. A simple example of a dynamical system would be the equations modeling the flight of a rubber ball fired at a wall by a small cannon. The boundary condition might be that the wall absorbs no kinetic energy (energy of motion) so that the ball is reflected off the wall with no loss of energy. The initial conditions would be the position and velocity of the ball as it left the mouth of the cannon. The dynamical system would then describe the flight of the ball to and from the wall.

Although some popularized discussions of chaos have claimed that it invalidates determinism, there is nothing inconsistent about systems having the property of unique evolution and exhibiting chaotic behavior. While it is true that apparent randomness can be generated if the state space (see below) one uses to analyze chaotic behavior is coarse-grained, this produces only an epistemic form of nondeterminism. The underlying equations are still fully deterministic. If there is a breakdown of determinism in chaotic systems, that can only occur if there is some kind of indeterminism introduced such that the property of unique evolution is rendered false (e.g., §4 below).
1.2.2 Nonlinear Dynamics

The dynamical systems of interest in chaos studies are nonlinear, such as the Lorenz model equations for convection in fluids:

(Lorenz)
dx
dt
= −σx+σy;

dy
dt
= −xz+rz−y;

dz
dt
= xy+bz

A dynamical system is characterized as linear or nonlinear depending on the nature of the equations of motion describing the target system. For concreteness, consider a differential equation system, such as dx ⁄ dt = Fx for a set of variables x = x1, x2,…, xn. These variables might represent positions, momenta, chemical concentration or other key features of the target system, and the system of equations tells us how these key variables change with time. Suppose that x1(t) and x2(t) are solutions of the equation system dx ⁄ dt = Fx. If the system of equations is linear, it can easily be shown that x3(t) = ax1(t) + bx2(t) is also a solution, where a and b are constants. This is known as the principle of linear superposition. So if the matrix of coefficients F does not contain any of the variables x or functions of them, then the principle of linear superposition holds. If the principle of linear superposition holds, then, roughly, a system behaves linearly if any multiplicative change in a variable, by a factor a say, implies a multiplicative or proportional change of its output by a. For example, if you start with your stereo at low volume and turn the volume control a little bit, the volume increases a little bit. If you now turn the control twice as far, the volume increases twice as much. This is an example of a linear response. In a nonlinear system, such as (Lorenz), linear superposition fails and a system need not change proportionally to the change in a variable. If you turn your volume control too far, the volume may not only increase more than the amount of turn, but whistles and various other distortions occur in the sound. These are examples of a nonlinear response.
1.2.3 State Space and the Faithful Model Assumption

Much of the modeling of physical systems takes place in what is called state space, an abstract mathematical space of points where each point represents a possible state of the system. An instantaneous state is taken to be characterized by the instantaneous values of the variables considered crucial for a complete description of the state. One advantage of working in state space is that it often allows us to study the geometric properties of the trajectories of the target system without knowing the exact solutions to the dynamical equations. When the state of the system is fully characterized by position and momentum variables, the resulting space is often called phase space. A model can be studied in state space by following its trajectory from the initial state to some chosen final state. The evolution equations govern the path—the history of state transitions—of the system in state space.

However, note that some crucial assumptions are being made here. We are assuming, for example, that a state of a system is characterized by the values of the crucial variables and that a physical state corresponds via these values to a point in state space. These assumptions allow us to develop mathematical models for the evolution of these points in state space and such models are taken to represent (perhaps through an isomorphism or some more complicated relation) the physical systems of interest. In other words, we assume that our mathematical models are faithful representations of physical systems and that the state spaces employed faithfully represent the space of actual possibilities of target systems. This package of assumptions is known as the faithful model assumption (e.g., Bishop 2005, 2006), and, in its idealized limit—the perfect model scenario—it can license the (perhaps sloppy) slide between model talk and system talk (i.e., whatever is true of the model is also true of the target system and vice versa). In the context of nonlinear models, faithfulness appears to be inadequate (§3).
1.2.4 Qualitative Definitions of Chaos

The question of defining chaos is basically the question what makes a dynamical system like (1) chaotic rather than nonchaotic. Stephen Kellert defines chaos theory as “the qualitative study of unstable aperiodic behavior in deterministic nonlinear dynamical systems” (1993, p. 2). This definition restricts chaos to being a property of nonlinear dynamical systems (although in his (1993), Kellert is sometimes ambiguous as to whether chaos is only a behavior of mathematical models or of actual real-world systems). That is, chaos is chiefly a property of particular types of mathematical models. Furthermore, Kellert's definition picks out two key features that are simultaneously present: instability and aperiodicity. Unstable systems are those exhibiting SDIC. Aperiodic behavior means that the system variables never repeat values in any regular fashion. I take it that the “theory” part of his definition has much to do with the “qualitative study” of such systems, so we'll leave that part for §2. Chaos, then, appears to be unstable aperiodic behavior in nonlinear dynamical systems.

This definition is both qualitative and restrictive. It is qualitative in that there are no mathematically precise criteria given for the unstable and aperiodic nature of the behavior in question, although there are some ways of characterizing these aspects (the notions of dynamical system and nonlinearity have precise mathematical meanings). Of course can one add mathematically precise definitions of instability and aperiodicity, but this precision may not actually lead to useful improvements in the definition (see below).

The definition is restrictive in that it limits chaos to be a property of mathematical models, so the import for real physical systems becomes tenuous. At this point we must invoke the faithful model assumption—namely, that our mathematical models and their state spaces have a close correspondence to target systems and their possible behaviors—to forge a link between this definition and chaos in real systems. Immediately we face two related questions here:

1. How faithful are our models? How strong is the correspondence with target systems? This relates to issues in realism and explanation (§5) as well as confirmation (§3).
2. Do features of our mathematical analyses, e.g., characterizations of instability, turn out to be oversimplified or problematic, such that their application to physical systems may not be useful?

Furthermore, Kellert's definition may also be too broad to pick out only chaotic behaviors. For instance, take the iterative map xn + 1 = cxn. This map obviously exhibits only orbits that are unstable and aperiodic. For instance, choosing the values c = 1.1 and x0 = .5, successive iterations will continue to increase and never return near the original value of x0. So Kellert's definition would classify this map as chaotic, but the map does not have any other properties qualifying it as chaotic. This suggests Kellert's definition of chaos would pick out a much broader set of behaviors than what is normally accepted as chaotic.

Part of Robert Batterman's (1993) discusses problematic definitions of chaos, namely, those that focus on notions of unpredictability. This certainly is neither necessary nor sufficient to distinguish chaos from sheer random behavior. Batterman does not actually specify an alternative definition of chaos. He suggests that exponential instability—the exponential divergence of two trajectories issuing forth from neighboring initial conditions—is a necessary condition, but leaves it open as to whether it is sufficient.

cool story, fag

No, it gets better. Did you read 1.2.2? Put it all into perspective for me.