Sensitive Dependence on Initial Conditions

the following problem illustrates how the smallest changes to a problem can have large consequences. as explained at the end of the post, this problem is a little artificial, but it illustrates difficulties that come up in realistic problems.

suppose you have a simple differential equation,

where y ( t ) gives displacement in meters at a time measured in seconds. assume initial conditions y (0) = 1 and y ' (0) = -1.

then the unique solution is y ( t ) = exp(- t ). the solution is a decreasing, positive solution. but a numerical solution to the equation might turn around and start increasing, or it might go negative.

suppose the initial conditions are y (0) = 1 as before but now y ' (0) = -1 ± ε. you could think of the ε as a tiny measurement error in the initial velocity, or a limitation in representing the initial velocity as a number in a computer.

the following graph shows the solutions corresponding to ε = 10 ^-6 , 0, and -10 ^-6 .

the exact solution is

if the initial velocity is exactly -1, then ε = 0 and we only have the solution exp(- t ). but if the initial velocity is a little more than -1, there is an exponentially increasing component of the solution that will eventually overtake the exponentially decreasing component. and if the initial velocity is a little less than -1, there is a negative component increasing exponentially in magnitude. eventually it will overtake the positive component and cause the solution to become negative.

the solution starts to misbehave (i.e. change direction or go negative) at

if ε is small, 2/ε is much larger than 1, and so the final 1 above makes little difference. so we could say the solution goes bad at approximately

whether the error is positive or negative. if ε is ±0.000001, for example, the solution goes bad around t = 7.25 seconds. (the term we dropped only effects the 6th decimal place.)

on a small time scale, less than 7.25 seconds, we would not notice the effect of the (initially) small exponentially growing component. but eventually it matters, a lot. we can delay the point where the solution goes bad by controlling the initial velocity more carefully, but this doesn’t buy us much time. if we make ε ten times smaller, we postpone the time when the solution goes bad to 8.41, only a little over a second later.

[looking at the plot, the solution does not obviously go wrong at t = 7.25. but it does go qualitatively wrong at that point: a solution that should be positive and decreasing becomes either negative or increasing.]

trying to extend the quality of the solution by making ε smaller is a fool’s errand. every order of magnitude decrease in ε only prolongs the inevitable by an extra second or so.

this is a somewhat artificial example. the equation and initial conditions were selected to make the sensitive dependence obvious. the sensitive dependence is itself sensitive: small changes to the problem, such as adding a small first derivative term, make it better behaved.

however, sensitive dependence is a practical phenomena. numerical methods can, for example, pick up an initially small component of an increasing solution while trying to compute a decreasing solution. sometimes the numerical sensitivity is a reflection of a sensitive physical system. but if the physical system is stable and the numerical solution is not, the problem may not be numerical. the problem may be a bad model. the numerical difficulties may be trying to tell you something, in which case increasing the accuracy of the numerical method may hide a more basic problem.