Category Archives: Dynamical Systems

Python in my dynamical systems class

I have been using Mathematica in my dynamical systems class for a few years. I don’t have a systematic curriculum related to it, though, and need to develop clearer computational learning goals, as well as a pathway for students to develop computational skills.

Ideally, by the end of the semester, students would be able to do an analysis of a one-parameter dynamical system with the aid of computational tools. They would find fixed points, identify stability, create phase portraits and bifurcation diagrams, and perhaps create stability diagrams. I would expect them to be able to identify global bifurcations, as well. For limit cycles, I need to make a decision on my expectations. I suppose I would like for students to be able to create a curve of initial conditions, use “Events” in Mathematica of Matlab integration, and identify the stability / existence of a limit cycle in a 2d system.

At the moment I’ve been relying on Mathematica and some students have chosen to use Matlab. I would like to move towards Python. Currently the dynamical systems course is the only place students work with Mathematica, while Python is an option across a range of courses.

This means I need to learn how to set up Python for a class. At the moment I’m taking a look at Koehler and Kim, 2018 for some guidance on this. They go in the direction of Jupyter Notebooks so I will explore that for now.

1) Install the Anaconda application on my Mac.
2) Open the Anaconda Navigator: it has seven options when I first open it (Jupyter lab, Jupyter notebook, Qt console, Spyder, Glueviz, Orange 3, RStudio). The first four are already installed and I have the option to Launch them. For the other three, I have the option to install them. I’ll launch Jupyter notebook.

Koehler and Kim 2018: Interactive Classrooms with Jupyter and Python. The Mathematics Teacher. Vol. 111, No. 4 (January/February 2018), pp. 304-308 (5 pages)

Meiss: Differential Dynamical Systems (chaos)

I am reading James Meiss’ text Differential Dynamical Systems (SIAM).  I am specifically interested in how he tells the story of chaos.

In the Preface, he mentions the following: That  Chapter 5 focuses on invariant manifolds:

  • stable and unstable sets
  • heteroclinic orbits
  • stable manifolds
  • local stable manifold theorem
  • global stable manifolds
  • center manifolds

That the “stable and unstable manifolds, proved to exist for a hyperbolic saddle, give rise to one prominent mechanism for chaos — heteroclinic intersection”.

That Chapter 7 is background for understanding chaos (“Lyapunov exponents, transitivity, fractals, etc”):

  • chaos
  • Lyapunov exponents / definition / properties
  • strange attractors / Hausdorff dimension / strange, nonchaotic attractors

And that in Chapter 8 he’ll discuss Melnikov’s method (onset of chaos): sections 8.12 and 8.13.

He notes that he doesn’t discuss discrete dynamics (maps).

 

After the preface, doing a word search for “chaos” or “chaotic”:

Chaos next comes up in the examples in section 1.4: Meiss introduces an example called the “ABC flow” from Arnold 1965.  He mentions this is a “prototype chaotic system” and introduces the idea that “nearby trajectories will often diverge exponentially quickly in time”.  Then he defines the Lyapunov exponent.

Section 1.7 is about Quadratic ODEs: the simplest chaotic systems, after the Lorenz model is introduced in section 1.6.  The introduction of the Lorenz model includes an image of the setup, a mention of convective rolls, the idea of the Galerkin truncation, etc.  So he introduces this system by deriving the ODEs for it.

In section 1.7 he says “informally, chaos corresponds to aperiodic motion that exhibits ‘sensitive dependence on initial conditions'”.  He’ll provide a formal definition in chapter 7.  He mentions that 3-dimensional systems “are the lowest dimensional autonomous ODEs that can exhibit chaos”.  There is a chart of Sprott’s quadratic chaotic differential equations (the simplest quadratic systems with chaos).

In section 4.1 Definitions, “orbits can be quasiperiodic, aperiodic, or chaotic”.  When he introduces orbits, he introduces the idea of a periodic orbit as well as other options.

Meiss returns to chaos in section 5.2 Heteroclinic orbits.  (See Diacu and Holmes 1996 for the story of Poincare, his mistake, and its correction).  He defines a heteroclinic orbit as an orbit that is backward asymptotic to one invariant set and forward asymptotic to a different one.  The homoclinic orbit (doubly asymptotic) is then a special case that is forward and backward asymptotic to the same invariant set.

In a 2d system, if a branch of W^u intersects a branch of W^s then the branches coincide.  Orbits that separate phase space are called separatrices: “they separate phase space into regions that cannot communicate”.  In section 8.13, we’ll see that higher-dimensional systems are different from 2d ones, and that this doesn’t have to happen!  Meiss also defines “saddle connection” and mentions that Hamiltonian systems in the plane often have separatrices.

Chaos comes up again in section 5.5 Global stable manifolds.  The global set comes from flowing the local set backward in time.  For finite time, it will be smooth.  To think about its structure in general, Meiss introduces the idea of an “embedding”.  He also defines an “immersion” and notes that “an immersion is locally a smooth surface”.  Note that immersions can cross themselves.  The topologist’s sine curve is an example that “has infinitely many oscillations and accumulates upon the interval [-1,1] on the y-axis”.  “We will see later that the global stable manifold can have this accumulation problem: indeed, this is one of the indications of chaos”.

Next, in section 6.6, when the Poincaré-Bendixson theorem is introduced, it is introduced as a statement that “There is no chaos in two dimensions”.  So Meiss is building intuition for the idea of chaos from the very first day in the course, and is distinguishing between it and what happens in 2d, even as he introduces 2d.

Chapter 7 focuses on chaotic dynamics, so is where the story will be more fully built out.  The chapter begins with quotes from Poincaré and Lorenz and an informal definition.  To formalize it will require defining “unpredictable” and “sensitive dependence”.  This chapter occurs before the chapter on bifurcations.

Two very simple linear examples with sensitive dependence are given to build some intuition for the stretching between nearby trajectories that happens for some initial conditions in a system with sensitive dependence, and to show that sensitive dependence alone is not “chaotic”.

Aperiodicity or “wanders everywhere” on an invariant set is the next idea introduced, leading to a definition of “transitive”.  Then “a flow is chaotic on a compact invariant set X if the flow is transitive and exhibits sensitive dependence on X”.  This gives us an idea of mixing and unpredictability.

For a lot of systems, their trajectories look chaotic “when solved numerically”.  Chaos was verified in the Lorenz system for r = 28 in Tucker 2002.

Note: I also should read the text for references to the Lorenz system, because that system is used as an example.

 

Dynamical Systems: Strogatz Chapter 5

This chapter is mainly review of topics from prerequisite courses.  Steve does introduce the (Delta, tau)-plane for classifying fixed points of linear systems.  This chapter is a return to linear systems.

There isn’t a “summary” section in between Chapter 4 and Chapter 5.  That is probably a worthwhile spot to review the differences between the behaviors we see in linear systems vs in nonlinear systems in 1d.  It isn’t noted in chapter 4, but I suppose the only possible linear system for flow on the circle is actually the constant velocity one (because of the periodicity requirement for the vector field).  What are the phenomena that we can encode via a nonlinear model that we can’t get to via a linear model?

  • Section 5.0: Introduction
    • We are back to linear systems, now in 2d.  There is a bit more to this than there was in 1d.  We’ll use this info to classify fixed points in nonlinear systems.  Is it possible to build that out a bit more in 1d?  (To think of classification in 1d as seeing which type of linear fixed point our fixed point is similar to?)  I wonder if building out an analogy there would make the distinction between linear and nonlinear systems more intuitive.
  • Section 5.1: Definitions and examples
    • Students probably need to be reminded how to translate a higher order linear differential equation into a linear system (see Example 5.1.1).
    • Distinguishing hyperbolic from non-hyperbolic fixed points could be helpful again here.
    • Some students confuse “saddle point” and “saddle-node”.
  • Section 5.2: Classification of linear systems
    • The role of eigenvalues and eigenvectors in solutions is something some students remember and others find confusing.  Same with complex eigenvalues.
    • Classifying fixed points both using the (Delta, tau)-plane, and providing the eigenvalue classification information in the (Real, Imaginary)-plane would be a good idea because one can memorize the Delta-tau plane without remembering where it came from.
  • Section 5.3: Love affairs
    • This example actually gets used a lot (so may have been seen in other courses).  For that reason, I skip it.

Dynamical Systems: Strogatz Chapter 4

This chapter is not included in Steve’s youtube videos.

  • Section 4.0: Introduction
    • The connection between putting the vector field on a circle and oscillation is not obvious.  Showing time series x(t) or y(t) for a uniform oscillator might help (the time series figures in the text have to do with bottlenecks).
  • Section 4.1: Examples and definitions
    • The mechanics of checking whether a vector field is well-defined on the circle are fine.  I think problems arise when this is approached via intuition instead of calculation.
  • Section 4.2: Uniform oscillator
    • The idea of a phase angle, and what it represents isn’t obvious.
    • We are sometimes making a model of a phase angle and sometimes making a model of a phase difference.  These two contexts lead to different interpretations of the phase portrait, so being explicit about which one we’re talking about is important.
  • Section 4.3: Nonuniform oscillator
    • That nonuniform oscillators have a number of applications is stated.  Intuition for the functional form isn’t provided though.
    • The calculation to find the period of oscillation, and how it changes as the bifurcation is approached, isn’t so easy to follow.  This type of period calculation comes back in the van der Pol.
  • Section 4.4: Overdamped pendulum
    • An overdamped pendulum isn’t so intuitive (because we tend to picture the swinging case), so I’m not sure about this example.
  • Section 4.5: Fireflies
    • Firefly phase locking is a fun example!  In this model, the fireflies reach a fixed phase difference (they phase-lock), but they don’t flash in unison.  That is a bit unintuitive: since they are not in unison, students sometimes struggle with the idea that they can reach a non-unison steady state (the math works out cleanly, but translating it to the model can be hard).
    • It may be worth looking up the Ermentrout 1991 paper about a species that shifts its frequency.
  • Section 4.6: Superconducting Josephson junctions
    • I skip this section.  Other possible applications: jet lag, rat brain grid cells, ??

Dynamical Systems: Strogatz Chapter 3

 

  • Section 3.0: Introduction
    • I need to help students distinguish between parameters and variables.
    • The beam bending example is ok, but the intuition isn’t so clear.  If I back up on the load does the beam straighten (is this a supercritical pitchfork)?
    • It would be nice to introduce an intuitive example of a saddle-node bifurcation.
  • Section 3.1: Saddle-node bifurcation
    • Example 3.1.1 is algebraic, which is great.  Example 3.1.2 is geometric, which is also great.  For each of those examples, adding procedural instructions for creating an associated bifurcation diagram would be helpful.
    • The explanation of normal forms, including the Taylor expansion and Figure 3.1.7 (where f(x) has a local minimum and is being shifted up and down) is clear in the text.  It comes across less clearly in Steve’s youtube videos.
    • Figure 3.1.4 is the only example of a saddle-node bifurcation diagram.  Because it is associated with the normal form it might give the impression that the bifurcation happens at (0,0) and that saddle-node bifurcations always require a perfect parabola.  Adding more examples of bifurcation diagrams would be helpful.
  • Section 3.2: Transcritical bifurcation
    • Instructions / examples of making the bifurcation diagram are also missing here.
    • A “bifurcation curve” in parameter space is an idea that is introduced here.  The idea of parameter space can be confusing (we have phase space, parameter space, and the mixed space that is used for bifurcation diagrams).  It is probably worth introducing this example and the idea of a stability diagram explicitly at this point.
  • Section 3.3: Laser threshold
    • I skip this section.  I should find a different application example of a transcritical bifurcation to replace it with.
  • Section 3.4: Pitchfork bifurcation
    • In the text, the idea of an equation being invariant under a change of coordinates is introduced.  We should do the coordinate replacement in class… (x to -x).  It would be worth building more intuition around the idea of “symmetry”
    • I should assign 2.4.9 on critical slowing down.  I think I assign a modified version that isn’t very helpful.  The idea that solutions decay more slowly than exponential is hard to convey.
    • The geometry associated with a pitchfork bifurcation doesn’t come across very well (that f(x) = g(x) – h(x) and the bifurcation happens when g(x) and h(x) become tangent).
    • There are instructions for plotting a bifurcation diagram here.  The trick to find r in terms of x but then plot in rx-space is introduced.
    • Steve’s youtube videos skip the subcritical pitchfork.  I have a video introducing it, but it could probably be better.
    • The idea of slowly varying a parameter is introduced here.  When varying a parameter, the fixed point will change.  That the fixed point is chained to the parameter is a point of confusion for some students (they sometimes think of the parameter value responding to the fixed point value).  Explicitly defining hysteresis and talking about jumps would be a good thing to do here.
  • Section 3.5: Overdamped bead on a rotating hoop
    • I skip this section and replace it with an introduction to dimension and nondimensionalization.  Understanding this example requires being able to follow a nondimensionalization argument, as well the discussion about neglecting a term.
  • Section 3.6: Imperfect bifurcations and catastrophes
    • Some years I have explicitly introduced the imperfect bifurcation content.  Other times I have skipped it.
    • Providing a reminder of the definition of a bifurcation curve and the distinction between parameter space and other coordinate planes that we use is important here.
    • The definition of a cusp point and the term codimension-2 bifurcation both appear here.
    • Stability diagrams, parameter space, etc are all introduced better in the text than they are in Steve’s youtube videos.
    • The catastrophe example of a bead on a tilted wire seems fun to play with: could we work with it in simulation/animation or do I need a physical version?
  • Section 3.7: Insect outbreak
    • Separation of time scales comes up here and is worth emphasizing.
    • This is a nice modeling example.
    • Making sure to present the terms cusp, bifurcation curve, stability diagram, and catastrophe in advance of this example might help make it easier to follow.
    • Building more intuition for thinking about bifurcations via the intersection of two curves, f(x) = g(x) – h(x), could also help.
    • The idea of bistability comes up in section 3.4 but is introduced here.

Dynamical Systems: Strogatz Chapter 2

Following this text, students study 1d, then 2d, then 3d flows.  In 1d, we find stability, construct phase portraits, and in chapter 3, make bifurcation diagrams.  We loop back to these topics with more complexity in 2d.  This creates natural “spacing”.

A few notes on spacing:

Spacing improves induction/generalization from examples (but learners perceive otherwise): https://journals.sagepub.com/doi/abs/10.1111/j.1467-9280.2008.02127.x

The gap between perception by a learner of what is effective for them, and the actual performance of learners, is reasonably well documented for retrieval practice.  I wasn’t aware that it also occurs with spacing.
“Across experiments, spacing was more effective than massing for 90% of the participants, yet after the first study session, 72% of the participants believed that massing had been more effective than spacing”.  https://doi.org/10.1002/acp.1537 (From Kornell. “Optimising learning using flashcards: Spacing is more effective than cramming”, Applied Cognitive Psychology 2009)

It is also worth noting that spacing isn’t a panacea if info needs to be retained for a long time.  In a study where it was a long time between learning and testing (about a year), subjects in one study retained about 20% (even with spacing in their learning):
https://www-jstor-org.ezp-prod1.hul.harvard.edu/stable/pdf/40064895.pdf?refreqid=excelsior%3A3cf8dd6aa66f4bf0ab7b6ba6428a14e0

Back to my notes on chapter 2:

  • Section 2.0: Introduction
  • Section 2.1: A geometric way of thinking
    • The idea of interpreting the differential equation via a vector field does not come through sufficiently.  I think more care in distinguishing the vector field itself from the phase portrait (phase line) would help.
    • Sketching the qualitative shape of solutions occurs here.  I’ll call those plots “time series” plots, so that we have a way to refer to them.
  • Section 2.2: Fixed points and stability
    • When “phase portrait” is defined, the f(x) plot is present (but it doesn’t need to be: the phase portrait is just what is happening along the x-axis).
    • Writing f(x) = g(x) – h(x) and comparing g(x) and h(x) to construct the phase portrait is a method introduced with a single example.  Writing out the procedural steps for the general procedure could be helpful.
  • Section 2.3: Population growth
    • Figure 2.3.1 doesn’t make it into the lecture videos and is great for explaining the logistic model (that we’re just choosing a simple way to have a carrying capacity).
    • It would be nice to find some of the data for these population models.
    • The “per capita” growth rate is something students have found confusing some semesters.
  • Section 2.4: Linear stability analysis
    • This section is the heavy lift in this chapter…  Adding a visual of the small perturbation may help.  I’m not sure how intuitive the idea of a small perturbation is…
    • The big-O notation needs to be introduced.  When we talk about “higher order terms”, students need to be reminded about the meaning of the word “order” in this context.  In addition, that O(eta^2) encompasses all the higher order terms should be made explicit (https://en.wikipedia.org/wiki/Big_O_notation#Infinitesimal_asymptotics).
    • The connection to exponential growth and decay is relying on students having the solution to x’ = a x very accessible in their memories.  Without this being second nature, a lot of the intuition in this section is lost.
    • Needed background is the definition of a separable diff eq and then how to solve it (this could be introduced in the population growth section).
    • The f ‘ (x*) = 0 case always leads to a number of questions.  I think introducing the terms “hyperbolic” and “nonhyperbolic” would actually help.
    • Most of my students mix up f ‘ (x) and d^2 x/ dt^2.  I need to think about how to avoid that confusion…
    • I’d like to emphasize that we can solve the linearized system.  We can even use it to get the timescale of decay and to sketch solutions near the equilibrium solution.  Is it worth using this to piece together time-series plots of trajectories?  The distinction between linear and nonlinear systems doesn’t currently come through very well.
    • I’d like to compare the solution to the linearized system for small perturbations to numerical approximations via RK4 / Mathematica.
  • Section 2.5: Existence and uniqueness
    • I should provide some intuition for what we mean by the word “smooth” and the term “smooth enough”.
  • Section 2.6: Impossibility of oscillations
    • Reintroducing the definition of “monotonic” could be helpful here.
    • The analogy to the “over-damped” limit is probably not illuminating for students without physics/engineering interests/background.
  • Section 2.7: Potentials
  • Section 2.8: Solving equations on the computer
    • Also not in Steve’s youtube videos.  I haven’t been introducing it explicitly, but probably should.  It converts a flow to a map…
    • They should know about slope fields from Math 1b, so I could remind them and anchor on that.
    • This is a good place to introduce computers.
      • Find fixed points symbolically (Solve) and with root finding (FindRoot).
      • Take the derivative symbolically (and perhaps numerically?).
      • Create time series plots.

2.0 – 2.6 is the prep for a single class meeting.  In the following class, the goals would be to gain procedural fluency with

  • finding fixed points (either given a diff eq or a plot of f(x))
  • determining their stability from a graph of f(x)
  • determining their stability by finding f ‘ (x*) and interpreting it
  • sketching approximate time series of trajectories
  • making phase portraits on the x-axis

Some extra stuff that would be good too:

  • recognizing and setting up an integral for the solution of separable diff eqs
  • distinguishing between linear and nonlinear equations
  • identifying phenomena that occur in linear vs nonlinear autonomous diff eqs
  • doing the procedural stuff above in Mathematica

 

Dynamical Systems: Math 21b differential equations background

For students who have taken Math 1b, AM/Math 21a, Math 21b, there was 6-7 week of differential equations background (11 classes in Math 1b + 9 classes in 21b).  See my prior post for the Math 1b diff eq content that is relevant to Dynamical Systems.

Student diff eq background from Math 21b:

  • Linear first order systems
    • Interpret a linear system written in vector/matrix notation.
    • Use an eigenbasis and eigenvalues for the matrix of a linear system to construct a general solution.
    • Sketch a phase portrait for a 2d system.
    • Match a 2d system to its phase portrait.
    • Work with the matrix exponential.
    • Define “asymptotically stable” and “equilibrium point”.
    • Use eigenvalues of a linear system (with fixed point at the origin) to determine whether the origin is asymptotically stable.
    • Relate the eigenvalues, determinant, and trace of the matrix to the phase portrait about the origin for a linear system (with fixed point at the origin).
  • Nonlinear first order systems
    • Sketch nullclines, identify equilbrium points, and add vectors of the vector field to regions of the phase plane.
    • Use the Jacobian to identify the behavior of a system near an equilibrium point.
  • Higher order linear constant coefficient homogeneous differential equations
    • Convert from a second order equation to a first order system.
    • Find the characteristic polynomial.
    • Construct a general solution by converting to a matrix equation.
    • Solve higher order homogenous constant coefficient linear differential equations.
    • For a nonhomogeneous equation with a sinusoidal forcing (not at the natural frequency), find the general solution.

Many of these topics overlap with Math 1b, but are presented in matrix form in 21b.

Dynamical systems: Math 1b differential equations background.

I have been using the Strogatz textbook for teaching dynamical systems.  The course has multivariable calculus and linear algebra prerequisites.  Students might take the prerequisite courses different places.  For students who have taken Math 1b, AM/Math 21a, Math 21b, there was 6-7 week of differential equations background (11 classes in Math 1b + 9 classes in 21b).

Student diff eq background from Math 1b:

  • Intro to diff eq
    • Write equations from a description of a model where the rate of change of a variable depends on the variable.
    • Determine whether a provided function is a solution to a differential equation.
    • Identify the order of a differential equation.
    • Find a solution to a linear, autonomous, first order differential equation.
  • Approximate solutions
    • Find equilibrium solutions to a differential equation.
    • Use slope fields to sketch approximate solutions.
    • Use Euler’s method to find approximate solutions.
    • Construct a differential equation that would have a particular slope field / a given solution behavior.
  • Separable differential equations
    • Identify whether a differential equation is separable.
    • Use separation of variables to solve separable differential equations.
  • Mass-spring systems
    • Find the characteristic equation for linear, constant coefficient, second order, homogeneous equations.
    • Learn Euler’s formula.
    • Use the characteristic equation to construct a general solution.
    • Identify whether a differential equation has periodic / oscillatory solutions.
    • Rewrite the second order equation as a system of two first order differential equations.
  • First order systems
    • Sketch a solution to a first order system in the phase plane.
    • Distinguish between competition and predator-prey relationships in interaction equations.
    • Draw nullclines in the phase plane.
    • Do a phase plane analysis: identify all nullclines, find all equilibria, orient each nullcline, draw arrows indicating the flow direction within each subregion of the phase plane.
    • Relate the values of dx/dt and dy/dt to the slope of the solution trajectory at a point.
    • Sketch several representative solution trajectories in a phase plane, constructing a phase portrait.
    • Work with the SIR model.