Skip to the note Memos

A Note On Infinitesimal Generators and Lyapunov-Foster Arguments

Yanni Bills

For the sake of the broader readers' understanding, this pedagogical piece attempts to over-explain wherever possible. Thus, for the more savvy reader, this writeup may appear a bit too exhaustive in some places and a bit too superficial in others; nonetheless, that is the nature of this document.

Motivation

Human nature possesses this inexplicable propensity towards controlling and parameterizing the universe to better suit their frameworks used to explain it. Statistical mechanics concerns itself with canonical ensembles, probability theory with well-defined probability spaces, and classical mechanics with Hamiltonian phase space. These are all, when taken at face value, attempts by mankind to play as God. But a less cavalier take obviously goes beyond this notion, suggesting that it is an innately human desire to yearn for that which is not tangible. Humanity has shown to have a great sense of universality and otherism, so it comes as no surprise that this tendency appears in the hard sciences.

The above constitutes a very flowery way to say that we like studying systems in a way that is tractable and non-explosive because it provides us with information that we can realistically use and build upon. In dynamical systems theory, we like to see that a system's trajectories upon evolving for a sufficiently long time have their probability laws converge to a stationary distribution on some topologically mixing attractor or limit cycle (some notion of ergodicity, if you will). This is the essence of convergence to a stationary measure in dynamical phase space; to prove the existence of such a measure is not a trivial endeavor. But akin to the flowery motivation, in a lower-dimensional fashion, we will attempt to explain something that seems out of reach using methods in the subset of broader human mathematics.

Introduction to Basic Concepts

For the sake of exploring the concepts which follow, we start by defining an arbitrary diffusion process whose evolution is governed by

\[d\textbf{X}_t = f(\textbf{X}_t,t) dt +g(\textbf{X}_t,t)dW_t, \tag{1}\]

for an \(n\)-dimensional diffusion process \(\textbf{X}_t\in \mathbb{R}^n\), an \(m\)-dimensional Wiener process \(\textbf{W}_t\in\mathbb{R}^m\), a drift vector \(f(\textbf{X}_t,t)\in\mathbb{R}^n\), and a diffusion matrix \(g(\textbf{X}_t,t)\in\mathbb{R}^n\times \mathbb{R}^m\). Here, \(f\) and \(g\) are deterministic functions of time and/or state. To be concrete, and thereby aid the visualization of this structure beyond a euphemistic description with arbitrary vectors and matrices, let us consider the case of a two-dimensional diffusion process governed by

\[\begin{aligned} dx_t &= f_x dt + \sigma_x dW_x(t)+a_x dW_y(t)\\ dy_t &= f_y dt + a_y dW_x(t) + \sigma_y dW_y(t). \end{aligned} \tag{2}\]

Here, \(f_x\) and \(f_y\) are drift functions while \(\sigma_x\) and \(\sigma_y\) are functional components of a diffusion matrix. Namely, the drift vector is effectively a two-dimensional object given by

\[f(\textbf{X}_t,t)=\begin{pmatrix} f_x\\ f_y \end{pmatrix}. \tag{3}\]

Similarly, for functions \(a_x\) and \(a_y\), the diffusion matrix is a 2x2 object given by

\[g(\textbf{X}_t,t)= \begin{pmatrix} \sigma_x & a_x\\ a_y & \sigma_y \end{pmatrix}. \tag{4}\]

One can surely envision that for \(a_x=a_y=0\), the matrix becomes diagonal and there are no cross-injections of stochastic fluctuations between the two stochastic differential equations (SDEs).

It\(\hat{\text{o}}\)'s Lemma

Now that we are, ostensibly, comfortable with the multi-dimensional formulation of a diffusion process, we are hereby ready to state the multi-dimensional formulation of the renowned It\(\hat{\text{o}}\)'s lemma. However, in the name of pedagogy, it would be hue of us to start with the canonical one-dimensional form. To do just that, we opt to derive it firsthand in a loose manner. One can start with the general, one-dimensional dynamics

\[dX_t = f(X_t,t)dt + g(X_t,t) dW_t.\]

Let \(Y_t=F(X_t)\) constitute an invertible map. We then Taylor expand \(F(X+dX)\) about \(X\), giving us

\[\begin{aligned}F(X+dX) &= F(X) + F'(X) dX + \frac{1}{2} F''(X) (dX)^2\\ \Rightarrow \underbrace{F(X+dX)-F(X)}_{=F(X_{t+dt})-F(X_t)=dF(X_t)} &= F'(X)dX+ \frac{1}{2} F''(X) (dX)^2.\end{aligned}\]

So, letting \(dt \downarrow 0\), we get that

\[\begin{aligned}dF(X_t) &= F'(X_t) dX_t + \frac{1}{2} F''(X_t) (dX_t)^2\\ &= fF' dt + gF' dW_t + \frac{1}{2} F'' (f^2 dt^2 + 2fg dtdW_t + g^2 dW_t^2)\\ &= \left( fF' + \frac{1}{2} F'' g^2 \right)dt + gF' dW_t,\end{aligned}\]

as needed.

Let us return, then, to the multi-dimensional process \(\textbf{X}_t\) whose dynamics are given by Eq. (1). For some function \(F(\textbf{X}_t,t)\), It\(\hat{\text{o}}\)'s lemma states that

\[\begin{aligned} dF(\textbf{X}_t,t) &= \partial_t F dt+(\nabla_{\textbf{X}}F)^{\text{T}}d\textbf{X}_t + \frac{1}{2}(d\textbf{X}_t)^{\text{T}}(\mathcal H_{\textbf{X}}F)d\textbf{X}_t\\ &= \left\{ \partial_t F +(\nabla_{\textbf{X}}F)^{\text{T}} f(\textbf{X}_t,t) +\frac{1}{2}\text{Tr}[g(\textbf{X}_t,t)^{\text{T}}(\mathcal H_\textbf{X}F)g(\textbf{X}_t,t)] \right \}dt + (\nabla_{\textbf{X}}F)^{\text{T}}g(\textbf{X}_t,t)d \textbf{W}_t. \end{aligned} \tag{5}\]

Here, \(\nabla_{\textbf{X}}\) is the gradient with respect to \(\textbf{X}\), \( \mathcal H_{\textbf{X}}\) is the Hessian matrix applied to \(F\), and Tr is the trace operator. Of note is that It\(\hat{\text{o}}\)'s lemma effectively describes a chain rule in the context of stochastic calculus, thereby yielding that extra second order stochastic fluctuation term.

Infinitesimal Generators

Recall once more the general It\(\hat{\text{o}}\) diffusion, \(\textbf{X}_t\), whose dynamics are given by Eq. (1). Its covariance matrix is given by

\[a=g(\textbf{X}_t,t) g(\textbf{X}_t,t)^{\text{T}}. \tag{6}\]

We now consider an observable function, \(H(\textbf{X}_t)\in C^2(\mathbb{R}^n)\), where \(C^2(\mathbb{R}^n)\) is the set of twice-differentiable smooth functions with a domain of \(\mathbb{R}^n\). Thus, the infinitesimal generator, \(\mathcal{L}\), applied to \(H\) is given by

\[\mathcal{L} H(\textbf{X}_t) = f(\textbf{X}_t,t) \cdot \nabla_{\textbf{X}} H(\textbf{X}_t) + \frac{1}{2} \text{Tr} \left[ a \nabla_{\textbf{X}}^2 H(\textbf{X}_t) \right]. \tag{7}\]

Evidently the drift contribution, given by \(f\) from Eq. (1) dotted with \(\nabla H\), describes the deterministic rate at which the system dynamics change \(H\). In fact, if \(g(\textbf{X}_t,t)=0\) for all components, then this infinitesimal generator applied to \(H\) is simply the directional derivative of \(H\) along the deterministic flow. The diffusion contribution, given by \(\frac{1}{2} \text{Tr}(a\nabla^2 H)\), encodes the additional effect of stochastic fluctuations on the expected evolution of \(H\). This is why for quadratic observables, \(H\), the infinitesimal generator produces explicit It\(\hat{\text{o}}\) corrections even though Wiener increments have zero mean.

As it pertains to It\(\hat{\text{o}}\)'s lemma

The perceptive reader may recall, and presciently recognize a consequence of, the fact that It\(\hat{\text{o}}\)'s lemma, for a smooth observable \(H(\textbf{X}_t)\), gives

\[dH(\textbf{X}_t) = \nabla_{\textbf{X}} H(\textbf{X}_t) d\textbf{X}_t+ \frac{1}{2} \text{Tr} \left[ a \nabla^2_\textbf{X} H(\textbf{X}_t) \right]dt. \tag{8}\]

This, upon substituting the dynamics of \(\textbf{X}_t\) via the form given in Eq. (1), is equivalently expressed as

\[dH (\textbf{X}_t) = \underbrace{\mathcal{L} H(\textbf{X}_t)}_{\text{drift of observable}}dt + \underbrace{\nabla H(\textbf{X}_t)^{\text{T}}g(\textbf{X}_t,t)}_{\text{stochastic fluctuation}}d\textbf{W}_t. \tag{9}\]

Thus the infinitesimal generator applied to a smooth observable is precisely the coefficient of \(dt\) in It\(\hat{\text{o}}\)'s formula applied to said smooth observable. An immediate result of this is that, by taking expectations of both sides, and relying upon the fact that \(\mathbb{E}[d\textbf{W}_t]=0\), we get

\[\frac{d}{dt} \mathbb{E}[H(\textbf{X}_t)] = \mathbb{E}[\mathcal{L} H(\textbf{X}_t)]. \tag{10}\]

If the diffusion process, \(\textbf{X}_t\), were to be initialized with some a priori stationary distribution \(\pi\), then we would note that a smooth observable of said diffusion process would exhibit a stationary mean with respect to said measure. This yields a useful stationarity criterion wherein one can abuse the result that

\[\mathbb{E}_\pi [\mathcal{L}H]=0, \tag{11}\]

or, equivalently,

\[\int_{\mathbb{R}^n} \mathcal{L} H(\textbf{X}) \pi (d\textbf{X})=0. \tag{12}\]

A further point of intuition, that is absolutely worth noting to the end of a complete understanding, is that the drift term \(f\) from Eq. (1) is axiomatically defined by

\[f(u)=\lim_{h\downarrow 0} \frac{\mathbb{E}[\textbf{X}_{t+h} -\textbf{X}_t|\textbf{X}_t=u]}{h}. \tag{13}\]

Considering once more a smooth observable \(H:\mathbb{R}^n\to \mathbb{R}\), we can then note that the infinitesimal generator applied to \(H\) is equivalently representing

\[\mathcal{L}H(u) = \lim_{h\downarrow 0} \frac{\mathbb{E}[H(\textbf{X}_{t+h}) -H(\textbf{X}_t)|\textbf{X}_t=u]}{h}. \tag{14}\]

A Simple Example

Consider a one-dimensional Ornstein-Uhlenbeck (OU) process

\[dX_t = -X_t dt + \sqrt{2} dW_t \tag{15}\]

and an observable \(H(x)=x^2\). It follows that the infinitesimal generator applied to \(H\) is

\[\mathcal{L} H =2-2x^2. \tag{16}\]

Observe below some graphics plotting the relevant results of this quantitative framework.

The generator in action

For the Ornstein–Uhlenbeck process \(dX_t=-X_t\,dt+\sqrt{2}\,dW_t\), explore how the generator changes the observable \(H(x)=x^2\).

Three-panel OU example: H(x) = x squared; its generator is 2 minus 2x squared; conditional expected H approaches 1 from initial values 0 and 4. Twenty faint sample paths illustrate fluctuations around the exact means.
Interactive reconstruction of the supplied figure. The mean curves follow \(\mathbb{E}[H(X_t)\mid X_0=x_0]=1+(x_0^2-1)e^{-2t}\). The ensemble uses newly simulated, reproducible paths with exact Gaussian OU transitions at Δt = 0.005; the original random draws were not supplied.
Original figure and simulation details Original supplied figure, with observable, generator decomposition, conditional means and twenty sample paths.

The exact transition is \(X_{t+\Delta t}=e^{-\Delta t}X_t+\sqrt{1-e^{-2\Delta t}}\,Z\), with independent standard normal draws \(Z\). Lines between sampled points are visual interpolation. Every ensemble contains ten paths from each of the two initial states; changing the ensemble leaves the exact mean curves unchanged.

Note that the faded orange and blue profiles in subplot (c) represent 10 independent sample trajectories of \(H(X_t)=X_t^2\) initialized at \(X_0=2\) and \(X_0=0\) respectively. We further observe that the respective means, regardless of initialization, converge to that associated with the stationary distribution \(\pi\) of the OU process (which is known to be a Gaussian distribution). This would then imply that \(\frac{d}{dt}\mathbb{E}_\pi[H(X_t)]=0\) and, thus, \(\mathbb{E}_\pi[\mathcal{L} H]=0\).

Foster-Lyapunov Arguments

Deterministic Pedagogy

At the highest level, the Foster-Lyapunov argument is a means of demonstrating that a stochastic dynamical system possesses enough restoring behavior (dissipative effects, if you will) to prevent its probability distribution from escaping to \(\infty\). To the end of explicating this, start by considering a deterministic system

\[\frac{du}{dt} = f(u), \quad u\in\mathbb{R}^n. \tag{17}\]

Suppose we construct a smooth, nonnegative function \(V(u)\) (akin to the observable we have discussed before) that grows as \(||u||\uparrow\infty\). Along trajectories, we have that

\[\frac{d}{dt} V(u(t)) = f(u) \cdot \nabla V(u). \tag{18}\]

If we can establish

\[f(u)\cdot \nabla V(u)<0 \tag{19}\]

outside of some bounded region, then we can claim that trajectories, \(u(t)\), experience a restoring tendency whenever they move sufficiently far from that region. This constitutes the essence of the section in a deterministic case.

Return to Stochasticity

We now consider, once again, the diffusion process \(\textbf{X}_t\) whose dynamics are given by Eq. (1). We note that individual trajectories, \(\textbf{X}_t\), can fluctuate away from a stationary state by virtue of the Brownian forcing \(d\textbf{W}_t\) regardless of what the drift term attempts to do. Thus, in this stochastic world, we opt to examine the instantaneous conditional expected rate of change of an observable \(H(\textbf{X}_t)\), which is given precisely by the infinitesimal generator

\[\mathcal{L} H = f\cdot \nabla H + \frac{1}{2} \text{Tr} [a \nabla^2 H]. \tag{20}\]

The rigorous condition by which we determine whether or not a process \(\textbf{X}_t\) exhibits non-explosive tendencies (doesn't run off to \(\infty\) thereby obviating any chance of developing a stationary distribution) is given by

\[\mathcal{L} H(\textbf{X}_t) \leq K-\lambda H(\textbf{X}_t) \tag{21}\]

for \(K,\lambda >0\) and \(H(\textbf{X}_t)\stackrel{||\textbf{X}_t||\uparrow\infty}{\longrightarrow} \infty\) (we call \(H\) coercive if it satisfies this latter criterion). Eq. (21) is thus the Foster-Lyapunov criterion. This is nonetheless a probabilistic control, whereby an individual trajectory may indeed pursue an arbitrarily large Brownian-forcing-induced excursion all while its average dynamics exhibit a restoring tendency.

Now, recall that It\(\hat{\text{o}}\)'s lemma connects to infinitesimal generators via the dynamics

\[dH = \mathcal{L} H dt+ dM_t, \tag{22}\]

where \(M_t\) is the stochastic-integral term (we just wrapped up everything neatly and conveniently into here for parsimony of notation). We then recall that after taking expectations, we have

\[\frac{d}{dt} \mathbb{E}[H]=\mathbb{E}[\mathcal{L} H], \tag{23}\]

where the Foster-Lyapunov criterion gives us that

\[\frac{d}{dt} \mathbb{E}[H(\textbf{X}_t)]\leq K -\lambda \mathbb{E}[H(\textbf{X}_t)]. \tag{24}\]

Solving this simple differential inequality gives

\[\mathbb{E}[H(\textbf{X}_t)]\leq e^{-\lambda t} H(\textbf{X}_0) + \frac{K}{\lambda} (1-e^{-\lambda t}), \tag{25}\]

which then yields

\[\limsup_{t\uparrow\infty} \mathbb{E}[H(\textbf{X}_t)]\leq \frac{K}{\lambda}. \tag{26}\]

Ergo, we are given an upper bound on moments of sufficiently well-behaved observables \(H\) that satisfy the Foster-Lyapunov condition. For example, if \(H\) is positive and quadratic, then we have control over the second moments of the system.

This is one of mankind's many ways in which it enacts control over the structure and universe about it.