Nonlinearity vs Linearity

Nonlinear optics, a specific context of nonlinearity, studies how light interacts with materials to produce effects that depend on the light's intensity, making these effects nonlinear in relation to the strength of the underlying fields. This might sound trivial, but it is the key to quantum optics because it is how we can produce interesting quantum states of light. It also introduces necessary complexity within our reservoir systems. Given its importance, it is worthwhile to discuss the general concept of nonlinearity.

non-linearity example — Linearity vs non-linearity

Nonlinearity as a high-level concept

The idea of nonlinearity is a simple but powerful one. The question of whether a mathematical model is linear essentially asks whether the solutions obey the property of superposition. Linear models obey this property and nonlinear models do not. In our case, that means very roughly that the whole is equal to the sum of the parts. As an example of why nonlinearity is important, a completely linear neural network, no matter how large, could be described by simple matrix-vector multiplication. By removing the possibility of such a description, even simple nonlinearity can make a neural network much more powerful.

To see the effects of nonlinearity, we consider solving differential equations. Solutions to linear differential equations obey superposition, while those of nonlinear equations do not. From fundamental calculus, derivatives can be defined in a linear way, only involving the first powers of a variable at different points as seen below

\frac{dx}{dt}=\underset{h \rightarrow 0}{\mathrm{lim}}\frac{x(t+h)-x(t-h)}{2\, h}

so differentiation is a linear operation. The derivative of the sum of two functions is equal to the sum of the derivatives of each individually $\frac{d}{dt}(x+y)=\frac{dx}{dt}+\frac{dy}{dt}$ $\frac{d}{d t} (x + y) = \frac{d x}{d t} + \frac{d y}{d t}$ . This is the superposition property we discussed before. Since higher-order derivatives are defined by repeatedly applying the same definition, arbitrarily high-order derivatives also obey superposition. As a result, any differential equation involving sums of constants, first powers of a variable $x$ $x$ , and arbitrarily high derivatives of $x$ $x$ will be linear. Terms such as $x^2$ $x^{2}$ , $x\frac{dx}{dt}$ $x \frac{d x}{d t}$ or $\left( \frac{dx}{dt}\right)^2$ $(\frac{d x}{d t})^{2}$ are nonlinear. Unlike derivatives, applying any of the operations mentioned in the previous sentence to a sum of two terms does not give the same result as applying it to each individually and summing, for example $(x+y)^2=x^2+y^2+2\, x\, y \neq x^2+y^2$ $(x + y)^{2} = x^{2} + y^{2} + 2 x y \neq = x^{2} + y^{2}$ . The presence of any nonlinear terms makes the entire differential equation nonlinear. It is important to remember that nonlinearity is a mathematical property, not a physical one. It is sometimes useful to refer to physical systems where the most convenient model of the system is nonlinear as “nonlinear” (as is often done with nonlinear optics) and those where nonlinearity can usually be ignored as “linear”. Such terminology, however, is not mathematically rigorous, it is just a convenient shorthand.

Mathematically, linear systems of differential equations tend to be much easier to solve than nonlinear systems. This is somewhat intuitive, given that new solutions can be built by summing up already known solutions. A concrete example here is a technique known as matrix diagonalization where a complete set of eigenvectors, special vectors which are only multiplied by a constant when a matrix is applied, is generated that can be summed to solve how a system behaves from any starting point. An example of this technique can be found in these lecture notes, specifically here. Such techniques do not work for nonlinear systems of equations. Being easily described mathematically is a double-edged sword since it implies that the behavior of a system must be relatively simple. For example, neural networks need some nonlinearity to be suitable for sophisticated learning tasks.

A useful example for understanding the difference can be seen above. On the left, simple waves that represent individual solutions to the equation can be added to form a more complex waveform. On the right we see a nonlinear example, a Lorenz attractor, which arises as a solution to a set of nonlinear differential equations known as the Lorenz equations which arise in fluid dynamics. In contrast to solutions to linear equations, superposition does not work here. In fact, the difference between two very similar starting conditions will grow quickly in time until their behavior is very different. A solution with this property is known that chaotic linear equations can never have chaotic solutions, although nonlinear equations will not always have chaotic solutions.

Linearity is a mathematical concept, not a physical one. The same system can sometimes be described in both a linear and nonlinear way. Note that this is not a paradox since physical models are always approximations of the behavior of the actual physical system. The linearity or nonlinearity of the model depends on which approximations are made. For example, the Schrӧdinger equation, the fundamental equation of quantum mechanics, is linear in quantum amplitudes, the summing of which describes quantum interference. However, the number of possible states described in multi-particle quantum systems tends to grow exponentially with the number of particles, so this linear equation quickly becomes too large to solve. While nonlinear optics can be described in a quantum mechanical (and technically linear) way, the practical way of describing it is often in terms of light particles rather than amplitudes. In nonlinear optics, the light particles do not obey superposition and are therefore not described in a linear way.

Linear and nonlinear optics

Linear optics is a subfield of optics, which focuses on the regime where the dynamics are well approximated as being linear in electromagnetic fields of the light in the system. Recall that it is the model, not the system itself, that is linear or nonlinear. According to a linear mathematical model, increasing the light intensity will only lead to brighter light, which is not always physically realistic. For strong enough fields, linear optics always breaks down as an approximation. As we discuss elsewhere a consequence of quantum field theory is that this breakdown even occurs for light particles in vacuum, and certainly will in air or some other medium. Classically, when we do not take into account the particle nature of the light, linear optical systems are simple to describe mathematically, and the behavior of large systems can be stimulated without much difficulty. Perhaps surprisingly, the behavior of such systems is not easy to solve mathematically when quantum states of light (for example, single light particles) are considered. Such states cannot be produced using linear optics, so in a way, if we consider a system that includes the source of the light, these systems are nonlinear overall. As we have also discussed elsewhere, just weakening a light field (a linear process) cannot produce interesting quantum states of light.

Nonlinear optics, where the light does not obey superposition, is the key to quantum optics, a concept that we discuss in-depth elsewhere. In a nonlinear model of optics, the optical fields do not just add independently. An extreme example of nonlinear behavior is the quantum Zeno blockade, where even a single light particle can change the behavior of another light particle by effectively “measuring” it. The underlying process that leads to the blockade is a process known as sum frequency generation, where two light particles of a different frequency combine to create another particle that has a frequency equal to the sum of the two. Such a process is clearly nonlinear in the fields of the light. Nothing happens unless both frequencies are present, in which case no light at the combined frequency is produced. This nonlinear process can be used for many other tasks, for example, producing a source of single light particles. The specific kind of nonlinear processes underlying this effect and many other important effects are parametric processes, they are important enough to be given their own lesson.

Conclusion

Nonlinearity is a significant mathematical concept which has an important role in many settings. Linear equations, in particular linear differential equations, can be solved by using the superposition property, where a complete set of solutions can be added together to build any other solution. On the other hand, nonlinear equations cannot be solved in this way. When viewed through this lens, adding nonlinearity is a way of adding complexity. Quantum mechanics is technically a linear theory, but this description is only valid on a state space which is often impractically large to be used in calculations. As a result, it is often more practical to work with more tractable nonlinear descriptions of these systems in terms of individual particles.