The Free Energy Principle

The free energy principle, developed by the neuroscientist Karl Friston, says that any system that keeps existing (a cell, a brain, an organism) must, on average, avoid surprising states. It does that by minimizing variational free energy, a quantity that sets an upper bound on surprise. It can change its model of the world, which is perception and learning, or change the world to fit its model, which is action. Supporters see a unifying theory of life and mind; critics question whether it explains anything testable.

One principle, very big claims

In 2006, Friston and his University College London colleagues James Kilner and Lee Harrison published "A free energy principle for the brain." Their starting point was old: Hermann von Helmholtz's ideas about perception. Recast in modern terms, as the problem of inferring the causes of sensory input, they argued, those ideas yield a model of perceptual inference and learning that "can explain a remarkable range of neurobiological facts" (Friston, Kilner & Harrison, 2006).

Then they went further. Perception, the paper said, is just one aspect of what any system does if it conforms to a free energy principle. Such a system can lower its free energy in two ways: by changing the distribution it encodes, or by changing how it samples the environment. The first is perception. The second is action (Friston et al., 2006).

By 2010 Friston was asking, in Nature Reviews Neuroscience, whether this was "a unified brain theory" (Friston, 2010). Since then the claims have grown. Friston has claimed it as the principle of all biological reaction, applying to mental disorders as well as to artificial intelligence (Wikipedia). Few ideas in neuroscience have been praised so highly or argued over so hard. Both reactions make more sense once you see what the principle actually says.

What does "surprise" mean here?

Start with the problem every living thing has. You are an improbable arrangement of matter. Your body temperature, your blood chemistry and the concentrations inside your cells sit in narrow ranges. Drift too far outside them and you stop being you. Staying alive means staying in a small set of expected states.

But you never observe your state directly. You sense the world through receptors that respond to light, molecules and pressure. Everything you know arrives through that narrow, noisy channel.

So the FEP frames the organism's job in terms of sensations. In Friston's technical vocabulary, "surprise" is the negative log-probability of a sensory outcome under the organism's model of the world (Friston, 2010). An expected sensation has low surprise. A wildly unexpected one has high surprise. A fish that keeps sensing "dry" is in trouble. The claim is that agents who persist must keep their long-term average surprise low (Friston, 2010).

This is a precise, statistical sense of the word. It isn't the feeling of being startled, though the two often line up.

What is variational free energy?

Here is the catch. To calculate surprise, you'd need the true probability of your sensations across every possible state of the world. No organism has that. It only has its own model.

The trick, borrowed from statistics, is to minimize something you can compute that is never smaller than surprise. That quantity is variational free energy. In the 2006 paper it is written as:

F = −ln p(y) + D(q ‖ p(ϑ | y))

The first term is surprise: how improbable the sensory data y are. The second is a Kullback-Leibler divergence, a measure of how far the organism's working guess about hidden causes (q) is from the ideal, fully Bayesian answer, p(ϑ | y) (Friston et al., 2006). A divergence can't be negative, so F is always at least as large as surprise. Push F down and you push down a ceiling on surprise, while making your guess about the causes better at the same time.

That's the mathematics behind the slogan. It is also the method known in machine learning as variational Bayes, which is where the name comes from: Friston et al. note that the technique borrows its form from statistical thermodynamics (2006).

Which raises the obvious question. This "free energy" is not the Gibbs free energy of chemistry, the energy available to do work in a reaction. It is a quantity about probabilities and beliefs, measured in units of information. The two share a mathematical structure and a name, which is the source of endless confusion. The principle is about the first kind.

Two ways to reduce surprise: perception and action

If your sensations don't match what your model predicts, you have two options.

Change your mind. Update your beliefs about what's out there. This is perception and, over longer timescales, learning. Friston argued that under some standard simplifying assumptions, the brain's best version of this looks like predictive coding: higher levels of a cortical hierarchy predict the activity of lower levels, and what flows upward is mainly the error, the part of the signal that wasn't predicted (Friston, 2010). The idea that the brain is fundamentally a prediction machine has its own long history; the site's essay on the Bayesian brain traces it.

Change the world. Act so that what you sense comes to match what you predicted. Feeling cold, you can revise your belief that you are warm, or you can put on a coat. If your model of yourself includes being warm, the coat brings your sensations back into line. This second route is called active inference, and it is the boldest part of the framework, because it treats action as a form of inference. The full treatment is in Active Inference: When Perception Becomes Action.

What's distinctive is not either route on its own but the claim that they are the same thing: two ways of reducing one quantity (Friston, 2010). Most theories of the brain treat perception and motor control as separate problems. The FEP treats them as two moves in the same game.

Is the free energy principle the same as predictive processing?

No, though they are close relatives. Predictive processing (or predictive coding) is a family of hypotheses about how brains work: that they constantly generate predictions and pass prediction errors up a hierarchy. You can test it against neural data, and it can turn out wrong. There is a fuller account in the predictive processing essay.

The FEP is broader and more abstract. It is meant to apply to anything that persists, brains included, and predictive coding is one way a brain might implement it. Friston himself draws the line sharply. In a 2018 interview he contrasted the principle with "things like predictive coding and the Bayesian brain hypothesis", which are hypotheses that can be tested and refuted (quoted in Wikipedia). You could reject predictive coding and keep the FEP, or the other way around.

Markov blankets: what makes a "thing"?

For any of this to work, there has to be a line between the system and everything else. The FEP draws it with a Markov blanket.

The term comes from Judea Pearl, who coined it in 1988 for probabilistic networks (Wikipedia). A variable's Markov blanket is the set of other variables that, once you know them, make everything else irrelevant for predicting it. In the FEP, the blanket is split into sensory states, through which the world affects the system, and active states, through which the system affects the world. Internal and external states touch only through the blanket.

A cell membrane is the textbook picture. On one influential reading, blankets nest: "Markov blankets of Markov blankets", running "all the way down to individual cells, all the way up to you and me, and all the way out to include elements of the local environment" (Kirchhoff et al., 2018). That nesting is what lets the principle claim to apply at many scales at once. The full story, including why it is contested, is in Markov Blankets: The Boundaries That Make Things Things.

A principle, not a theory: can it be tested?

This is where most confusion, and most of the hardest criticism, lives.

Friston's position is that the FEP is a mathematical principle, not an empirical hypothesis. "Like Hamilton's principle of stationary action," he said in 2018, "it cannot be falsified" (quoted in Wikipedia). A 2023 paper by Friston and colleagues spells the position out: much like Hamilton's principle of least action, the FEP "is not a falsifiable theory about the way 'things' behave"; it is "a general description of 'things' that are defined in a particular way" (Friston et al., 2023).

The same paper is candid about what follows. Is such a description useful? "In itself, the answer is probably no," the authors write, in the sense that the principle of least action does not tell you how to throw a ball. What it does is give you what you need to simulate a particular throw. In the same way, the FEP becomes testable once you commit to a specific model of a specific system: at that point you have made, in their words, a commitment to a specific and falsifiable theory (Friston et al., 2023).

So the structure is this. The principle is not falsifiable, by design. The models built with it are. Whether that makes the FEP a deep insight or an elaborate redescription depends on how much those models explain that rival models don't. The site keeps a running list of testable predictions drawn from FEP models; the honest summary is that the evidence bears on the models, not the principle.

What do the critics say?

Three lines of criticism are worth taking seriously, and they come from people who have read the mathematics closely.

The mathematics has gaps. In 2021 Martin Biehl, Felix Pollock and Ryota Kanai published a technical critique of the original formulation. They found that the various definitions of "Markov blanket" used in different papers are not equivalent. They showed that crucial steps in the argument are "not generally correct without additional (previously unstated) assumptions", and they proved by counterexample that the original free energy lemma, "when taken at face value, is wrong." Because the Bayesian interpretation hinges on that lemma, they concluded it "is not sufficiently justified." They were careful to add that these problems "do not rule out conclusively that the general ideas behind the free energy principle are worth pursuing" (Biehl, Pollock & Kanai, 2021).

The blankets are doing two jobs. Jelle Bruineberg, Krzysztof Dołęga, Joe Dewhurst and Manuel Baltieri argued in Behavioral and Brain Sciences that the literature confuses two uses of Markov blankets. One is Pearl's original, epistemic use: a tool for inference within a model. The other is a new, metaphysical use, meant to mark the real physical boundary between an agent and its world. They proposed calling the first "Pearl blankets" and the second "Friston blankets", and argued that the metaphysical claims need extra philosophical premises that the success of the mathematics alone can't supply (Bruineberg et al., 2022).

A principle that fits everything may explain nothing. If any persisting system can be described as minimizing free energy, then saying a particular system does so might tell you little about it. Friston's own answer, quoted above, concedes the point for the principle on its own and moves the explanatory weight onto specific models. Critics reply that if the specific models do the explaining, the principle is a framing rather than a discovery. That debate is still open, and the site's critics and controversies essay goes through it in more detail.

None of this shows the FEP is wrong. It shows that the strongest version of the claim, one equation that explains life and mind, is a hypothesis and an ambition, not a result.

The argument here: free energy as coherence

This section is the site's own reading, not part of the principle.

The argument here is that the FEP is best understood as a theory of coherence: of what a system has to do to stay itself over time. A system that keeps its surprise low is one whose model and world stay in step, and that fit between model and world is what this site calls coherence. AToM, the site's framework, treats meaning as coherence sustained over time, and so it reads the FEP as a formal sketch of the conditions under which anything can have a point of view at all.

That is an interpretation. It inherits every open question above, and it would be weakened if the critics are right that Friston blankets don't pick out real boundaries. The synthesis essay at the end of this series (Part 10, below) makes the case in full.

The series

This series has 10 parts. This page is Part 1, the overview.

1. The Free Energy Principle, Explained Simply (this page)
2. Surprise Is the Enemy: Why Living Systems Minimize Free Energy
3. Markov Blankets: The Boundaries That Make Things Things
4. Variational Inference for Humans: The Math Made Intuitive
5. Active Inference: When Perception Becomes Action
6. The Bayesian Brain: Prediction All the Way Down
7. Beyond Biology: FEP as a Theory of Everything That Persists
8. Critics and Controversies: What FEP Gets Wrong (Maybe)
9. FEP Implementations: From Theory to Working Systems
10. Synthesis: The Free Energy Principle and the Geometry of Coherence

Part of the Frontier Science collection. See also Active Inference Applied and Basal Cognition.

Where this leaves us

The free energy principle is a precise mathematical claim wrapped in an enormous ambition. The precise part says that persisting systems can be described as keeping a bound on surprise low, through perception and action. The ambitious part says that this description explains life and mind. The first is mathematics; the second is still being argued, and the arguments are worth following.

The pebble: a principle that cannot be wrong earns its keep only through the models that can.

FAQ

What is the free energy principle in simple terms?

It is the idea that anything that keeps existing must avoid states it doesn't expect to be in. It does that by minimizing variational free energy, a computable stand-in for surprise, either by updating its model of the world or by acting to change what it senses.

Is the free energy principle the same as predictive processing?

No. Predictive processing is a testable hypothesis about how brains work: they predict their inputs and pass errors up a hierarchy. The FEP is a more general principle meant to apply to any persisting system. Predictive coding is one proposed way a brain might implement it.

What does "free energy" mean here? Is it the same as thermodynamic free energy?

No. Variational free energy is an information-theoretic quantity: surprise plus a measure of how far a system's beliefs are from the ideal Bayesian ones. It borrows its mathematical form, and its name, from statistical physics, but it isn't the Gibbs free energy of chemistry.

Is the free energy principle falsifiable?

According to Friston and colleagues, the principle itself is not, any more than Hamilton's principle of least action is. Specific models built with it are falsifiable. Critics argue that this leaves the principle with little explanatory work of its own.

What is active inference?

Active inference is the action half of the FEP: reducing surprise by changing the world so that sensations match predictions, rather than changing the predictions. It treats acting and perceiving as two ways of minimizing the same quantity.

What are the main criticisms of the free energy principle?

Three stand out. Biehl, Pollock and Kanai (2021) found technical gaps, including a lemma that fails as originally stated. Bruineberg and colleagues (2022) argued that Markov blankets are used in two incompatible senses. And many critics worry that a principle that fits every persisting system explains little about any one of them.

Further Reading

• Friston, K., Kilner, J. & Harrison, L. (2006). "A free energy principle for the brain." Journal of Physiology-Paris 100, 70-87. https://doi.org/10.1016/j.jphysparis.2006.10.001
• Friston, K. (2010). "The free-energy principle: a unified brain theory?" Nature Reviews Neuroscience 11, 127-138. https://doi.org/10.1038/nrn2787
• Friston, K. et al. (2023). "The free energy principle made simpler but not too simple." Physics Reports. https://doi.org/10.1016/j.physrep.2023.07.001 (preprint: https://arxiv.org/abs/2201.06387)
• Biehl, M., Pollock, F. A. & Kanai, R. (2021). "A technical critique of some parts of the free energy principle." Entropy 23(3), 293. https://doi.org/10.3390/e23030293
• Bruineberg, J., Dołęga, K., Dewhurst, J. & Baltieri, M. (2022). "The Emperor's New Markov Blankets." Behavioral and Brain Sciences. https://doi.org/10.1017/S0140525X21002351
• Kirchhoff, M., Parr, T., Palacios, E., Friston, K. & Kiverstein, J. (2018). "The Markov blankets of life: autonomy, active inference and the free energy principle." Journal of the Royal Society Interface. https://doi.org/10.1098/rsif.2017.0792
• "Free energy principle." Wikipedia: https://en.wikipedia.org/wiki/Free_energy_principle
• "Markov blanket." Wikipedia: https://en.wikipedia.org/wiki/Markov_blanket