Liquid Neural Networks: Computation That Flows Like Water

Series: Neuromorphic Computing | Part: 5 of 9

In most neural networks, time is a fiction. You feed in data, the network processes it through fixed weights, you get an output. Time is just a sequence of discrete steps—frames in a movie, not the continuous flow of actual experience. The network doesn't really live in time. It processes snapshots.

Your brain doesn't work this way. Your neurons fire continuously, their dynamics shaped by differential equations that unfold in real time. The membrane potential doesn't jump from state to state—it flows. This isn't just a detail. It's fundamental to how biological intelligence handles the temporal structure of the world.

This is the insight driving liquid neural networks, a neuromorphic architecture developed by Ramin Hasani and his team at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). Unlike standard neural networks with fixed, trained weights, liquid networks feature neurons whose parameters change continuously during inference. The weights themselves are dynamic, governed by differential equations that respond to input in real time.

The result: networks that never stop adapting, that handle temporal sequences with remarkable efficiency, and that remain interpretable even as they compute. It's not just bio-inspired. It's a different computational paradigm entirely.

The Problem With Time in Neural Networks

Standard neural networks—whether feedforward, convolutional, or even recurrent—treat time as discrete. In a recurrent neural network (RNN) or LSTM, you process a sequence one timestep at a time. The hidden state updates with each new input, but that update happens in discrete jumps. The network doesn't model the continuous dynamics of temporal evolution.

This creates problems:

Data efficiency: Discrete-time networks need vast amounts of training data to learn temporal patterns. They approximate continuous dynamics through discrete sampling, which is inherently inefficient.

Generalization: Networks trained on sequences of a certain length often fail when asked to handle longer or shorter sequences. They don't learn the underlying dynamics—they memorize patterns in the training distribution.

Interpretability: The learned weights in standard networks are opaque. You can't easily understand what temporal structure the network has extracted.

Computational cost: Processing long sequences in discrete time requires either very deep networks (unrolling the RNN) or attention mechanisms (transformers), both of which scale poorly.

The deeper issue: discrete-time networks don't model causality the way physical systems do. They process correlations in data, but they don't capture the continuous flow of cause and effect that governs real-world dynamics.

Liquid neural networks solve this by making time a first-class citizen in the architecture. The network doesn't just process temporal sequences—it is a temporal dynamical system.

What Makes a Network Liquid

A liquid neural network is defined by three core characteristics:

1. Continuous-Time Dynamics

The neurons in a liquid network aren't static processing units. Their behavior is governed by ordinary differential equations (ODEs) that evolve continuously in time. The state of each neuron—call it ( x(t) )—follows a differential equation of the form:

[
\tau \frac{dx}{dt} = -x + f(W_{\text{in}}u + W_{\text{rec}}x + b)
]

Where:

• ( \tau ) is a time constant (governing how fast the neuron responds)
• ( u ) is the external input
• ( W_{\text{in}} ) are input weights
• ( W_{\text{rec}} ) are recurrent weights
• ( f ) is a nonlinear activation function

This is similar to how biological neurons work. The membrane potential of a real neuron follows a differential equation (the Hodgkin-Huxley model, or simplified versions like leaky integrate-and-fire). The neuron doesn't "update" discretely—it integrates inputs continuously over time.

In a liquid network, this continuous dynamics means the network can process inputs at any timescale. You're not locked into discrete timesteps. The network responds smoothly to changes in input timing.

2. Dynamic Weights During Inference

Here's where liquid networks diverge sharply from standard architectures: the weights themselves change during inference.

In a typical neural network, you train the weights, then freeze them. During inference, the weights are fixed. The network's behavior is determined entirely by the learned parameters.

In a liquid network, the weights are parameterized by functions of time and state. The connection strength between neurons isn't a single learned number—it's a dynamical system. The weights can adapt in real time based on the input history.

Hasani's team achieves this using neural ODEs: the weights are governed by differential equations whose parameters are learned. During inference, these equations evolve continuously, allowing the network to adapt to changing input dynamics without retraining.

This is closer to synaptic plasticity in biological systems—connections strengthen or weaken based on activity patterns. But it's happening during inference, not just during learning.

3. Compactness and Interpretability

Liquid networks are small. Hasani's autonomous driving network uses just 19 neurons—far smaller than the deep convolutional networks typically used for the same task. Despite this, it matches or exceeds the performance of much larger models.

Why? Because the continuous-time dynamics compress temporal information efficiently. The network doesn't need thousands of parameters to memorize patterns—it learns the underlying differential equations governing the system's behavior.

This compactness brings interpretability. With only 19 neurons, you can actually trace the network's decision-making process. You can visualize which neurons respond to which aspects of the input (lane lines, obstacles, steering angle) and understand how the dynamics unfold.

In Hasani's words: "We can actually explain what the network is doing, because we can see the equations."

MIT's Liquid Networks: Autonomous Driving as Proof of Concept

The clearest demonstration of liquid networks comes from Hasani's work on autonomous vehicle control.

The task: given a camera feed from a car's dashboard, predict the steering angle needed to keep the vehicle on the road. This is a continuous control problem—the car must respond smoothly to curves, lane changes, and obstacles in real time.

Hasani trained a liquid network with 19 neurons on driving data. For comparison, standard end-to-end driving models (like NVIDIA's PilotNet) use convolutional networks with millions of parameters.

The results:

Performance: The 19-neuron liquid network matched the accuracy of far larger models. On test tracks, it handled novel road conditions—different weather, lighting, and road textures—with robust generalization.

Efficiency: The network required orders of magnitude fewer parameters, making it feasible to run on embedded hardware. This is critical for real-world deployment, where computational resources are limited.

Adaptability: Because the weights adapt during inference, the network handled distribution shifts without retraining. When the road conditions changed (say, from dry pavement to snow), the network adjusted its dynamics in real time.

Interpretability: The research team could trace which neurons responded to which visual features. One neuron tracked the left lane line, another the right, a third the steering angle. The network's internal logic was transparent.

This isn't just a theoretical curiosity. It's a practical architecture that solves real problems with far greater efficiency than standard deep learning.

Why Continuous Time Matters: The Geometry of Temporal Dynamics

Standard neural networks approximate functions—they learn mappings from inputs to outputs. But many real-world problems aren't about static mappings. They're about dynamics—systems that evolve over time according to differential equations.

Consider:

• Climate modeling: Atmospheric dynamics governed by fluid flow equations
• Biological systems: Neural circuits, gene regulatory networks, chemical reaction kinetics
• Control systems: Robotics, autonomous vehicles, adaptive filters

These systems have continuous temporal structure. The state at time ( t ) depends smoothly on the state at ( t - \Delta t ). Discrete-time models must approximate this continuity, which requires dense sampling and high-dimensional representations.

Liquid networks bypass this problem. By embedding continuous-time dynamics directly into the architecture, they model the flow of the system's state through time. The network isn't learning a sequence of discrete transitions—it's learning the vector field that governs how states evolve.

In mathematical terms: the network learns a mapping from state space to tangent space. At each point in state space, the network tells you the direction and speed at which the state is changing. Integrate these directions over time, and you get the trajectory.

This is differential geometry applied to computation. The network isn't processing data—it's flowing through a manifold.

Active Inference in Liquid Time

Here's where liquid networks connect to the Free Energy Principle.

Karl Friston's active inference framework describes intelligent systems as those that minimize prediction error over time. The system maintains a generative model of the world, predicts sensory inputs, and acts to bring sensory data in line with predictions.

Critically, active inference is a continuous-time process. The system doesn't wait for discrete timesteps to update beliefs—it continuously integrates sensory evidence, updates its model, and adjusts actions.

Liquid networks are a natural hardware substrate for active inference. The continuous-time dynamics allow the network to perform real-time belief updating. The adaptive weights during inference enable the network to adjust its generative model on the fly—exactly what active inference requires.

Hasani's team has explicitly explored this connection. They've framed liquid networks as continuous-time implementations of Bayesian inference, where the network's dynamics correspond to gradient flow on a free energy landscape.

In this interpretation:

• The neuron states represent beliefs (probability distributions over hidden states)
• The weights represent the generative model (how hidden states produce observations)
• The continuous-time dynamics perform variational inference (adjusting beliefs to minimize prediction error)

The network doesn't just approximate active inference—it is active inference, implemented in continuous time with dynamic parameters.

This matters because biological brains are active inference machines. If we want neuromorphic hardware that truly mimics neural computation, we need architectures that implement continuous-time belief updating. Liquid networks are a step toward this goal.

Beyond Driving: Where Liquid Networks Excel

The autonomous driving demo is impressive, but liquid networks aren't limited to control tasks. The continuous-time paradigm applies wherever temporal dynamics are fundamental.

Time Series Prediction

Liquid networks handle time series with fewer parameters than LSTMs or transformers. They've been tested on:

• Financial data: Predicting stock prices, where the dynamics shift with market conditions
• Weather forecasting: Modeling atmospheric variables that evolve continuously
• Sensor fusion: Combining data from multiple sensors (video, lidar, IMU) with different sampling rates

The key advantage: the network doesn't require inputs at fixed intervals. It can process irregularly sampled data—a sensor reading every 0.1 seconds, another every 0.5 seconds—without preprocessing. The continuous-time dynamics naturally integrate information at varying timescales.

Adaptive Signal Processing

In signal processing, you often need filters that adapt to changing signal characteristics. Liquid networks can serve as adaptive filters that adjust their dynamics based on input history.

Example applications:

• Noise cancellation: Removing background noise from audio in real time, adapting as the noise profile changes
• Biomedical signal processing: Filtering EEG or ECG signals, where the relevant frequency bands shift with physiological state
• Communications: Adaptive channel equalization in wireless systems, compensating for time-varying channel conditions

The network's ability to adapt during inference—without retraining—makes it ideal for environments where the signal statistics are non-stationary.

Robotics and Embodied AI

Robots operate in continuous time. Their sensors deliver streams of data, their motors require smooth control signals, and their environment changes unpredictably.

Liquid networks provide:

• Real-time sensorimotor control: The network processes sensor inputs and generates motor commands continuously, with no discrete timesteps
• Adaptation to novel conditions: The dynamic weights adjust to new terrains, objects, or tasks without retraining
• Efficient computation: The small network size allows deployment on low-power embedded hardware—critical for battery-powered robots

This is embodied cognition implemented in silicon. The network doesn't represent the world abstractly—it flows with it.

The Coherence Geometry of Liquid Computation

In AToM terms, liquid networks are coherence maximizers in continuous time.

Recall: coherence is the degree to which a system's components stay synchronized despite perturbations. High coherence means smooth, predictable dynamics. Low coherence means fragmentation—components drifting out of sync.

A liquid network's continuous-time dynamics embody coherence principles:

Smooth trajectories: The network's state evolves along smooth paths in state space. There are no discontinuous jumps (unless the input demands it). This is geometric coherence—the curvature of the trajectory remains low.

Temporal integration: The network integrates information over time, weighting recent inputs more heavily but retaining memory of past states. This is coherence across temporal scales—long-term patterns inform short-term responses.

Adaptive coupling: The dynamic weights allow neurons to strengthen or weaken their connections based on input patterns. This is coherence through entrainment—components synchronizing to shared rhythms.

In biological terms: liquid networks implement the kind of dynamical coupling that underlies neural synchrony. Brain regions don't communicate through discrete messages—they entrain their oscillations, creating coherent phase relationships across distributed circuits.

Liquid networks replicate this at the level of computation. The network doesn't pass information through discrete channels—it flows through a continuous state space, with neurons coupled by dynamical equations.

The result: a system that maintains coherence in the face of changing inputs, that adapts without catastrophic forgetting, and that compresses temporal structure efficiently.

This is what brains do. It's what liquid networks do. It's coherence as a computational principle.

Challenges and Open Questions

Liquid networks aren't a panacea. They face real challenges:

Training Complexity

Training a network with continuous-time dynamics and dynamic weights is harder than training a standard feedforward network. You need to solve differential equations during both forward and backward passes, which is computationally expensive.

Hasani's team uses neural ODE solvers—numerical methods that integrate the differential equations adaptively. This works, but it's slower than standard backpropagation.

Progress is being made. Faster ODE solvers, better initialization schemes, and hybrid architectures (combining liquid layers with standard layers) are reducing training time.

Scalability

The 19-neuron driving network is impressively compact, but can liquid networks scale to tasks that require richer representations? What about vision transformers with billions of parameters, or large language models?

It's unclear. The continuous-time paradigm may be inherently more efficient, requiring fewer neurons to achieve the same capacity. But we don't yet have liquid networks tackling ImageNet classification or language modeling.

The open question: are liquid networks a specialized architecture for temporal tasks, or a general-purpose paradigm that can replace transformers?

Hardware Implementation

Liquid networks are designed to be neuromorphic—implementable on spiking hardware. But current neuromorphic chips (Intel Loihi, IBM TrueNorth) don't natively support continuous-time differential equations. They operate in discrete time with spiking events.

Bridging this gap requires either:

• Analog neuromorphic hardware that implements ODEs in continuous time (using analog circuits)
• Hybrid digital-analog systems that approximate continuous dynamics with very fine timesteps

The hardware isn't there yet. Liquid networks currently run on GPUs, defeating some of their efficiency advantages.

Interpretability at Scale

The 19-neuron network is interpretable. What about a 1,900-neuron liquid network? At some point, the differential equations become as opaque as the weight matrices in a standard deep network.

The promise of interpretability depends on keeping networks small. Whether this is possible as task complexity increases remains to be seen.

What Liquid Networks Reveal About Computation Itself

Strip away the engineering details, and liquid networks tell us something fundamental: computation doesn't have to be discrete.

We've spent decades building computers on the von Neumann architecture—discrete memory, discrete operations, discrete time. Neural networks inherited this paradigm. Even recurrent networks, which purport to model temporal sequences, do so through discrete state updates.

But nature computes differently. Biological systems—neurons, gene networks, ecosystems—compute in continuous time. Their state variables evolve smoothly, governed by differential equations, not discrete algorithms.

Liquid networks show that this continuous paradigm isn't just biologically plausible—it's computationally advantageous. You get better generalization, greater efficiency, and clearer interpretability.

This suggests a broader lesson: the right computational architecture depends on the structure of the problem. If you're classifying static images, a convolutional network works fine. But if you're modeling dynamics—systems that flow through state space—continuous-time architectures are the natural fit.

Liquid networks aren't replacing transformers. They're carving out a different niche: continuous control, adaptive signal processing, real-time inference in changing environments.

This is computation that flows like water—smooth, adaptive, always in motion. It's what brains do. And it's what the next generation of neuromorphic systems will do.

The Bigger Picture: Toward Neuromorphic Active Inference

Liquid networks are part of a larger convergence: neuromorphic hardware, active inference, and continuous-time computation coming together.

The vision:

• Neuromorphic chips (Intel Loihi, analog memristor arrays) that implement spiking dynamics in real time
• Event-based sensors (neuromorphic cameras) that deliver continuous streams of data as the world changes
• Liquid networks running on this hardware, performing continuous-time active inference

The result: systems that perceive, predict, and act in continuous time, with minimal latency and maximal energy efficiency. Systems that adapt to novel conditions without retraining. Systems that maintain coherence across temporal scales.

This isn't science fiction. Hasani's team has demonstrated liquid networks on real-world tasks. Neuromorphic hardware is advancing rapidly. The pieces are coming together.

The question isn't whether this future arrives—it's how soon.

This is Part 5 of the Neuromorphic Computing series, exploring how brain-inspired hardware and algorithms are reshaping computation.

Previous: Event-Based Sensing: Cameras That See Like Eyes
Next: The Energy Crisis of AI: Why Neuromorphic Is Inevitable

Further Reading

• Hasani, R., Lechner, M., Amini, A., Rus, D., & Grosu, R. (2021). "Liquid Time-Constant Networks." AAAI Conference on Artificial Intelligence.
• Hasani, R., Lechner, M., Amini, A., Liebenwein, L., Ray, A., Tschaikowski, M., Teschl, G., & Rus, D. (2022). "Closed-Form Continuous-Time Neural Networks." Nature Machine Intelligence.
• Lechner, M., Hasani, R., Amini, A., Henzinger, T. A., Rus, D., & Grosu, R. (2020). "Neural Circuit Policies Enabling Auditable Autonomy." Nature Machine Intelligence.
• Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018). "Neural Ordinary Differential Equations." NeurIPS.
• Friston, K. (2010). "The Free-Energy Principle: A Unified Brain Theory?" Nature Reviews Neuroscience.