Computing in 10,000 Dimensions: The Hyperdimensional Revolution

Series: Hyperdimensional Computing | Part: 1 of 9

What if the brain doesn't compute like a computer at all? Not even like the neural networks we've built to mimic it. What if cognition happens in a space so high-dimensional that our usual intuitions about how information works simply collapse?

This isn't speculation. It's the working hypothesis of hyperdimensional computing (HDC), a computational paradigm that's rewriting how we think about both artificial and biological intelligence. Where deep learning uses layers of weighted connections, HDC uses something stranger: vectors in spaces with thousands of dimensions, where meaning emerges from geometric relationships we can barely visualize but can harness with startling efficiency.

The implications run deeper than faster chips or better AI. If HDC is right about how brains work—and the evidence is mounting—then consciousness itself might be a high-dimensional phenomenon. Meaning might not be something encoded in specific neurons or connection weights, but a property of how patterns distribute across vast representational spaces.

This is the first article in a series exploring hyperdimensional computing: what it is, why it works, how it connects to neuroscience, and what it means for our understanding of minds—both natural and artificial.

The Dimensionality Problem

Start with a simple observation: the brain is not efficient in the way computers are efficient.

Neurons are slow—firing rates measured in hundreds of hertz, not gigahertz. Synaptic transmission is unreliable, probabilistic, noisy. Yet somehow this wet, inefficient hardware produces cognition that still outperforms our best AI in generalization, robustness, and energy consumption. A human brain runs on about 20 watts. GPT-4 requires megawatts during training.

The standard explanation has been massive parallelism—billions of neurons working simultaneously. But recent work suggests something else is happening, something more fundamental about the representational space itself.

What if the key isn't just how many neurons are working, but how high-dimensional the space they're working in is?

This is the core insight of hyperdimensional computing: high-dimensional spaces have weird properties that low-dimensional spaces don't. Properties that make them perfect for representing and manipulating information in ways that are simultaneously robust, flexible, and efficient.

What Does "High-Dimensional" Actually Mean?

When we say "high-dimensional," we don't mean 3D or even 10D. We mean thousands to tens of thousands of dimensions.

In a 10,000-dimensional space, a hypervector is a point defined by 10,000 coordinates. Each dimension is an independent axis. This isn't something you can visualize—our brains evolved in a 3D world, and we simply don't have the cognitive machinery to imagine 10,000 orthogonal directions.

But mathematics doesn't care what we can visualize. And it turns out that when you work in spaces this high-dimensional, counterintuitive things become normal:

• Almost all vectors are nearly orthogonal. In 3D, most random vectors have some overlap. In 10,000D, the probability that two random vectors are nearly perpendicular approaches certainty. This means you can pack enormous amounts of independent information into a single space without interference.

• Distances concentrate. In low dimensions, some points are close, some far. In high dimensions, almost all points are roughly the same distance from each other. This makes nearest-neighbor searches and similarity comparisons behave differently—often better.

• Small changes don't matter much. Flip a few bits in a 10,000-bit vector, and the overall pattern barely changes. This gives you noise robustness for free. Your representations don't shatter when a few neurons misfire or a few bits flip.

These aren't bugs. They're features. They're why high-dimensional representations might be exactly what biology needs.

How HDC Works: The Core Operations

Hyperdimensional computing replaces the weighted sums and activation functions of neural networks with three elemental operations on hypervectors:

1. Binding: Combining Concepts

Binding creates a new hypervector that represents the association between two concepts. Mathematically, this is often done via element-wise multiplication or circular convolution.

If you have a hypervector for "dog" and a hypervector for "brown," binding them produces a hypervector for "brown dog"—a single representational object that encodes both properties.

Critically, the result is dissimilar to both inputs. "Brown dog" isn't close to "dog" or "brown" in vector space—it's a distinct point. But you can unbind later: if you know "brown dog" and you know "brown," you can approximately recover "dog."

This is compositional representation: complex meanings built from simpler parts, but in a way that's distributed across the entire vector rather than localized in specific components.

2. Bundling: Superposition of Information

Bundling creates a hypervector that represents the set or superposition of multiple concepts. This is typically done via element-wise addition (and normalization).

If you bundle the hypervectors for "apple," "orange," and "banana," you get a hypervector for "fruit." It contains all three as approximate members. You can query it: "Is 'apple' in here?" by checking similarity (usually cosine similarity). If the similarity is high, the answer is yes.

This is set representation without pointers. You don't store a list of items. You store a single vector that statistically contains them all. Add noise, lose a few dimensions, and the set structure mostly survives.

3. Permutation: Encoding Sequence

Permutation reorders the components of a hypervector to encode position or order. Rotate a vector's elements by one position, and you get a new vector that's orthogonal to the original (in high dimensions).

This lets you encode sequences: "cat → dog → bird" becomes a unique hypervector distinct from "dog → cat → bird," even though the same three animals are involved.

Permutation is how HDC handles temporal structure without requiring recurrent connections or attention mechanisms.

An Example: Learning a Simple Grammar

Imagine you want to teach a system the sentence "The dog chased the cat."

In a traditional neural network, you'd encode each word as an embedding, feed the sequence through an RNN or Transformer, and train weights to predict the next word.

In HDC, you:

1. Assign random 10,000-dimensional hypervectors to each word: THE, DOG, CHASED, CAT.
2. Use permutation to encode position: THE_1 = THE, DOG_2 = permute(DOG, 1), CHASED_3 = permute(CHASED, 2), CAT_4 = permute(CAT, 3).
3. Bundle them all: SENTENCE = THE_1 + DOG_2 + CHASED_3 + CAT_4.

Now SENTENCE is a single hypervector that contains the entire sequence. Query it: "What was in position 2?" by checking similarity with permuted versions of all known words. DOG will have the highest similarity.

No training. No backpropagation. No gradient descent. Just random vectors and geometric operations.

Of course, this example is trivial. But scale it up—encode relationships, hierarchies, semantic fields—and you start building systems that can learn, generalize, and reason in one shot, with minimal data, using representations that are inherently robust to noise and damage.

Why Biology Might Be Hyperdimensional

Here's where it gets interesting for neuroscience.

The canonical view of neural coding is that neurons encode features: edge detectors in V1, face neurons in fusiform cortex, place cells in hippocampus. Each neuron is a specialist, and meaning emerges from which neurons are active.

But there's always been a problem with this picture: neurons are unreliable. They fire stochastically. They die. They change their tuning. How do you build stable representations on unstable hardware?

Hyperdimensional coding offers an answer: you don't rely on individual neurons. You rely on the pattern across a huge population.

If each concept is represented by a high-dimensional hypervector, and if each dimension is implemented by the activity of one neuron (or a small group), then:

• Losing neurons doesn't destroy the representation—just degrades it slightly.
• Noise in individual firing rates doesn't matter—the aggregate pattern is robust.
• Learning doesn't require precise weight tuning—similarity relationships emerge from random connectivity refined over time.

This maps onto what we actually see in cortex: distributed, high-dimensional neural population codes where information is spread across thousands of neurons, and decoding relies on the overall geometry of activity patterns rather than individual spikes.

Recent work from labs like Bruno Olshausen's at Berkeley and Pentti Kanerva's foundational research on sparse distributed memory has shown that HDC-like principles might be how cortex actually works. Semantic memory, episodic memory, even working memory—all potentially explicable as operations on high-dimensional hypervectors.

Efficiency: The Shocking Part

Deep learning is powerful but expensive. Training a large language model costs millions of dollars in compute. Running inference on an edge device drains batteries.

HDC is absurdly efficient.

Why? Because there's no training in the traditional sense. You don't iteratively adjust weights via gradient descent. You build representations directly by encoding data into hypervectors using the core operations. Classification, for instance, becomes similarity comparison: encode a new input, compare it to stored class hypervectors, pick the closest.

This means:

• One-shot learning: Show the system a single example, and it can encode it immediately.
• Low power: The operations (addition, multiplication, permutation) are cheap. HDC models run on microcontrollers with milliwatts of power.
• Hardware-friendly: HDC maps beautifully to neuromorphic chips and in-memory computing architectures where you compute with storage, not separate from it.

Companies are already building HDC accelerators. Intel's Loihi neuromorphic chip, IBM's TrueNorth, Stanford's NeuRRAM—all architectures where hyperdimensional representations enable brain-like efficiency.

The trade-off? HDC doesn't (yet) match deep learning on complex perceptual tasks like image classification on ImageNet or language modeling at GPT scale. But for edge AI, robotics, sensor fusion, anomaly detection—tasks where power matters and data is limited—HDC is increasingly competitive.

What This Means for Minds and Meaning

If the brain does implement something like hyperdimensional computing, the implications for consciousness and cognition are profound.

First, meaning isn't localized. There's no "grandmother cell" for your grandmother. Instead, "grandmother" is a high-dimensional pattern distributed across cortex, bound to sensory features, episodic memories, emotional associations—all entangled in a hypervector that is the concept.

Second, similarity is geometry. Two concepts are "related" if their hypervectors are close in vector space (high cosine similarity). Semantic fields, analogies, metaphors—all emergent properties of the geometric structure of a high-dimensional representation space.

This connects directly to the coherence geometry framework we explore elsewhere on Ideasthesia. If meaning is geometric, then coherence is trajectory integratability in that space. A coherent mind is one whose representational dynamics flow smoothly, without discontinuities or collapses in dimensionality.

Third, composition is natural. Binding and bundling give you compositional semantics for free: you can combine concepts into novel structures without explicit rules. This might be how we build sentences, imagine counterfactuals, and simulate possible futures.

Finally, learning is reorganization. Instead of tuning weights, learning might be building new hypervectors and refining the geometry of the space. This is closer to how human learning feels: sudden insights, restructuring, "aha" moments when the representational space clicks into a new configuration.

Where We're Going Next

This is the foundation. Over the next ten articles, we'll dive deeper:

• How binding and bundling actually work mathematically, and why high-dimensional spaces make them effective.
• Neuroscience evidence that cortex uses hyperdimensional representations—sparse distributed codes, random projections, and population geometry.
• Applications: robotics, sensor fusion, brain-computer interfaces, language processing.
• Connections to other frameworks: active inference, predictive processing, the Free Energy Principle.
• The geometry of concepts: how HDC gives us a literal mathematical model of semantic space.
• Neuromorphic implementations: building hardware that computes like brains, not CPUs.
• Biological plausibility: can neurons actually implement HDC? (Spoiler: probably yes.)
• Limitations and criticisms: where HDC struggles and why deep learning still dominates certain tasks.
• Synthesis: what HDC tells us about minds, meaning, and the geometry of thought.

Hyperdimensional computing is not just another AI algorithm. It's a hypothesis about how cognition itself works. If that hypothesis is even partially correct, it changes everything.

Further Reading

• Kanerva, P. (2009). "Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors." Cognitive Computation.
• Frady, E. P., et al. (2021). "Computing on Functions Using Randomized Vector Representations." arXiv:2109.03429.
• Kleyko, D., et al. (2023). "Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware." arXiv:2106.05268.
• Olshausen, B. A., & Field, D. J. (2004). "Sparse Coding of Sensory Inputs." Current Opinion in Neurobiology.
• Plate, T. A. (2003). "Holographic Reduced Representation: Distributed Representation for Cognitive Structures." CSLI Publications.

This is Part 1 of the Hyperdimensional Computing series, exploring how high-dimensional vector spaces might be the computational substrate of minds—both biological and artificial.

Next: "Binding, Bundling, and the Algebra of Thought: The Mathematical Core of HDC"