ideasthesia

Sign in Subscribe

Ryan Collison

I write about cool science and share it here.

Sparse distributed memory architecture visualization

Pentti Kanerva and the Origins of Hyperdimensional Computing

Series: Hyperdimensional Computing | Part: 2 of 9 In 1988, a Finnish computer scientist working at NASA’s Research Institute Center published a book that wouldn’t make sense for another thirty years. Sparse Distributed Memory introduced an architecture so unlike conventional computing that most dismissed it as biological speculation—interesting

High-dimensional vector space with holographic distributed representations

Computing in 10,000 Dimensions: The Hyperdimensional Revolution

Series: Hyperdimensional Computing | Part: 1 of 9 What if the brain doesn’t compute like a computer at all? Not even like the neural networks we’ve built to mimic it. What if cognition happens in a space so high-dimensional that our usual intuitions about how information works simply

Synthesis of interpretability insights revealing coherence geometry

Synthesis: What Neural Network Internals Teach Us About Coherence

Series: Mechanistic Interpretability | Part: 9 of 9 When Chris Olah first opened the hood on a vision network in 2017, he found something unexpected. The model hadn’t learned arbitrary statistical correlations. It had learned circuits—composable, interpretable mechanisms that built understanding hierarchically. Curve detectors fed into wheel detectors, which

Human and AI cognitive systems aligning through interpretable interfaces

Human-AI Coherence Teams: Why Interpretability Matters for Collaboration

Series: Mechanistic Interpretability | Part: 8 of 9 In the summer of 2023, a team at Anthropic discovered something unexpected while probing Claude’s internal representations: the model had learned separate “circuits” for truthfulness and helpfulness that sometimes competed with each other. The researchers could watch, in real time, as different

Parallel visualization of artificial and biological neural circuits

What AI Interpretability Teaches Us About Biological Brains

Series: Mechanistic Interpretability | Part: 7 of 9 For most of neuroscience’s history, we’ve been stuck with a fundamental limitation: we can’t see what we’re studying. We can record electrical activity. We can ablate regions and watch behavior change. We can image blood flow and call it

Feature manifold showing geometric structure of neural representations

Where Interpretability Meets Information Geometry

Series: Mechanistic Interpretability | Part: 6 of 9 When Anthropic’s researchers visualized what they called “features” inside Claude—the concepts their AI had learned to represent—they weren’t just looking at activations in neural networks. They were looking at geometry. The features existed as directions in a high-dimensional

Sparse autoencoder decomposing neural activations into interpretable features

Sparse Autoencoders: Extracting the Dictionary of Neural Concepts

Series: Mechanistic Interpretability | Part: 5 of 9 The problem seemed insurmountable. Neural networks work brilliantly, achieving superhuman performance on increasingly complex tasks. But when you try to understand how they work—when you peer inside to find the concepts they’ve learned—you encounter something that looks less like orderly