ideasthesia

Sign in Subscribe

Ryan Collison

I write about cool science and share it here.

Neural network internals with interpretable circuits and features revealed

Reading the Mind of AI: The Mechanistic Interpretability Revolution

What's actually happening inside a neural network when it solves a problem? Mechanistic interpretability researchers are finding out — identifying circuits, features, and algorithms that emerge from training. The goal isn't just to understand AI: it's also teaching us something about natural minds.

Synthesis of categorical structures—composition as geometric coherence

Synthesis: Category Theory as the Geometry of Composition

Category theory is abstract enough to describe almost anything — which turns out to be the point. It reveals structural identities invisible from inside any single field. When neural networks, probabilistic inference, and functional programming satisfy the same laws, something real is being said.

Active inference diagram with categorical structure—Markov categories

Category Theory for Active Inference: The Mathematical Backbone

Active inference needs a mathematics precise enough to prove things. Category theory, specifically Markov categories, provides that backbone: a framework for composing probabilistic processes that mirrors how agents update beliefs and plan actions. Dry notation, profound implications.

Operad tree structure with multiple inputs composing—algebra of operations

Operads and the Algebra of Composition: From Syntax to Semantics

Operads are the mathematics of plugging things into other things. From syntax trees to neural networks to biological pathways, they formalize how complex wholes are assembled from parts.

Sheaf over topological space—local data assembling into global meaning

Sheaves and Contextuality: How Category Theory Models Context-Dependent Meaning

Words mean different things in different sentences. Quantum measurements differ by experimental context. Sheaves are the mathematical tool built for this: data that's locally consistent but may fail to patch globally. The same structure appears across physics, logic, and the geometry of meaning.

String diagram with flowing wires and boxes—visual categorical calculus

String Diagrams: Drawing Your Way to Mathematical Insight

What if you could prove theorems by drawing? String diagrams are the graphical calculus of category theory — a notation so well-suited to composition that manipulating the pictures is the proof.

Neural network architecture as functor diagram—categorical deep learning

Neural Networks as Functors: Category Theory Meets Deep Learning

Category theory treats structure-preserving maps between mathematical objects as first-class citizens. Neural networks, it turns out, are functors — and this isn't just a metaphor. Applied category theory gives deep learning a mathematical grammar for why certain architectures work.