ideasthesia

Sign in Subscribe

Ryan Collison

I write about cool science and share it here.

Evolution from o1 to o3 reasoning models

From o1 to o3: How OpenAI Discovered Inference Scaling

Series: Test-Time Compute Scaling | Part: 2 of 9 The story of how OpenAI discovered test-time compute scaling isn’t a story of planned research proceeding according to hypothesis. It’s a story of surprising results, paradigm violations, and researchers noticing something that shouldn’t have worked—but did.

Inference-time reasoning scaling with extended computation

The New Scaling Law: Why Thinking Harder Beats Training Bigger

Series: Test-Time Compute Scaling | Part: 1 of 9 For years, AI researchers operated under a deceptively simple assumption: intelligence scales with size. Want a smarter model? Train it on more data with more parameters using more compute. The scaling laws discovered by OpenAI and others seemed to confirm this—

Synthesis of Graph RAG insights about structured coherence

Synthesis: What Graph RAG Teaches About Structured Coherence

Series: Graph RAG | Part: 10 of 10 Graph RAG started as an engineering solution to a retrieval problem: how to make AI agents reason about structured relationships instead of just finding similar text. But the deeper insight isn’t about retrieval. It’s about coherence. Knowledge graphs work because they

Knowledge graph as active inference generative model

Graph RAG Meets Active Inference: Knowledge as Generative Model

Series: Graph RAG | Part: 9 of 10 Knowledge graphs aren’t just databases. They’re generative models. Not in the LLM sense—they don’t generate text from learned patterns. In the active inference sense: they provide a structured prior over the state-space of possible answers. They constrain what’

Production Graph RAG infrastructure at scale

Graph RAG at Scale: Production Engineering Challenges

Series: Graph RAG | Part: 8 of 10 Graph RAG works beautifully on a demo corpus of 100 documents. You extract entities, build the graph, query it, and marvel at how well it handles multi-hop reasoning. Then you deploy to production with 100,000 documents, 10 million entities, and 100

Hybrid retrieval combining vector search and graph traversal

Hybrid Retrieval: Combining Vectors and Graphs

Series: Graph RAG | Part: 7 of 10 Neither vector search nor graph traversal alone is sufficient for production AI agents. Vector search casts a wide net—it finds semantically similar content even when terminology varies. But it misses structured relationships and fails at multi-hop reasoning. Graph traversal follows precise

Microsoft GraphRAG architecture diagram

Microsoft GraphRAG: Architecture and Lessons Learned

Series: Graph RAG | Part: 6 of 10 In April 2024, Microsoft Research released GraphRAG—an open-source implementation of knowledge graph-based retrieval for AI agents. It wasn’t the first Graph RAG system, but it was the first production-quality reference architecture from a major research lab with documented