Category:Main

We build a vector for a word, a sentence, or an image, and we casually say it lives in embedding space. But the moment we start reading clusters, distances, and directions as if they map neatly onto how the mind carves up the world, we have wandered into conceptual space. The pull is real. Two …
Continue reading Why Embedding Space Is Not the Same as Conceptual Space

How Context Windows Shape What a Model Can Remember Walk into any library and you’ll find shelves that stretch beyond the reach of a single glance. A reader can only hold a few pages in their mind at once. The rest must be fetched and re-read. Computational systems that process language face a similar constraint: …
Continue reading How Context Windows Shape What a Model Can Remember

Think back to the last long novel you read. By the time you reached the final chapter, you probably weren’t holding every comma or adverb in your head. You remembered the broad strokes—key themes, a few characters’ defining moments, a twist that reframed everything. Information-processing systems work under a similar constraint: they have a fixed …
Continue reading The Memory Horizon: How Context Windows Shape What a Model Can Remember

When we come across a piece of generated text, our initial reflex is to judge it. Does it sound natural? Does it hold together? Does it say what it’s supposed to say? On the surface, these questions look straightforward, but underneath they hide a tangle of theoretical and practical problems. Judging the quality of language …
Continue reading The Problem With Evaluating Language Generation Quality

Hand a sentence to a system built on mountains of text, and you expect meaning to stay put. Words ground us—or so we like to think. But the meanings that surface can warp quietly, sometimes radically. That warping—semantic drift—isn’t just a headache for engineers. It unsettles anyone watching language buckle under computational weight. I’ve spent …
Continue reading Why Semantic Drift in Language Models Is Hard to Measure