Grammar and meaning are usually treated as separate layers of language. One is a system of formal rules; the other is a network of concepts and referents. In transformer language models, that separation gets harder to hold onto. A model that predicts the next token has to learn something about agreement, word order, and argument …
Continue reading The Relationship Between Grammar and Meaning in Neural Networks
Author:Terri Lane
Why Narrative Coherence in Language Models Is Not Story Understanding
Ask a language model to write a short story. You will get something that reads like one. A protagonist, a conflict, a turning point, a resolution. Grammatical prose. Emotional shifts at the moments you would expect. Skim it and you might not notice anything wrong. That is exactly the problem. What looks like storytelling is, …
Continue reading Why Narrative Coherence in Language Models Is Not Story Understanding
Why Alignment Research Needs Philosophical Rigor
Alignment research is the study of how to make the goals of an artificial system match the goals of its designers and users. It sits at the intersection of technical specification, behavioral testing, and normative judgment. Adjacent concepts include value specification, reward modeling, interpretability, and robustness. For a blog that examines where language models fail …
Continue reading Why Alignment Research Needs Philosophical Rigor
How Pre-Training Data Determines Model Behavior
Pre-training data is the initial, large-scale text corpus used to shape a model’s statistical expectations before any task-specific adjustment. In morphosyntax research, this corpus is not a neutral substrate. It encodes frequency distributions, register biases, orthographic conventions, and cross-linguistic asymmetries that later surface as model behavior. Adjacent concepts include data mixture ratios, tokenization artifacts, corpus …
Continue reading How Pre-Training Data Determines Model Behavior
How Pre-Training Data Shapes Morphosyntactic Behavior: A Distributional View
When a model spits out a sentence with mangled grammar, it’s easy to shrug and call it a glitch. But look closer. The error usually isn’t random—it’s a fossil of the pre-training corpus, a direct reflection of what the model saw, how often it saw it, and in what contexts. This piece digs into that …
Continue reading How Pre-Training Data Shapes Morphosyntactic Behavior: A Distributional View