According to the story told at OpenAI, the discovery happened by accident. In 2021, a researcher training a small neural network on a simple math problem stepped away and left the run going far longer than intended. They returned to something nobody expected: a model that had appeared completely stuck for thousands of training steps had suddenly learned to generalize perfectly.
The phenomenon seemed so alien that it was named from science fiction. In Robert Heinlein's 1961 novel Stranger in a Strange Land, the Martian word "grok" means to understand something so completely that you merge with it and it merges with you. The research community now uses "grokking" to describe what that network did: first memorize its training data, then plateau, then suddenly develop genuine understanding.
But the leap is not the real lesson. Thanks to remarkable follow-up work, we now understand exactly what happens inside a grokking network, making it perhaps one of the most completely understood AI systems in existence. And what that work shows is that the leap was never sudden at all: the solution formed gradually, over thousands of steps, in a place standard metrics cannot see. Standard training metrics can completely miss actual learning. They cannot tell a model that memorized from a model that understands, and for AI that acts in the physical world, that difference separates a system you can trust from one that merely looks trained.
The puzzle: perfect training, terrible testing
The researchers were studying modular arithmetic, which you already do every time you read a clock. If it's 11 AM and you have a 3-hour meeting, it ends at 2 PM: 11 + 3 = 14, which wraps past 12 to leave 2. That is modular arithmetic with a modulus of 12.
They trained a small transformer (the same architecture underlying ChatGPT, just with far fewer parameters) on modular addition with a modulus of 113. Given inputs like "17 + 45 =", the model should output "62" (since 17 + 45 = 62, which is less than 113). Given "100 + 50 =", it should output "37" (since 150 mod 113 = 37). Some examples were held out for testing.
After a few hundred steps, the model reached 100% accuracy on the training data. Standard practice says stop here: the model appears to have learned the task. But on the held-out test problems it scored around 1%, no better than guessing. It had memorized the training examples without learning modular addition at all.
Normally that is the end of the story: the model overfit, try a different architecture or more data. But the run kept going. For thousands more steps, both curves stayed flat, training accuracy at 100%, test accuracy near 0%. Then test accuracy shot up, quickly reaching nearly 100%. The model could now solve problems it had never seen.
What was happening during those flat steps, and what triggered the transition?
The network invented Fourier analysis
In early 2023, Neel Nanda (then at Anthropic, now at Google DeepMind) and collaborators published the answer at ICLR 2023. They didn't just explain grokking: they reverse-engineered the exact algorithm the network had learned.
To solve modular addition, the network uses Fourier analysis and trigonometric identities. Picture a clock face: adding hours is rotating around a circle. 3 o'clock plus 4 hours means rotating from the 3 position (90 degrees) by 4 hours (120 degrees) to reach 7 o'clock (210 degrees). Modular arithmetic with modulus 113 works the same way, on a circle divided into 113 segments.
With no instruction to do so, the network learns to represent inputs as points on circles using sines and cosines, the mathematical language of rotation. For "17 + 45 =", it computes the sines and cosines of 17 and 45 at specific frequencies, then applies the identity cos(x)·cos(y) - sin(x)·sin(y) = cos(x+y) to compute the sum. It was never taught trigonometry. It was given no hint that circles or waves might be relevant. Through gradient descent alone, it discovered that Fourier representations are an efficient way to solve modular arithmetic, independently reinventing a technique humans took centuries to develop.
The evidence is visible neuron by neuron. Plot activations as a function of the inputs x and y and you find waves: some neurons compute cos(x), others sin(y), others products like cos(x)·cos(y). Plot pairs of neuron outputs against each other and you see circles and helixes, the same circular representations that make analog clocks intuitive for modular arithmetic. Certain neurons fire maximally for every input pair that adds to a specific value: one activates strongly whenever x + y = 65, regardless of the specific x and y. That diagonal symmetry is exactly what addition requires, and it emerges naturally from the Fourier representations.
Three phases, and metrics miss the one that matters
With tools to probe the network's internals, Nanda's team could watch learning unfold in detail. Training proceeds through three distinct phases.
Phase 1: memorization (steps 0–1,400). The model rapidly learns to answer every training example, but its internals show none of the Fourier structure of the final solution. It is building a high-dimensional lookup table from each input to its output. This reaches 100% training accuracy quickly, which is why conventional metrics suggest learning is complete.
Phase 2: circuit formation (steps 1,400–9,400). Externally, nothing happens: training accuracy stays at 100%, test accuracy near 0%. Inside the network, the sine and cosine patterns are slowly building up in the weights. The circuit that will implement modular addition is under construction while the network still answers from memory. To detect this hidden progress, the researchers built a new metric called "excluded loss" that measures how much the network relies on the key Fourier frequencies. It climbed steadily through the whole phase, revealing learning that conventional approaches completely missed.
Phase 3: cleanup (steps 9,400–14,000). Once the Fourier circuit is fully formed, weight decay (a regularization technique that penalizes large weights) begins erasing the memorization solution. The lookup table dissolves; the more efficient circuit remains. This is when test accuracy spikes. What looks like sudden insight is the unveiling of a solution that had been building for thousands of steps.
A practitioner who stopped this run when the metrics flatlined would have shipped the lookup table.
The same geometry shows up in Claude
Grokking in modular arithmetic might look like a curiosity about small networks on toy tasks. The implications reach the frontier models we use every day.
Large language models undergo phase transitions during training in which capabilities suddenly emerge, often surprising the researchers training them. Grokking suggests those transitions may not be truly sudden: the underlying circuits may form gradually, hidden from our metrics, until they finally dominate the model's behavior.
Anthropic's research on Claude offers a striking parallel. Studying how Claude 3.5 Haiku decides when to insert line breaks while writing, researchers found the model represents character count on a six-dimensional manifold, a geometric structure remarkably similar to the circles found in grokking. The model tracks its position on the current line with a helical representation in activation space. Different attention heads specialize in detecting different distances from the end of a line: one fires when about 5 characters remain, another when 10 remain. A geometric mechanism called the "QK twist" rotates these representations to compare current position against target line width.
It is the same phenomenon at a different scale: a network discovering that circular representations naturally solve a counting problem, with billions of parameters instead of thousands.
Understanding is possible, but still rare
The grokking network is a high-water mark for interpretability. We can state exactly what algorithm it learned, why the algorithm works, and how it developed during training: a transparent box rather than a black one.
For frontier models like GPT-4 or Claude, we have compelling clues, not complete explanations. Anthropic has identified millions of interpretable features using sparse autoencoders, traced some computational pathways using attribution graphs, and demonstrated causal manipulation of model behavior. Comprehensive understanding of how these models accomplish complex reasoning remains elusive.
That gap matters for safety. A model we cannot explain is a model whose failures we cannot predict and whose unintended behaviors we cannot detect. For physical AI systems (autonomous vehicles, robots, medical devices), those failures have real-world consequences. Grokking's contribution is proof of possibility: networks are not random soups of weights, they learn structured solutions that make sense when properly analyzed. The challenge is developing interpretability techniques that scale to models orders of magnitude larger.
What grokking says about training data
For those of us building AI infrastructure, grokking offers practical guidance.
Training dynamics matter as much as training data. Two models trained on identical data with slightly different hyperparameters can land in very different places: one memorizing, one generalizing. The conditions that promote genuine learning over memorization should inform how training pipelines are structured.
Data quality changes the timeline. The original grokking runs needed extensive training before generalization emerged. Follow-up research has shown that higher-quality data, better training procedures, and appropriate regularization can dramatically accelerate the transition; in the limit, the memorization phase might shrink to nothing.
This is directly relevant for physical AI. An autonomous vehicle or robot needs robust representations of the physical world (3D space, object permanence, physics, causality), not memorized sensor patterns. The difference between a model that memorizes training scenarios and one that groks physical reality is the difference between brittleness and robustness. Grokking also explains why certain data qualities matter: richer sensor data, combining cameras with LiDAR and radar, gives more complete information about physical structure; diverse scenarios prevent memorization of narrow patterns; temporal coherence helps models learn dynamics rather than snapshots.
The alien in the network
We want to close with something Welch Labs noted in their excellent video on grokking: these findings should make us humble about anthropomorphizing AI systems. As Andrej Karpathy recently observed, training large language models is "less like building animal intelligence and more like summoning ghosts." A ghost is a fundamentally different kind of point in the space of possible intelligences, something alien that we communicate with but may never fully understand.
The network that learned modular addition reinvented Fourier analysis, but it did not learn our way of doing arithmetic. Most humans would solve 17 + 45 mod 113 by computing 62 directly, then checking whether it exceeds 113. The network converts inputs to wave representations, multiplies sines and cosines, uses trigonometric identities, and recovers the answer from the resulting pattern. Both approaches work. The network's is mathematically beautiful, computationally efficient, and utterly alien to human intuition about arithmetic.
We communicate with AI systems in human language, but beneath that interface they process information in ways we are only beginning to map. We cannot assume they reason as we do, fail as we would, or generalize in ways we'd expect.
Grokking, understanding so complete that you merge with the subject, is ironically named. We grok how this small network grokked arithmetic. For frontier models, we do not yet, and until we do, humility is not a virtue but a requirement for anyone putting these systems to work in the physical world.
This piece draws heavily on Welch Labs' excellent video "Grokking: The Most Understood AI Model," which we recommend watching for its visualizations of these concepts.
The grokking phenomenon was first documented by Power et al. at OpenAI in January 2022 (paper). The origin story, a researcher accidentally leaving a training run going longer than intended, is told at Girl Geek X OpenAI Lightning Talks, though Welch Labs notes they weren't able to reach the team to independently confirm the details.
Neel Nanda's mechanistic analysis with collaborators appeared at ICLR 2023 (paper). For more on mechanistic interpretability, Neel's getting started guide is the best entry point. Quanta Magazine also has an excellent accessible overview: How Do Machines 'Grok' Data?
The Claude Haiku character-counting research by Emmanuel Ameisen, Wes Gurnee and team at Anthropic is available at transformer-circuits.pub, and it's in-depth and worth reading in full.
Andrej Karpathy's observation that training LLMs is "less like building animal intelligence and more like summoning ghosts" appears in his essay Animals, Aliens, and AIs, a meditation on the alien nature of these systems.
At Avala, we study how AI systems learn to understand the physical world, and how data quality influences that learning. If you're building physical AI and thinking carefully about training dynamics, we'd love to talk.