Training a model means working backwards from effects to causes: you see data and look for parameters that explain it. Checking whether a slope is stable is the forward problem; working out why it collapsed after the fact is the inverse problem.
Inverse problems come with a catch: the same data can be explained equally well by many different sets of parameters. The more parameters, the more of these equally good explanations there are. A training score tells you how well the data is explained, not which explanation you landed on, and different explanations behave very differently once you leave the data.
So the only reliable way to find out whether a model learned something is to look outside the data. Here are the three checks I use in my memory-model research, and a result where bigger was worse.
The experiment: shared common sense plus each person's own things
The memory system I'm building has two layers. One is common sense shared by everyone: the regularities of relations themselves, such as what "depends on," "is used for," and "belongs to" usually look like. The other is each user's own things: where specific people, events, and objects sit. The idea is to learn the shared layer once from large public or synthetic data and freeze it; when a new user arrives, only their small private part is learned.
To test that idea I had a language model generate about 1,800 small synthetic worlds, each with a few hundred relations and a hundred or so entities. About 1,700 trained the shared layer; 89 were held out entirely. In each held-out world, the shared layer stays frozen, only that world's own entities are learned, and I measure how well it fills in missing relations, compared against learning from scratch with no shared layer. This is an earlier corpus, not the 1.27-million-relation audit in the previous post.
The score is MRR: 1 point if the correct answer is ranked first, 1/2 if second, 1/3 if third, and so on, averaged. Higher is better.
| Shared-layer capacity | Training worlds | Unseen worlds | From scratch | Unseen worlds won by shared layer |
|---|---|---|---|---|
| 16 | 0.48 | 0.26 | 0.17 | 74 / 89 |
| 32 | 0.51 | 0.25 | 0.17 | 63 / 89 |
| 64 | 0.65 | 0.21 | 0.17 | 48 / 89 |
Quadrupling capacity took the training score from 0.48 to 0.65, while the unseen-world score fell from 0.26 to 0.21 and the number of worlds won fell from 74 to 48. The larger model memorized the training worlds better and carried less to new ones.
That is overfitting, and not surprising in itself. One caveat: these are three points from one training recipe with no regularization; a regularized large-capacity version has not been tested.
The interesting part is elsewhere. Looking only at training scores, I would have picked 64. Looking only at "how much better than scratch," I might have been very pleased with the shared layer. What made the conclusion hold up were the three checks below.
Check 1: test on a world the model has never seen, the way it will be used
The test world has to be one the model never saw, and the test has to match how it will run in production: shared layer frozen, only a few facts about the new world. Hiding a few edges inside a training world tests whether the model remembers; switching to a new world tests whether it learned.
Quant research is the same. An out-of-sample slice cut from the same stretch of history only shows that a strategy didn't memorize the answers outright. Holding up in a different market or period is stronger evidence consistent with a mechanism, not proof of one. A rule I set for my own quant research: what moves to a new market is the method for finding and validating signals, not the coefficients.
Check 2: compare against a shuffled control
This is the check I recommend most. Scramble which matrix in the shared layer belongs to which relation. The numbers, scale, and shapes all stay the same; only the identities are mismatched. Then run the exact same test.
At capacity 16: real shared layer 0.26, shuffled shared layer 0.22, from scratch 0.17.
In this run, 0.05 of the 0.09 advantage over scratch survives the shuffle. That is consistent with a generic benefit from starting at a trained point with a sensible scale. The remaining 0.04 is the relation-identity-sensitive component of this comparison; without repeat runs or confidence intervals, it is not a clean causal decomposition.
Reporting the full 0.09 as relation-specific would have been about twice the observed relation-sensitive gap in this run.
The quant equivalent: compare a signal against random signals with the same turnover and style exposure, or shuffle the signal across the cross-section and backtest again. A lot of what looks like alpha is style exposure or some structural, generic return. Strip first, then talk about alpha; that is the first gate in The Geometry of Alpha.
Check 3: audit for leaks, including leaks in the control
The 24-of-24 audit in the previous post is a later batch of worlds. That audit also used a control that shifted entities cyclically within the same kind, and it looked suspiciously good. On inspection, that shift happened to map 15 of the 40 hidden test items onto items seen in training. The control had effectively seen 37.5% of the answers in advance.
Leaks are not confined to models. They show up in controls, evaluation scripts, and data splits. In quant research the usual suspects are look-ahead information, labels that overlap in time, and universes that contain only survivors. The book chapters on Purged cross-validation and backtest pitfalls cover these in detail.
A finding I didn't expect: shared structure is what transfers
Back to the capacity table. I also measured how similar the different relation matrices in the shared layer are to each other:
| Shared-layer capacity | Average similarity between relation matrices | Unseen-world gain over scratch |
|---|---|---|
| 16 | 0.67 | 0.091 |
| 32 | 0.44 | 0.076 |
| 64 | 0.27 | 0.046 |
At low capacity, different relations are forced to share the same directions. At high capacity, each relation can claim its own space. Across these three settings, higher measured sharing is associated with better transfer. One plausible explanation is that relation-specific capacity fits details of the training worlds, but these three points do not identify the cause.
That ran against my intuition. I had planned to add a constraint pushing the relation matrices to be more orthogonal, more independent of one another. These results warned that the direction might be wrong; the constraint itself was not tested.
If the way to say it is that repeated structure can be merged, this experiment adds a hypothesis: in a prior meant to transfer, the shared component is a candidate source of transfer. "Merging" would then mean keeping one shared copy, not deleting everything that looks alike. This remains a correlation across three points, not a proof of cause.
An honest negative result: multiplying matrices is not multi-step reasoning
In the same experiments I tested multi-step reasoning: can "what did A's student create?" be computed by multiplying the "student of" matrix by the "created" matrix? On unseen paths, ranking against every entity, the correct order scores 0.072 and the reverse scores 0.074. The correct order is not higher. After the candidates are narrowed to tails of the second relation, the correct order scores 0.313 and the reverse 0.307. The observed gap is 0.007, too small here to distinguish confidently from noise or to be useful.
So multi-step questions in my memory system follow recorded fact paths and written rules, not matrix products. The previous post goes into that in more detail.
Conclusion
A training score answers "did it remember?" These three checks answer "did it learn?":
- Test on a world it has never seen, the way it will actually be used.
- Compare against a shuffled control and subtract the generic benefit.
- Audit for leaks, including leaks in the control itself.
The principles carry over to quant research: swap "world" for "market" and "regularities of relations" for "return mechanism." Markets add nonstationarity, costs, capacity, and execution, so passing these checks is still only evidence, not admission to trade. The next post pushes check 2 onto a factor library: Factor Deduplication: Eleven Factors, Fewer Than Five Independent Bets.