Blog

Finding what a new thing resembles is easy; making it inherit what it should know is hard. In my structured-memory experiment, embedding scores recovered 2 of 24 facts a new entity should inherit; a written rule recovered all 24 and answered "no evidence" for all 40 facts the system had no record of. How to split an AI memory into lookup, derivation, and labelled guessing.

The number of factors in a library is not the number of independent signals. In one frozen sample, eleven momentum, value and size series built from Ken French public data have an average pairwise correlation of 0.13; grouping them into four correlation clusters leaves full-sample gross Sharpe essentially unchanged. The effective-number formula, a deduplication procedure, and a reproducible script.

A high training score does not mean a model learned anything it can take elsewhere. In one transfer experiment on a memory model, quadrupling capacity raised the training score from 0.48 to 0.65 and lowered the score on unseen worlds from 0.26 to 0.21. Three checks that catch this: test on a new world, compare against a shuffled control, and audit for leaks.

The same "close issues idle for 30 days" task handed to Claude through an MCP server, a bare command line, and the command line plus a SKILL.md, five runs each, measuring model calls, tool calls, token footprint, wall time and failure modes. All 15 runs were correct, but the cost differed by more than 2x: MCP averaged 3.2 model calls, CLI 7.4, Skills 6.0. With a reproducible single-file harness, and why WebMCP is not on the same track.

You downloaded a sample of historical order book data. Now what? A dependency-free Python order book that replays 441,392 Nasdaq MBO messages for AAPL in 2.1 seconds, verified against the vendor MBP-10 feed at 148,402 events with zero mismatches, plus the MBO versus MBP distinction, the trade-record trap, and what you can measure once the book is rebuilt.

A 200-line Python agent: web search, file tools, memory across runs, and resume-from-checkpoint after a crash. Install commands, the full code, output from four real runs with token counts, and one thing I did not expect: the search tool ships with a hidden sandbox.

DeepSeek Harness crossed six figures of GitHub stars in days and got framed as an open-source Claude Code killer. After the docs and the Cordis paper, it reads as a framework for recomposing agents. Kocoro is a local contract: Skill / MCP / Gateway already layered, the loop pinned. Those are not the same plugin kernel.

Geometry of Alpha described the factory; One Person + AI described the org shape. This piece is the missing middle: how frozen gates become agent-executable contracts, what a candidate looks like walking the line overnight, which outputs are first-class (including every NO GO), and which doors must stay human — and fail closed — by construction.

Are Kimi K3 and DeepSeek V4 treating the same disease? A proposed unified reading—unbounded activations, discrete experts, cross-layer amplification—with different medicine. Stable LatentMoE, SiTU softcaps, and why MLA still sits in a hybrid stack.

A traditional quant fund scales research capacity with headcount: dozens of PhDs, each running a slice of the pipeline. Once the research process itself is frozen into an executable discipline, AI agents can take over enumeration, verification, monitoring, and reporting — and for the first time, throughput decouples from headcount. This piece covers the design principles of that new shape: what goes to the machine, what stays with the human, and why discipline is the precondition for AI, not the result of it.

Position sizing answers "how big a bet." Safety control answers "when do you cut me off." These are two control systems with fundamentally different mathematical properties, and the most dangerous risk-design mistake in the industry is merging them into one. This piece covers where that line sits, why market-neutral does not mean crisis-neutral, and why real safety control needs veto power at the architecture level.

The era of the single "magic signal" is over. The asset that actually compounds is not any one recipe — it is the factory that keeps producing, validating, and retiring signals. This piece is about the math behind weak-signal aggregation: why the √N law is the Pythagorean theorem, why factor stripping is a projection, why tail correlation is holonomy — and why reading markets and reading neural networks turn out to use the same geometry, the same four realms.

I argued that loss spikes are geometric tears, not optimization blowups. So I went and measured the tear directly — in released MoE weights, OLMoE and Qwen. It is a genuine C⁰ jump at every layer, severe in the ~24% block-output jump, directional, and it survives training. A number behind the metaphor.

The manifold lineage of Riemann, Poincaré, and Chern passed down to Jiang Zehan, Boju Jiang, and Gen-hua Shi. Chern left a question behind — can piecewise-stacked manifolds be extended to arbitrarily complex regions? Two unrelated worlds, rock mechanics and neural networks, later arrived at the same answer.

Sakana AI's DiffusionBlocks trains a network in independent blocks and finds B=3 is the sweet spot. I argue why: the residual stream is already trifurcated into three geometric phases the manifold view predicts — so B=3 partitions a structure that functionally already exists.

Models keep getting smarter while users keep doing more of the work — dragging in files, pasting emails, re-explaining who the client is. The missing piece is not a bigger context window. It is real memory. This is why I built Kocoro, a local, open-source Mac agent that remembers you.

Why training-time loss spikes are not optimization failures but geometric tears — DeepSeek V4 three defenses turn the "manifold prior" from an abstract stance into engineering evidence.

The complete English edition of my Transformer book is now online — 32 chapters and 3 appendices that walk through every component of the Transformer, from tokenization through attention to a from-scratch implementation, and on to RLHF, Mixture-of-Experts, reasoning models, and post-Transformer architectures.

A cultivation-novel reading of deep learning — PDE solvers, manifold geometry, gauge fields, and quantum attention — and why the last realm tells us AGI must be personal.

Separate signal failure from execution and accounting errors with a three-ledger comparison, a worked slippage example, and a reproducible commission-reconciliation test.

An agent turn can span 20 tool calls and 10 minutes. If the daemon dies at minute 9, naive designs throw everything away. A walk through why "just retry the turn" is wrong, the phase state machine that replaced it in Kocoro, and the checkpoint discipline that survives a SIGKILL mid-turn.

Prompt caching offers 90% cost savings when it works and 0% when it silently breaks. Why prompts are fragile in ways unit tests do not normally catch, and the testing discipline we built into Shannon to keep cache hit rates stable as the codebase evolves.

The single most safety-critical operation in any trading system is also one of the hardest correctness problems in execution engineering. A close-call incident, the race condition behind it, and the architectural pattern that replaced naive "close everything" logic with something correct by construction.

I built Kocoro — a Go agent runtime with tool dispatch, permissions, context management, and loop detection — before Claude Code went open source. When their architecture became public, the convergences were striking. Here is what we independently arrived at, what I learned from their caching practices, and what I think defines a production harness.

An Anthropic engineer described Claude Code's evolution from simple todo lists to dependency-aware task graphs — the same patterns Shannon independently arrived at. Five lessons mapped, including where Shannon needs to catch up.

Shannon laid the foundation in 1948. Seventy-eight years later, his framework still explains why Transformers work, what cross-entropy loss actually means, and why your model can never be smarter than its training data.

Why vertical integration — research and execution inside a single unified pipeline — is the real moat in quant trading, and how Dnalyaw builds it with hundreds of features, disciplined risk, and a polyglot Rust/Go/Python execution core across global markets.

A practical guide to building production-grade AI Agent systems. Covers single-agent design, multi-agent orchestration, MCP protocol, Computer Use, cost control, and enterprise deployment patterns.

A deep dive into every component of the Transformer architecture—from Tokenization to Attention mechanisms, from forward propagation to code implementation. For developers who want to truly understand how GPT and ChatGPT work.

Reflecting on how 2025 normalized agent workflows and reasoning models, and why 2026 feels less like a prediction and more like a state you can already opt into.

Bridging the gap between logic and learning—how tensor equations create AI systems that think both symbolically and intuitively.

An architectural deep-dive into Shannon, a self-hosted multi-agent platform that addresses the three hardest problems in production AI: runaway costs, non-deterministic failures, and security vulnerabilities—through deliberate technology choices in Rust, Go, and Python.

Demystifying quantitative trading for AI practitioners — what quant funds actually do, why reinforcement learning fits markets better than LLMs, and where the real barriers lie.

Reflecting on Andrej Karpathy’s 2015 RNN post and the surprising evolution of LLMs with transformers and fine-tuning.

A step-by-step walkthrough of the Transformer architecture: how the original encoder–decoder differs from decoder-only GPT, then input embeddings, positional encoding, self-attention, the FFN and output probabilities.