Cartesia Proposes H-Nets: From Tokens to Bytes via Dynamic Chunking
Cartesia's research team published the first part of a series, "From Tokens to Bytes," proposing Hierarchical Networks (H-Nets) to replace hand-designed BPE tokenization with end-to-end learning straight from raw bytes.
Key highlights
The core thesis: standard language models rely on offline BPE to compress text into fixed-size tokens, and this static tokenization is "one of the last pieces of the pipeline designed by humans rather than learned." H-Nets do not pre-process outside the model; they perform chunking dynamically inside the model, so the model learns how to segment text itself.
What happened
In "controlled scaling ladder" experiments, swapping only the token layer of a BPE Transformer for a 1-stage H-Net (parameters and FLOPs strictly matched) already yielded 1.28x compute efficiency on the validation set; downstream gaps (Paloma/OLMES) widened to about 2x, with 2.04x in-distribution median on code and 5.37x out-of-distribution (e.g., Prolog). At larger scale, a 2-stage H-Net matched open-source SmolLM3 (3B) trained on roughly 11T tokens using only about 1T tokens — about 10x savings in compute and data.
Technical details
H-Nets compress data via recursive, data-dependent dynamic chunking, executed end to end and nested across abstraction levels. Qualitatively, an emergent "in-context tokenization" appeared — adapting at inference to new words, repeated formats, or variable names. Across modalities, H-Net showed 6-7x compute gains on DNA modeling and spontaneously aligned codon/exon structure; in pixel-space diffusion it discovered image semantic segments unsupervised.
Versus rivals
Against BPE Transformers, H-Nets bind to no specific tokenizer and handle any data type including continuous signals, seen as the basis for multimodal "atomic models." It fits the "bitter lesson" — intelligence is learning from raw data end to end.
Industry impact
For model training, H-Nets mean efficient learning even in low-resource domains where humans struggle to hand-design tokenizers (science, biology, robotics); on the application side, lower training compute and data needs make large models more accessible, with stronger out-of-distribution generalization. Our take: this is another research direction, after MoE, that could change the efficiency paradigm of base architectures.
Practical Notes
The reason this matters beyond a benchmark is the "controlled scaling ladder" methodology: by holding parameters and FLOPs fixed and swapping only the tokenization layer, Cartesia isolates the contribution of learned chunking from everything else. That discipline is what makes the 1.28x and 10x figures credible rather than the usual "we trained a bigger model and it got better." For practitioners, the lesson is that the tokenizer — long treated as a fixed preprocessing afterthought — may be one of the largest untapped efficiency levers in the whole stack.
What to Watch
Two caveats. First, 10x savings at the 1T-token scale is promising but still a single reported comparison against one 3B model; the real test is whether H-Nets hold their advantage as models cross into the tens-of-billions and beyond, where chunking dynamics and memory costs change. Second, learned tokenization complicates the tooling ecosystem — every downstream system that assumes stable token IDs (caches, fine-tunes, safety classifiers) has to adapt to a representation that shifts with the data.
Closing
H-Nets are a serious candidate for the "bitter lesson" applied to the tokenizer: stop hand-engineering the unit of computation and let the model learn it from raw bytes. If the efficiency gains replicate at scale, the static BPE layer that every modern LLM inherits could become legacy overnight, and with it a whole category of tokenizer-maintenance work. That is a big "if," but the early numbers are the most interesting architecture result in this cycle.
Why It Matters Now
The tokenizer has been the quiet constant in every modern LLM — a hand-tuned preprocessing step few question. H-Nets challenge that assumption with evidence, not just theory, and the controlled-scaling methodology is what makes the claim credible. If learned chunking can deliver a 1.28x efficiency lift at parity and a tenfold data saving at scale, the economic case for ripping out BPE is suddenly serious. For a field obsessed with bigger clusters, the reminder that a structural change at the input layer can rival a hardware splurge is exactly the kind of lever worth chasing, because it compounds across every model trained afterward.
The Road Ahead
The open questions are scale and ecosystem. A single 3B comparison is a proof of concept, not a guarantee; the true test is whether the gains survive as models cross into the tens of billions, where chunking dynamics and memory costs shift. Equally important, learned tokenization breaks the stable-ID assumptions that caches, fine-tunes, and safety classifiers rely on, so adoption means rebuilding parts of the tooling stack. If those hurdles clear, the static tokenizer could become legacy; if not, H-Nets stay a fascinating result rather than a default. Either way, it is the most interesting architecture story of the cycle. If the efficiency gains hold at scale, the humble tokenizer — not the next big cluster — may turn out to be the cheapest performance lever available to the entire field.