A New Paradigm for Multimodal Large Model Training

Core Highlights

The technical team at Xiaohongshu (RED), the popular Chinese lifestyle and social-commerce platform, has recently open-sourced a new training pipeline paradigm called BigMac, which is specifically designed for 'multimodal' large models—that is, models that can see images, hear audio, and understand written text all at once. To put it plainly, BigMac tackles a very practical and long-standing engineering problem that many AI laboratories face: when training a model that looks, listens, and reads simultaneously, how can one make the training process run faster without blowing up the limited GPU memory available on the hardware? This question matters because multimodal training is notoriously memory-hungry, since it must load and activate several different encoders at the same time before any generation can happen. BigMac uses a model pipeline as its backbone and safely 'nests' the computations of the various encoders and the generator into that backbone, achieving meaningful acceleration without disrupting the original execution order that guarantees correct numerical results. For organizations that lack the engineering headcount of top-tier laboratories, a reusable and well-documented solution of this kind is especially valuable because it abstracts away an entire class of difficult systems problems that would otherwise require specialists to solve from scratch.

Specific Capabilities and Event Details

According to the detailed introduction published by the WeChat public account 'Xiaohongshu Technology (dots.llm),' BigMac is a dependency-safe nested pipeline solution whose source code has been made public and is available for anyone to use free of charge. Its core idea is to reorganize the modules that are normally executed serially in multimodal training—such as the vision encoder that processes images, the audio encoder that processes sound, and the language model backbone that handles text—into a single continuous pipeline, so that the computations belonging to different stages can overlap in time rather than waiting for one another to finish completely. In terms of measured results shared by the team, compared with a baseline implementation, BigMac delivers speedups ranging from 1.08x at the low end to 1.9x at the high end, with the exact figure depending on factors such as the specific model architecture and the batch configuration used during training. These gains come not from squeezing a single operator but from better orchestration of the whole training graph.

Technical Details

The key constraint of BigMac is that it keeps 'activation memory bounded,' meaning that while accelerating the training process it strictly controls the memory occupied by the activation values at each stage, avoiding the out-of-memory failures that would otherwise interrupt training and waste expensive compute. In practice, keeping activation memory bounded means engineers can often train larger effective batch sizes or longer sequences on the same hardware, which tends to translate directly into better model quality for a given budget. It accomplishes this by embedding the forward and backward computations of the encoders and the generator into the backbone pipeline, and by carefully scheduling the execution order so that computations overlap without compromising numerical correctness. This dependency-safe design means that it does not require modifying the model architecture itself, and therefore introduces relatively little intrusion into existing training frameworks—a practical advantage for teams that cannot afford to re-engineer their models from the ground up. Keeping memory bounded is especially important in production, where a single crashed job can cost significant time and money.

Comparison with Competitors

Compared with common pipeline parallelism or pure operator-level optimization, the differentiating feature of BigMac is that it is tailor-made for the 'encoding–generation' coupled structure of multimodal models, rather than being a general-purpose large-model parallelism scheme that treats every layer identically. It does not chase extreme peak throughput at the expense of stability; instead, it treats 'stable acceleration under bounded memory' as its primary objective. For a solution that already serves as a core component in the production training of Xiaohongshu's dots multimodal model, this kind of robustness matters more than theoretical peak numbers on paper, because real workloads are messy and unforgiving of fragile optimizations.

Industry Impact and Use Cases

BigMac has already been put into production as the core component for training Xiaohongshu's dots model—the multimodal system that can see, hear, and understand text—which indicates that it has been validated by real business-scale traffic rather than only by benchmark runs. For domestic teams working on multimodal training, the fact that it is open-source and free means they can directly borrow this paradigm to optimize their own training cost and efficiency. Especially against a backdrop of constrained GPU memory and continuously growing model scale, this 'save memory while speeding up' philosophy is likely to be adopted more and more widely, and the release of a working, production-tested implementation lowers the entry barrier for smaller research groups that want to train capable multimodal models without massive hardware budgets. As more Chinese companies choose to publish their internal training tooling, the cumulative effect is a faster-maturing domestic open-source ecosystem for multimodal research, which in turn lowers the cost of experimentation for the whole field.