Mini-AGI: Continual Learning on 8GB VRAM Is Now Possible

Mini-AGI: A Continual Learning Model That Fits on an 8GB GPU

A new open-source project called Mini-AGI is challenging the assumption that meaningful language model training requires data-center-scale hardware. Developed by Alexey Borsky, the system is a byte-level, continual learning model that trains from scratch on a single 8GB VRAM GPU—like an RTX 3070 Laptop—and keeps learning from every piece of text it reads, without catastrophic forgetting.

The project, which was posted to Hacker News and quickly gathered 181 points in 8 hours, is not just a toy. It showcases a sophisticated architecture that pages weights between disk and VRAM, dynamically assembles its own structure, and self-regulates its learning rate. The core claim: a truly personal, continually learning AI is possible on hardware that most developers already own.

How Mini-AGI Works

Mini-AGI operates on bytes rather than tokens, using a vocabulary of 256 byte values plus 9 structural markers. This eliminates the need for a tokenizer and allows the model to read any file type directly. The architecture processes each character through two dense prelude blocks and then a recurrent block applied up to 24 times, with each application selecting its own top-8 experts from a shared pool.

This design is built on three key mechanisms:

• Adaptive Depth (PonderNet): A halting head scores each character at every layer and stops when further computation won't change the output. Easy characters take one row, hard ones take many. This dynamic depth allows the model to allocate compute efficiently.
• Dynamic Expert Routing: Each block-application picks its own top-8 experts from a pool that grows and prunes itself. No expert is assigned a subject; routing distributes capability organically, enabling the model to combine fragments from multiple experts for a single prediction.
• Disk Paging: Weights are stored as files on disk, with only a working set of 32 experts (about 109M parameters) resident in VRAM at any time. The parameter count is bounded by disk space, not VRAM, allowing the model to scale well beyond the 540M parameters it currently has.

Solving Catastrophic Forgetting

The most significant technical achievement is addressing catastrophic forgetting—the tendency of neural networks to overwrite previously learned knowledge when trained on new data. Borsky's experiments show that naive training on a single subject (chess) for 524,000 characters degrades performance on other subjects by +2.23 nats, retaining only 50.68% of prior knowledge.

The fix was surprisingly simple: run the trunk learning rate at 0.1x the expert rate. This single change reduced forgetting to +0.0067 nats, retaining 99.84% of knowledge. The trunk—embeddings, attention, routers, and halting head—carries 97.6% of the squared gradient norm, so slowing it down prevents the shared components from being overwritten.

The project also revealed a counterintuitive insight: the expert pool itself is not what prevents forgetting. Even when 93 of 136 experts were frozen, the model still collapsed. The damage is displacement, not destruction—knowledge that appears lost can be recovered 380x faster on re-reading, suggesting the weights retain information that routing simply can't access.

Self-Assembling Architecture and Growth

Mini-AGI doesn't have a fixed architecture. It starts small and grows into its shape. The expert pool begins with a small number of experts and adds new ones when all five "brakes" agree: room, used, earning, fits, and honest. New experts are created by recombination—taking hidden units from existing experts—because clones of a single parent aren't novel enough, and random experts are useless.

Pruning is equally clever. The system tracks how long it's been since an expert was selected, not its gate value. Borsky found the gate is "anti-predictive": the smallest gates belong to the busiest experts, while high-gate experts can be dead for hundreds of thousands of segments. This insight, which took a week to debug, is one of the project's most valuable findings.

Benchmarks and Scaling

At 318.1M characters read, the model achieves a held-out loss of 0.8336 nats/char (1.2026 bits/byte) across eight subjects. The best-performing subject is chess (0.552 nats), while Wikipedia remains the hardest (1.280 nats). The model's data scaling follows a clean power law with exponent -0.239, between Kaplan's 0.095 and Chinchilla's 0.28, with R² of 0.96.

Compared to MambaByte-353M, which read 94x more data, Mini-AGI shows promising scaling. The fitted trend suggests reaching 1.00 BPB on its own mixture would require 0.75B bytes—about 6 days of training on a laptop GPU. This puts frontier-adjacent performance within reach of hobbyists.

Practical Implications

Mini-AGI's implications extend beyond technical novelty. It represents a shift toward truly personal AI—models that train on your data, on your hardware, and keep learning from every conversation. The project includes a simple command to read your own files, with a dry-run mode by default and held-out scoring to verify no knowledge is lost.

The author acknowledges the model is currently "toy-level" and not competitive with frontier models. But the point is different: it proves that continual learning from a single stream of data is possible on modest hardware. As Borsky writes, "the capabilities would be bounded by the actual hardware, scale and quality of the data available and the amount of time one willing to spend on training the model."

Getting Started

The project is fully open-source under the MIT license. To run it, you need a CUDA-capable GPU with 8GB VRAM and Python 3.10+. The setup involves cloning the repo, installing dependencies, and building the corpus (or pointing it at your own files). The weights are not yet published—the current run is still on its first pass over the corpus—but the code is ready for experimentation.

For researchers and hobbyists alike, Mini-AGI offers a unique window into dynamic architecture, continual learning, and the practical limits of consumer hardware. It's a reminder that the future of AI isn't just about scale—it's about adaptability.