Homelab Runs 2.8 Trillion Parameter Model on CPU with Intel Optane Memory

Why Intel Optane Persistent Memory Matters for Large Models

Traditional server memory, such as DDR4, reaches its limits when a model requires more than a few hundred gigabytes of RAM. Intel’s Optane Persistent Memory (PMem) offers a larger addressable space while keeping latency lower than typical storage solutions. The second generation of this technology expands capacity to 512 GB per module and improves bandwidth, making it a viable bridge between DRAM and SSDs for data‑intensive workloads.

Architecture of second generation Optane

Second generation Optane modules are built on 3D XPoint technology, which stores data in a non‑volatile matrix. This allows the memory to retain information after power loss, a feature that traditional DRAM does not provide. The modules can be configured in Memory Mode where the operating system sees a single large pool of memory, or in App Direct Mode where applications can address the persistent region directly.

For workloads that exceed DRAM capacity, Memory Mode simplifies programming because the operating system handles paging automatically. In App Direct Mode, developers gain finer control over data placement, which can be crucial for optimizing inference of massive language models.

Running a 2.8 trillion parameter model on a CPU

In a recent Reddit post, a homelab enthusiast described how they loaded Moonshot’s 2.8 trillion parameter Kimi K3 model onto a server equipped with Intel Optane PMem 200. The system relied solely on a mainstream CPU, without any graphics accelerator, and used a one bit quantization scheme to reduce the model’s memory footprint.

Quantization to one bit

Quantization compresses the numerical precision of model weights. A one bit approach stores each weight as a single binary value, effectively halving the storage requirement compared to traditional eight‑bit formats. While this reduces accuracy slightly, it enables inference of models that would otherwise be impossible on limited hardware.

Open‑source libraries such as BitNet provide tools for applying extreme quantization, and the Reddit user leveraged similar techniques to fit the 2.8 trillion parameter model into the available memory pool.

Performance metrics

The configuration achieved roughly 0.85 tokens per second during generation. Although this speed is far slower than GPU‑accelerated setups, it demonstrates that CPU‑only inference is technically feasible for research and hobbyist purposes. The key metric is not raw speed but the ability to run a model of unprecedented scale without specialized hardware.

Memory usage: approximately 1.2 TB total, split between DRAM and Optane PMem.
Quantization level: one bit per weight.
Inference speed: 0.85 tokens per second.
Hardware: mainstream x86 CPU, 256 GB DDR4, two 512 GB Optane PMem 200 modules.

Implications for homelab enthusiasts

Running a model of this magnitude on commodity hardware reshapes what is possible in a home lab. The cost barrier of high‑end GPUs is avoided, and the reliance on persistent memory opens new avenues for long‑running experiments that need to survive power cycles.

Cost considerations

While Optane PMem is more expensive per gigabyte than standard DRAM, the total cost can still be lower than a multi‑GPU rig capable of holding the same model in memory. A typical configuration may cost between $5,000 and $8,000, including the CPU, motherboard, and memory modules. For hobbyists who already own a server chassis, the incremental expense is primarily the Optane modules.

Future directions

As Intel continues to improve Optane technology, bandwidth and latency gaps with DRAM are expected to narrow. Combined with advances in quantization algorithms, future homelabs could run even larger models at higher speeds without needing graphics processors.

Researchers are also exploring mixed‑precision strategies that keep critical layers in higher precision while applying aggressive quantization elsewhere. Such hybrid approaches could further balance accuracy and performance on CPU‑only platforms.

For those interested in replicating the experiment, the following resources provide deeper insight:

The experiment highlights a shift toward memory‑centric design for deep learning workloads. By leveraging persistent memory, developers can bypass the traditional bottleneck of limited DRAM, opening the door to new research possibilities on modest hardware.