More than you ever wanted to know about Engrams in DeepSeek 4.1

1. Where Engram sits in the model

V4.1 Flash is a stack of 40 blocks—model layers, nothing to do with SSD blocks. In each one, attention pulls in context from earlier tokens and a mixture of experts transforms the result; both are added back to the working state. Engram adds a third contribution, but only in two blocks, just before attention.

The lookup works on token IDs, not text. The engine maps the current token and the three before it into a compressed vocabulary, so some spelling variants share an ID. It hashes the last two, three and four tokens with eight hash heads each: 24 row addresses per table, 48 per token. Each row holds 256 eight-bit values and eight one-byte scales, 264 bytes in all.

The rows become four keys—one per working stream—and one shared value. Each stream compares its state with its key, and the score becomes a gate between zero and one. It’s learned memory inside the forward pass, closer to an embedding table than to retrieval-augmented generation; nothing is inserted into the prompt. Image-placeholder positions are masked, and the speculative decoder’s draft model doesn’t use Engram at all.

2. One token, from lookup keys to output

The row addresses depend only on token IDs, so the CPU can compute them and start reading before the GPU reaches block 1. With the asynchronous reader, block 0 hides the Table 0 reads, and blocks 1 to 13 hide Table 1. macOS works differently—open the last drill-down or see chapter 3.

3. The timing model

The asynchronous token path uses TOKEN_READERS = 4. Each worker reads six rows per table in sequence, so the per-row budget is the table’s window divided by six. At 40 tokens/s Table 0 has about 625 µs, roughly 104 µs per row. Under four-reader load the median row read took 68 µs on the internal SSD and 99 µs on the external one—both inside the budget. At 100 tokens/s the budget drops to 42 µs and both miss.

With those medians, the internal drive keeps up to about 61 tokens/s and the external one to about 42. That’s arithmetic on a median; the tail is in chapter 4.

A late table costs its overshoot

Missing a window isn’t a cliff. A late fetch adds only the time by which it overshoots; the blocks already computed stay computed.

At 100 tokens/s that overshoot is 0.34 ms on a 10 ms token: 3.4% more time, about 96.7 tokens/s. The same 100 µs overshoot costs 0.4% of a 25 ms token and 6% of a 1.67 ms one—the faster the GPU, the less time storage has to hide. At a hypothetical 600 tokens/s even Table 1 misses with 0.59 ms fetches (442 tokens/s), and a 2.4 ms fetch, roughly one reader working through all 24 rows, leaves about 170.

w₀ = W / 40
w₁ = 13W / 40

extra = max(0, f₀ − w₀)
      + max(0, f₁ − w₁)

T = W + extra  (ms)
slowdown = extra / W
tokens/s = 1000 / T

264 bytes in, kilobytes off the flash

The logical payload is tiny: 1.27 MB/s at 100 tokens/s. The drive moves more. If every row read pulls a 4 KiB page, 600 tokens/s means 28,800 reads/s and about 118 MB/s; with 16 KiB pages it’s about 472 MB/s. Either way, for one stream random-read latency matters and an SSD’s sequential GB/s rating says little.

4. Inside the SSD experiments

I ran the disk tests on an Apple M5 Ultra with 256 GiB RAM under macOS, using Python’s os.pread with F_NOCACHE to keep the file cache out. The internal SSD read a 2 GiB zero-filled scratch file; the external Samsung 4 TB volume read an actual 94.4 GiB Engram table. One reader, then four, 264 bytes per read, about four seconds each. The harness times single reads, never a whole 24-row lookup—hence the derived 6 × 68 µs = 0.41 ms and 6 × 98.6 µs = 0.59 ms per table.

One read in a thousand takes 2 ms

On the external drive with four readers, the median row read was 98.6 µs, but the p99.9 was 2.05 ms and the slowest read took 36.8 ms. One read like that stalls its whole chain, and the fetch waits for the slowest of the four chains. Using the 121.4 µs mean instead of the median gives 0.73 ms per table and 4.8% extra token time at 100 tokens/s—and even that is a floor, because the slowest of four chains takes at least six times the mean on average.

Smaller quants buy space, not speed

Fewer bits per value look like an easy win. A 4-bit row would be 136 bytes instead of 264, and both tables would shrink from 188.8 to about 97 GiB. But the SSD doesn’t care: on the internal drive a 264-byte read had a 64.25 µs median and a 4 KiB read 68.38 µs; on the external drive 98.17 and 99.79 µs. Fifteen times the data barely moved the access time, because the drive fetches a whole page either way.

So a harder quant only pays if it changes where a table lives—if a 4-bit Table 0 at about 49 GiB fits in RAM where the 8-bit one at 94 GiB doesn’t. On the same SSD it saves no time and costs quality: every lookup, at every token, gets coarser values. I haven’t measured how much a 4-bit Engram table hurts output, but it’s a trade with nothing on the other side. If disk space is the constraint, buy a bigger SSD. The same logic runs the other way: a 512-byte 16-bit row is still far smaller than a 4 KiB page, so keeping more precision on SSD would cost disk space, not read time.

More readers, more drives?

A second harness ran 8 to 48 concurrent readers on each drive and split across both. With eight readers, splitting four per drive cut the per-read p99 from about 265 µs to about 150 µs, while the median got slightly worse (86 → 93 µs). At 24 readers the ranking reshuffled between repeat runs, and at 48 the median rose to about 235 µs: more outstanding reads mostly mean more queueing. A second drive may help the tail and can keep Engram away from expert-weight streaming. This was a two-drive split, though, and a RAID 0 array might behave differently.

Short runs, different file sizes, sustained load and hosts that weren’t fully idle limit how far this generalises, and the same drive may behave differently with another OS or I/O stack. The next useful measurement is a real token-paced fetch alongside the actual competing workload.

5. Remote RAM, Thunderbolt and Ethernet

Memory disaggregation sounds grand; here it just means putting a table in another machine’s RAM and asking it for rows. My test used a small C daemon over TCP, serving a synthetic 4 GiB buffer with one request in flight—transport plus service time, outside any inference engine.

Over a 20 Gb/s Thunderbolt link (TCP, no RDMA), one 24-row batch took 92–99 µs, with a p99 of 139–158 µs; a 48-row batch for both tables took about 100 µs. If each worker instead made six round trips of about 90 µs, a table would take about 0.54 ms—roughly what the external SSD manages. The win comes from one message per table, and the engine has to implement that. With it, Table 0 makes its window up to about 250–270 tokens/s in the model.

The memory still has to exist somewhere

The donor has to keep the whole table resident, plus room for the OS and the service: 128 GB is about 119.2 GiB, which leaves about 24.8 GiB once a 94.4 GiB table is loaded. You also need access control, matching table versions, timeouts, monitoring and a fallback for when the peer disappears. Remote RAM swaps a capacity problem for a service dependency—so if Table 0 fits in local RAM, put it there.

6. Nine storage paths, compared

Four of these paths are measured or derived; the rest are estimates and assumptions to see what happens as latency grows. “External SSD” means the drive I tested.

What that does to decode speed

The chapter 3 formula, applied to every path at baseline speeds from 25 to 300 tokens/s.

Scroll sideways for all nine paths; the first column stays put.

• No stall
• Some: above 0%, below 30%
• High: 30% or more

At 100 tokens/s the internal SSD gives about 98 tokens/s and the external one about 97. A 4 ms Wi-Fi round trip drops it to about 69, because it misses both windows. A constant 1.5 ms fetch fits Table 1’s window up to about 217 tokens/s; the 30 ms spinning disk misses it at every speed. The heatmap holds latency constant—the spread from chapter 4 comes on top.

7. What to build, and what to measure next

1. Place the urgent table first. Local RAM if it fits. Otherwise compare a measured SSD path with a properly batched remote-RAM setup. Capacity and sequential throughput are the wrong ranking.
2. Protect the read window. Expert streaming, other requests and background I/O all compete for the same drive. A measurement on an idle device says little about a shared production volume.
3. Measure the whole wait. Record Table 0 and Table 1 completion times, token stalls and tail percentiles. The slowest of four chains matters more than the median row.
4. Test degraded operation. A remote donor can restart, swap or disappear. Decide whether to fall back, retry or reject the request, and measure what the user sees.
5. Keep capacity and speed separate. These tables are only one part of model memory. Budget the other weights, attention state, runtime buffers and concurrency headroom too.
6. Don’t trade quality for disk space. On an SSD a smaller row reads no faster. Quantise the tables harder only if it moves Table 0 into RAM; otherwise keep the precision.

Prefill, speculation and batches change the workload

Prefill hashes and reads many prompt positions in one wide sweep. A speculative verification pass with k draft tokens covers k+1 positions and requests up to (k+1) × 48 rows; a serving batch multiplies that again. Draft tokens are known before verification, so an engine could prefetch their rows while drafting—an idea I haven’t measured.

The two measurements I’d do next: a token-paced trace of DwarfStar’s asynchronous reader, with expert streaming running alongside, and the synchronous F on a Mac, since that’s the time every token pays there today.

Sources, data and limits

Architecture and reader scheduling refer to DwarfStar at commit ee7e840, an inference engine for DeepSeek V4.1 Flash. See also the model configuration and DwarfStar model card; those two links point to main branches and may change.

The measurements come from my standalone storage harnesses, run on 25 October 2026. You can download the recorded SSD and split-disk measurements at their original precision, with local paths removed; the network samples aren’t included.