MemTensor Open-Sources MemT-1.0: The Memory-Native Model That Remembers in Its Forward Pass

On October 20, the Chinese team MemTensor （记忆张量） officially released and open-sourced MemT-1.0: a "memory-native" family of memory foundation models. The official repository describes itself in one dense sentence: "MemT-1.0 is a new generation of memory foundation models that makes persistent, dynamically evolving memory an intrinsic part of model computation."

In plain terms: previous models had goldfish brains — every conversation started from zero, and "memory" was always a bolt-on (cramming chat history into context, or RAG retrieval). MemT-1.0 builds memory into the forward pass itself: with every token it generates, the model reads and writes a fixed-size internal memory state. This isn't giving the model a notebook — it's writing "the ability to remember" into the model's bones.

A note on evidence caliber before we go further, because it determines how you should read every number below: architecture, parameters, training data, and licensing come from first-hand reading of the official GitHub README and the technical report (arXiv:2610.26825, submitted October 19). Figures like the SWE-bench 66.07%→69.73% and the 3.99× speedup at 256K are the team's official release figures as relayed same-day by multiple outlets (The Paper, Global Times Online, Shanghai Observer, NetEase Intelligence) — consistent across outlets, but the corresponding tables in the full technical report have not yet been independently verified. Each such figure is asterisked in the text.

The architecture: memory attention running alongside backbone attention

At MemT-1.0's core sits a Native Memory Block that runs alongside the backbone network's attention. During generation, memory attention reads from the current memory state, normalizes and gates the readout, and adds it to the backbone attention output; at the end of each turn (or segment), GDN-2 (Gated DeltaNet-2) performs stable token-level memory updates — erase, write, and forget — folding new information into existing memory.

Inside the memory block are two layers: the Local Memory Block maintains a dynamic multi-channel memory matrix that evolves across interaction steps, while the Hyper Memory Block carries trainable memory query/key/value projections plus erase, write, forget, and output gates. Capacity scaling comes from Mixture-of-Storage (MoS): the memory module expands into multiple "memory experts" with dense routing — the base MoE handles compute, MoS handles where and how things are remembered.

Training ran in two stages over 34 billion tokens: continued pre-training to teach the model to "predict the future from the past," then post-training with a "top-student" teacher model scoring memory extraction, using the error signal to force improvement. In the team's words, the judgment of what to keep and what to forget isn't innate — it's drilled in with 34B tokens.

Two models shipped: MemT-1.0-Nano (Qwen3.6-35B-A3B backbone; 36B total / 3B activated / 451M trainable memory parameters / 10M dynamic memory state) and MemT-1.0-Flash (DeepSeek-V4-Flash backbone; 288B / 17B / 3.61B / 336M). Note the pairing: Nano on a Qwen-family base, Flash on a DeepSeek-family base — domestic bases plus a domestic memory architecture, a fully "homegrown" stack.

The evaluation: what the team claims, and what it volunteered

First, the first-hand-confirmed part. The official README states that across 11 memory benchmarks, Flash and Nano rank first and second "among the evaluated parametric-memory methods." That qualifier matters — and the team immediately follows it with a refreshingly honest sentence: "Strong textual-memory systems remain ahead overall, particularly on long histories that require detailed evidence retrieval." The team itself admits: for fine-grained retrieval, bolt-on textual memory still wins. "Reporting the good with the bad" is rare in official READMEs, and it deserves credit.

Efficiency numbers confirmed first-hand from the technical report abstract: at 128K history length, MemT-1.0-Nano achieves roughly 1.86× end-to-end inference speedup versus full-context KV cache, with persistent storage reduced roughly 66×. That's the structural payoff of a fixed-size memory state: however long the history grows, the memory state stays the same size instead of ballooning linearly like a KV cache.

The following figures are the team's official release numbers as relayed same-day by multiple media outlets* (fully consistent across outlets; the full report's tables not yet independently verified): on 11 memory benchmarks, Flash averaged 50.6 and Nano 47.0; on SWE-bench Verified (500 tasks), Nano's average success rate rose from 66.07% to 69.73%*, with the share of tasks solved on all three attempts climbing from 54.00% to 62.80%; SWE-bench Multilingual rose from 60.00% to 63.56%*; at 256K history, end-to-end speedup reached roughly 3.99×*, with first-token latency of 104–112ms (versus ~445ms for full context at 256K); per-session memory state at 256K history was about 40 MiB versus 5.06 GiB for the conventional approach — roughly a 130× shrink*.

Two qualifiers matter when reading these. First, the SWE-bench gain comes from the "Experience-Guided Agentic Coding" framework: the execution harness triggers experience writes at key breakpoints, retrieved procedural experience is stored in native memory and invoked via memory attention during execution — without stuffing experience text into the task context. In other words, those +3.66 points measure "reusing historical problem-solving experience," not a smarter model per se. Second, baseline and experimental runs used the same execution harness, so the controls look clean; and the "all-three-attempts 54%→62.8%" metric suggests memory's bigger value may be stability — not flaking on repeated runs, which matters more than a single high score in real engineering.

Why "memory-native" may be a big direction

The judgment first: memory is the #1 pain point of coding agents, and the "parametric memory vs. bolt-on retrieval" architecture debate may be one of the most important of late 2026.

Vibe coders live this pain daily: open a new session and the agent forgets all your project conventions; stuff history into the context window and the token bill explodes while the actually important information drowns in small talk. Every current fix is a bolt-on — RAG retrieval, memory banks, re-feeding summarized history. They share one ceiling: memory and reasoning are two separate systems, and retrieved content must be "translated" for the model to see, losing information and burning tokens at every step.

MemT-1.0 bets the other way: make memory part of the forward pass, so retrieval and reasoning complete inside the same computation graph. The technical report includes a hardcore "closed-book" test* (via media accounts of the report): a user gives an exhibition's original date, reschedules to Sunday, casually mentions an unadopted Monday alternative — then 500+ turns of small talk pile up ~66K tokens of history. Traditional bolt-on retrieval crumbles on such multi-version, long-span information, yet MemT-1.0, with zero access to the original text, reported the original and rescheduled dates accurately, confirmed venue/price/contact were unchanged — cleanly separating "the Sunday verdict" from "the Monday alternative." Another detail: asked about a birthday never provided in 16K tokens of history, the model honestly said it didn't know — no confabulation. Knowing what you know is precisely the line between "memory" and "hallucination."

Of course, the team itself concedes textual-memory systems still lead on detailed retrieval. So the sharper framing is: MemT-1.0 isn't a RAG replacement — it's an "experience memory layer" for high-frequency, structured, evolvable knowledge. What belongs in parametric memory (your coding habits, a project's implicit conventions, recurring solution patterns) versus bolt-on retrieval (exact API docs, a full error stack trace) — that layering is exactly the architecture lesson vibe coders will now practice by hand.

The license: a necessary cold shower

Chinese press releases all say "released and open-sourced," but the README's license section deserves a verbatim read. The repository software uses the PolyForm Noncommercial License 1.0.0 — commercial use is not permitted; commercial licensing requires contacting the authors. The technical report is CC BY-NC-SA 4.0. So this is "visible source, free to play with, not for commercial use" — not open source in the OSI sense.

This isn't nitpicking; it directly determines what you can do with it: personal learning, demos, experiments — go wild; but integrating it into a product you charge for means negotiating a commercial license first. Also note: model weights, datasets, and benchmark assets are "unless explicitly stated otherwise… not covered by the licenses above" — check the license card on the Hugging Face page after downloading weights; don't assume it matches the code license.

One more footnote: the contact email is lizy@memtensor.cn, and the 74-author technical report is signed "MemTensor Team," led by Zhiyu Li — the same person behind the earlier MemOS project. From MemOS (a memory operating system: "how memory is scheduled") to MemT-1.0 (a memory-native model: "how memory is internalized"), the product line is one continuous thread: first the bolt-on memory OS, then memory baked into the weights. That lineage also prevents confusion — one is a framework, the other is a model; one schedules, the other internalizes. (And for the record: PKU/NUS's Mem-T paper and the yanweiyue/Mem-T repository are a different team's memory-agent project, unrelated to MemTensor — don't let the names tangle you.)

How to play with it: Nano is 36B, genuinely within reach

The friendliest news for vibe coders: Nano is only 36B total parameters with 3B activated — within reach of a single machine or small cluster today. The official stack is SGLang-based: install the server from the inference directory, connect the memt_sglang client, and you're having "conversations with memory"; save_state() persists the memory state as a snapshot file, restorable later via state= — memory becomes, for the first time, a file you can save to disk and version-manage, not a black box locked in some SaaS account.

Three experiments worth trying: first, a simplified version of that "500-turn exhibition reschedule" closed-book test — verify by hand whether fixed-size memory really doesn't drop details; second, compare outputs on the same task with memory on versus off (enable_memory=False is plain chat mode) and see where experience reuse actually shows up; third, measure first-token latency on long histories — if that 104ms figure reproduces, it's a material win for "always-on personal agent" product shapes.

But manage expectations: 186 stars at press time, 2 commits — the ecosystem is just starting; Flash's 288B needs a serious cluster; every asterisked number awaits the full technical report and third-party replication. The right posture now: treat Nano as a study specimen and "memory-native" as an architecture course — and let time decide whether it becomes the standard base for the next generation of agents.

Closing judgment

What truly matters about MemT-1.0 isn't any single benchmark score — it's that the release turns an argument from hot air into downloadable weights: should memory be a bolt-on, or part of the model itself? The team backs the "native" camp with 34B tokens of training, 66× storage reduction, and +3.66 points on SWE-bench* — while openly admitting the bolt-on camp still leads on detailed retrieval. That "radicalism with a sense of its own boundaries" reads more credible than a page full of SOTAs.

For vibe coders, three takeaways are actionable now: internalize the "parametric memory vs. bolt-on retrieval" layering (what gets baked into weights vs. what goes through retrieval); watch engineering patterns like save_state that make "memory persistable to disk" — it may reshape personal agents' data model; and remember the PolyForm Noncommercial line — play freely, license before commercializing. The long-term question belongs to time: when memory truly becomes part of the forward pass, will "tuning an agent" shift from "writing better prompts" to "raising better memories"?

Transparency note: architecture, model parameters, training data, licensing, and the 1.86×/66× at 128K come from first-hand reading of the official GitHub README and the arXiv:2610.26825 abstract (repository created 2026-10-19, inside the window); asterisked figures (SWE-bench 66.07%→69.73%, Multilingual 60.00%→63.56%, 3.99× at 256K, 40 MiB vs 5.06 GiB, the 500-turn closed-book test details) are the team's official release figures relayed same-day by multiple outlets (The Paper/Global Times Online/Shanghai Observer/NetEase Intelligence), consistent across outlets, with the full report's tables not yet independently verified; the Hugging Face weight collection is the official link as relayed by media (huggingface.co/collections/MemTensor/memt-1), with the README carrying an official Hugging Face badge link.