MEMA vs. continual learning: three kinds of AI memory

“Memory” can refer to three different mechanisms in an AI system. If those mechanisms are blurred together, a context window starts to sound like learning, an LLM extraction call starts to sound like model training, and experimental neural adaptation starts to sound like an auditable database. None of those equivalences holds.

For MEMA, the useful boundary is concrete: session/context memory, external explicit records, and model-internal continual learning have different state, evidence, and deletion properties.

Boundary 1: session and context memory

Session memory is the information available to a model during an invocation: the current prompt, earlier messages that still fit, tool results, and retrieved passages inserted into the context. It can make the interaction feel continuous, but the model does not thereby acquire a durable skill or fact.

If a later request omits that information, the model may no longer have it. An application can reconstruct the context from a conversation log or external store, but that persistence belongs to the application. Supplying different tokens changes the input; it does not by itself update the model’s learned parameters.

Boundary 2: MEMA’s external, explicit records

MEMA keeps memory outside the model as explicit records with lifecycle, retrieval, provenance, policy, and audit surfaces. Selected records can be placed into a model’s context, while the source record remains separately addressable and governable. This makes correction, supersession, retention, and deletion questions visible at the data layer.

That visibility has a limit: a hash can show whether bytes differ from the bytes previously hashed. It cannot show that a claim is true. An approval status documents a workflow decision; it does not turn an extraction into ground truth. Integrity is not truth. Evidence quality and policy decisions still need to be evaluated on their own terms.

The current version boundary matters here:

• The publicly inspectable baseline is MEMA package 2.22.13 at commit c37dbf7.
• The internally verified baseline is v2.26.0 at commit d542fba. It is not publicly available.
• The website does not assert production readiness for either baseline.

The MEMA evidence and release-status page keeps public source, internal implementation, experimental work, roadmap items, and production-readiness claims separate.

What /v2/observe actually does

In the public baseline, /v2/observe calls an LLM extractor by default. The automatic path filters the extracted facts and entities, then stores the results with approved status. Separate draft and approval endpoints exist for other untrusted producers.

This is an inference and external-record pipeline, not model-weight training. The extractor produces structured output; MEMA applies application logic and writes records outside the model. The LLM’s weights are not updated by that operation. “Approved” describes how the automatic path stores a filtered result, not an independent factual guarantee.

Boundary 3: model-internal continual learning

Continual learning changes learned state as new data arrives, while trying to preserve useful prior capabilities. That state may live in model parameters, optimizer-like memory, or adaptive neural modules. It is different from adding text to a prompt and different from writing a human-readable external record.

Google Research’s introduction to Nested Learning treats a model as nested or parallel optimization problems, each with its own context flow and update frequency. The accompanying arXiv v1 paper and NeurIPS 2025 publication describe the work. The NeurIPS abstract lists three core contributions: deep optimizers, a self-modifying sequence model, and a continuum memory system. HOPE is the architecture that combines the self-modifying sequence model with the continuum memory system; deep optimizers are a separate contribution, not a component in that combination.

The authors report promising results across language modelling, continual-learning, knowledge-incorporation, and long-context tasks. Google describes HOPE as a proof-of-concept architecture. That is research evidence for the proposed architecture, not evidence that HOPE is a production component of MEMA.

The proposed MEMA–HOPE boundary

A governed MEMA–HOPE integration is planned architecture, not shipped functionality. No HOPE runtime exists in the public 2.22.13/c37dbf7 baseline or the internally verified v2.26.0/d542fba baseline.

The proposal keeps MEMA as the explicit system of record and places a future HOPE module behind a narrow interface:

1. Optional and disabled by default. Enabling a neural continual-learning module would require an explicit deployment decision.
2. Shadow mode initially. The module may generate evaluation output, but it does not influence stored records or retrieval results while the boundary is being tested.
3. Untrusted, bounded output. HOPE may propose candidate records or bounded ranking signals. A score has a capped influence and recorded provenance; it is not an authority decision.
4. No control bypass. HOPE output cannot bypass policy, evidence requirements, approval, erasure handling, or audit. Promotion into an authoritative record remains an explicit MEMA operation.
5. Disposable derived state. Model-internal state does not become human-readable or individually erasable merely because MEMA surrounds it. A future integration needs reset, discard, or rebuild rules for derived neural state and must not claim record-level erasure inside that state.

These constraints are the architecture proposal. Their implementation and operating evidence do not exist in the named MEMA baselines, and production readiness is not asserted.

A practical test for any “memory” claim

Ask four questions before comparing systems:

• Where does the state live: active context, an external record, or learned internal state?
• Does the operation run inference, write data, or update parameters?
• Can a person inspect, correct, supersede, and delete the authoritative record?
• Which version and evidence support the claim?

Those questions keep useful ideas from being oversold. Context can support a session. MEMA can govern explicit records. Continual-learning research can explore adaptive internal state. A future integration may connect them, but it must preserve the boundary between a model signal and an auditable decision.