unified-lm-v1

A single micro language model, trained from scratch with numpy only, that handles six different cognitive operations on the same task. Demonstrates that content-addressable memory removes the retrieval bottleneck but not the working-memory bottleneck, and that format resolution determines measured capability.

Model Description

This model is a proof of concept for two claims from Craft Erasure (Chapters 8, 9):

1. Content-addressable memory and working memory are separate resources. A model with direct memory access succeeds at single-hop retrieval but fails at chained retrieval. Removing the retrieval bottleneck does not remove the manipulation bottleneck.

2. Format resolution determines measured capability. The same underlying problem asked at two resolutions (exact value vs. directional comparison) produces different measured accuracy, and the difference is not symmetric noise — it is a structural property of the resolution.

The model is trained on a synthetic memory-lookup task with six modes selected by a single mode token in the input. It has one set of weights for all modes. It is not pretrained. There is no distillation. It is ~51k parameters trained in under 10 minutes on a laptop CPU using only numpy.

Task

Each input sequence has the form:

[BOS] [STORE p0 v0] [STORE p1 v1] ... [STORE pN vN] [Q pk MODE] ANSWER

Where:

• p_i are position tokens drawn from a pool of 20 distinct positions
• v_i are value tokens drawn from a pool of 10 values (0–9)
• pk is one of the stored positions
• MODE is one of six mode tokens
• ANSWER is the target output

Loss is computed only on ANSWER tokens.

The six modes

Mode	Name	Answer format	Example
E	exact	the stored value token	Vv
D	directional	GT if value ≥ 5, else LT	GT
F	forget	PAD (suppress output)	<pad>
A	attribute	the queried position token	P7
H	hint	bounded three-token pointer	HINT_OPEN Vv HINT_CLOSE
M	multi-hop	chained lookup: value→position→value	Vv2

All six modes share the same transformer body, memory read, and output projection. The only signal distinguishing modes is the mode token.

Content-Addressable Memory Read

At each position t, the model receives an additional input embedding mem_emb[read_t] where read_t is computed deterministically from the input by walking the sequence and tracking (position -> value) pairs as they are STOREd. When the walker encounters Q followed by a position token, it looks up the corresponding value and emits it as the read. For all other positions, read_t = PAD, and the PAD embedding row is fixed at zero.

The read is provided at every position, not only at the answer position. This lets the model read the answer directly from the memory channel without attending backward through the sequence. The store is not learned; the model learns to use it.

Architecture

Component	Value
Parameters	51,524
Layers	3
Attention heads	2
d_model	40
d_ff	80
Vocabulary	43 tokens
Max sequence length	160
Activation	ReLU
Normalization	LayerNorm (pre-norm)
Optimizer	AdamW
Framework	Pure numpy
Training steps	600
Batch size	48

Training Data

Synthetic, generated on the fly. Sequences have 1–6 stored pairs during training. Mode is sampled with probabilities (0.28, 0.20, 0.08, 0.12, 0.15, 0.17) for (E, D, F, A, H, M) — multi-hop is over-represented relative to its difficulty.

Adversarial pairing. Half of the training batches use pairs that share the same value. This forces the model to bind value to position correctly rather than outputting any value that matches the queried position's value.

Results

Per-mode accuracy (n=200 per mode, 6 stores)

Mode	Accuracy
exact	1.000
directional	1.000
forget	1.000
attribute	0.750
hint	1.000
multi-hop	0.250

Length generalization

Trained on sequences up to 6 stores; evaluated up to 20.

Stores	Exact	Directional	Δ (E − D)
2	1.000	1.000	+0.000
4	1.000	1.000	+0.000
6	0.977	1.000	−0.023
8	1.000	1.000	+0.000
10	0.919	1.000	−0.081
12	0.895	0.971	−0.076
14	0.878	1.000	−0.122
16	0.837	0.960	−0.123
18	0.718	0.917	−0.199
20	0.729	0.906	−0.177

Exact accuracy degrades from 1.000 to 0.729 across the range. Directional accuracy is stable. The coarse format survives imprecision that the fine format cannot.

Cross-mode correlation at 12 stores

Metric	Value
Exact accuracy	0.675
Directional accuracy	0.917
Pearson correlation	+0.277

Joint breakdown:

Joint outcome	Fraction
both correct	65.5%
exact only	2.0%
directional only	26.2%
neither	6.2%

The 13:1 asymmetry (directional-only vs. exact-only) is the empirical signature that the two modes measure different capabilities, not the same capability at different precisions.

Intended Uses

• Research on memory architecture: study the difference between content-addressable and attention-based retrieval at minimal scale.
• Format-resolution studies: measure how a model's apparent capability depends on the resolution at which the answer is requested.
• Working-memory bottleneck studies: test which operations are retrieval-bounded and which are manipulation-bounded.
• Education: complete, self-contained demonstration of the claims in Craft Erasure, reproducible from scratch in under 10 minutes.

How to Use

This model is not intended for production. It is a research and education artifact.

python unified_lm_v1.py

The script trains the model from random initialization in ~10 minutes on a CPU and prints the tables above. There is no from_pretrained method. The model is defined and trained entirely within the script.

Limitations

• Tiny scale. 51k parameters.
• Synthetic task. A memory-lookup task. No natural language, no real-world data.
• Deterministic memory. The store is computed from the input; the model learns to use it, not to build one.
• Single training run. No seed variation or hyperparameter sweep.
• Small positions. 20 distinct positions. Length generalization beyond 20 was not tested.
• No production safety. No alignment, no filtering, no refusal. Research artifact only.
• Attribute mode underperforms. 0.750 vs. 1.000 for other modes. Likely a training-distribution issue, not a claim about the mode.

What This Demonstrates

Two claims from the Craft Erasure paper, made measurable in a controlled setting:

1. Content-addressable memory and working memory are separate. Single-hop retrieval reaches 1.000. Multi-hop chaining reaches 0.250. The memory read removes the retrieval bottleneck (single-hop is trivial) but does not remove the manipulation bottleneck (multi-hop fails). This is the working-memory bottleneck in a from-scratch model.

2. Format resolution determines measured capability. At 20 stores, exact accuracy is 0.729 while directional accuracy is 0.906. Same weights, same underlying value, different requested resolution. The correlation between the two formats is +0.277, with a joint breakdown that is sharply asymmetric: 26.2% of problems are directional-correct-exact-wrong, 2.0% the reverse. The two formats measure related but distinct capabilities.

Any model — human or machine — with a fixed precision of internal representation will show the same pattern: coarse formats survive imprecision, fine formats do not.

Citation

@software{unified_lm_v1,
  title = {unified-lm-v1: A From-Scratch Micro Language Model for Format Resolution and Working-Memory Bottleneck Studies},
  author = {zeechimp},
  year = {2026},
  url = {https://huggingface.co/zeechimp/unified-lm-v1}
}

Related Work

• Craft Erasure: How Fields Absorb Techniques and Discard the Minds That Produced Them (in preparation). Chapters 8, 9, and 11 provide the theoretical framework this model tests.
• directional-llm-v1 (companion model). A minimal two-mode model demonstrating format sensitivity.
• content-address-v1 (companion model). A minimal model demonstrating the difference between attention-based and content-addressable retrieval.
• band-edge-detector (companion library). Detects format-resolution compression in benchmark scores.

License

Apache 2.0. Model weights, training script, and documentation are all released under the same license.

Acknowledgments

The architecture is motivated by the observation that human long-term memory is content-addressable and is not bottlenecked by the working-memory slot the way reconstructive retrieval is. This model is a minimal test of that observation, extended to the format-resolution claim from Chapter 9 of the paper.