Research

Inside the mind

A language model is not a mystery — it is a very long arithmetic expression that nobody bothered to annotate. Below is the annotation: four stages, scrolled through, from raw characters to a chosen word.

How Noema processes a prompt

Characters → subword units

  1. Stage 01

    The prompt is broken into pieces that are not words

    Noema never sees letters. Text is cut into subword tokens — fragments chosen because they are statistically cheap to encode. “interpretability” arrives as four pieces, “the” as one. Everything downstream is arithmetic on those pieces.

    Vocabulary: 128,000 tokens · avg 3.7 chars

  2. Stage 02

    Each piece becomes a direction in a very large space

    Every token is looked up as a vector — a list of numbers describing a direction. Related meanings point similar ways. This is the first place the model has anything you could call an idea, and the first place we can measure one.

    Embedding width: 12,288 dimensions

  3. Stage 03

    Every token asks every other token what it means

    Attention is a weighted lookup: each position queries the whole sequence and pulls back a mixture of what it finds. Run it 96 times in parallel, 120 layers deep, and structure appears — coreference, syntax, arithmetic carry. These are the heads we name.

    120 layers × 96 heads = 11,520 circuits

  4. Stage 04

    The result is a probability over everything sayable

    The final vector is projected back onto the vocabulary, producing a score for all 128,000 tokens. One is chosen. Then the whole machine runs again for the next word — and the derivation trace records which heads moved the decision.

    Decode: predicate-checked, cited on refusal

Programmes

Four standing questions.

Each programme is expected to publish negative results. A year without a retraction is a year we were not looking hard enough.

Programme A

Circuit stability

Do the mechanisms we identify survive fine-tuning, quantisation and distillation? If an explanation evaporates under 4-bit weights, it was never an explanation.

Programme B

Auditable refusal

Behaviour specified as predicates over the derivation, evaluated at decode time. The model cites the clause it invoked, and the clause is public.

Programme C

Explicit forgetting

At a million tokens, eviction is inevitable. We treat it as an interface problem: the model reports what left the context and what it cost.

Programme D

Legibility under pressure

Explanations that hold on benchmarks but dissolve under adversarial prompting are decorative. We attack our own traces.

Publications

Everything we have said out loud.

7 of 7 shown

Try it yourself.

The playground runs the same visualisations against text you type.

Open the playground