---
title: Stage 4
type: reference
tags: :passepartout:roadmap:
created: 2026-05-24
---

← [[id:4a1f23b0-abc4-4def-9876-543210abcdef][Stage 3 — Lisp Machine]]  → [[id:4a1f23b0-abc6-4def-9876-543210abcdef][Stage 5 — Weights]]

# Stage 4: Inference

*Summary: The LLM runs in-process under the gate. Every token is inspected
during generation. No external API, no separate trust domain.*

## What is added

- CFFI binding to llama.cpp (or pure-Lisp inference engine) — LLM in the same
  SBCL image as the agent, editor, and gate
- [[id:45ea493b-94ad-5885-aa65-0c846e5c3c1d][Gate-level token interception]]: the Dispatcher inspects every partial token
  sequence during generation. Trajectories that would produce unauthorized
  actions are suppressed mid-stream, not filtered after the fact
- Weights loaded from the Lisp machine's Merkle-verified store as a macro-tag blob (one
  tagged Lisp object pointing to flat binary)
- Deterministic inference: same input, same output, same hash — auditable
  and replayable
- **Distribution match check runs at evaluation loop speed:** the gate's validity predicate for neural network models becomes a fast native function — compute distance to training distribution and compare against threshold in the same process, no IPC

*Two neural components on the same substrate:* Stage 4 hosts the LLM for
generative reasoning and action proposals. The LeCun-type world model (sensory
prediction) joins at Stage 6. Both run on the same tensor unit with the same
plist-native weight architecture and gate-level interception. The LLM proposes
actions to the gate (authorization — binary, deductive). The world model
proposes predictions to the gate (falsification — structural, empirical). The
gate handles both through different procedures.

## What is eliminated

- **LLM as a separate trust domain** — no external server, no API call, no
  cloud provider to breach
- **Remote inference model attacks** — the model cannot be swapped without
  changing the Merkle hash
- **Prompt injection as an action bypass** — the gate sees partial generation.
  A jailbreak that would produce an unauthorized action is caught before the
  tokens complete
- **Model integrity ambiguity** — you know exactly which model is running,
  at which commit, with which hash

## What does this cost?

- **~10x compute, RAM, and storage** — the gate evaluates semantics at every
  token, not just logits. A model running at 50 tok/s natively runs at ~5 tok/s
  under gate-level interception
- **GPU/accelerator constraints** — CFFI to GPU libraries is the bottleneck.
  Exotic [[id:84a537b4-4256-50c8-91f5-dd5b4538418f][hardware]] may not be supported
- **Memory pressure** — the LLM shares the address space with the entire system.
  A 70B param model at 4-bit takes ~35GB. At 10x multiplier, effective
  conventional-equivalent cost is ~350GB
- **Determinism is double-edged** — auditable but cannot adapt or creatively drift
- **No model parallelism** — Passepartout runs on one machine. Frontier-scale models
  may be too large for a single address space

## What does this enable?

Action proposals from the LLM are intercepted by the gate before they reach
the system. The gate authorizes or denies in real time based on policy, not
on trust in the LLM. This eliminates the entire class of "LLM escaped its
sandbox" attacks.

## When is this viable?

- **Server hardware (A100/H100 class, 512GB+ RAM):** today. 5 tok/s on 7B
  models is usable for chat
- **Consumer hardware (RTX 5090 class):** 3-5 years
- **On ASIC tensor unit:** the 10x overhead drops closer to 2-5x because the
  gate runs on the symbolic core in parallel with inference on the tensor unit

## In practice

The LLM is no longer a black box connected by API. The gate watches every
token as it's generated and can stop any generation trajectory that would
produce an unauthorized action. The model is powerful; the gate is in control.

Chat is noticeably slower — roughly 5 tok/s instead of 50 — but the security
guarantee is structural, not probabilistic. For bulk processing or real-time
applications, the 10x tax may be prohibitive without dedicated acceleration.

The weights are still a verified *blob* — you know the file's hash but can't
prove anything about individual weights. Training provenance is not tracked.
The inference is FFI-mediated, so trust in llama.cpp remains.

← [[id:4a1f23b0-abc4-4def-9876-543210abcdef][Stage 3 — Lisp Machine]]  → [[id:4a1f23b0-abc6-4def-9876-543210abcdef][Stage 5 — Weights]]

:PROPERTIES:
:CREATED:  [2026-05-24 Sun]
:WEIGHT:  17
:ID:       4a1f23b0-abc5-4def-9876-543210abcdef
:END:
