passepartout/docs/ROADMAP.org

#+TITLE: Passepartout Evolutionary Roadmap
#+STARTUP: content
#+FILETAGS: :docs:roadmap:

* The Evolutionary Roadmap

Understanding Passepartout as a function in time is not nostalgia. It is architectural guidance. Every decision in v0.x should be made with awareness of where the system is going. Code written today becomes the substrate for v3.0. Skills designed today become the vocabulary the symbolic engine speaks tomorrow.

The probabilistic beginning is not a weakness to overcome. It is the bootstrap. The system learns the domain through probabilistic inference, and that learned knowledge becomes the seed for the symbolic engine. By the time the symbolic engine takes over, it has a rich knowledge graph to reason about, grown from thousands of probabilistic interactions.

This is how you build a reasoning machine: start with a learner, make it learn to verify by watching itself and its user, let verification become the core. Every blocked action becomes a rule. Every approved exception becomes a pattern. The symbolic layer grows at the probabilistic layer's expense. Remove the learner once it has learned enough. 

Each version expands the deterministic layer. The Dispatcher writes rules from approved exceptions. Shadow mode runs trial executions. Tool permission tiers mature from simple allow/deny to nuanced context-aware policies. The agent becomes less likely to attempt dangerous actions not because it is smarter but because the guard has more complete information.

The roadmap is designed working backwards from SOTA parity (v1.0.0), guiding each version toward a fully autonomous, self-editing agent. Each version builds on the previous, with features designed to be implemented in pure Common Lisp + Org-mode.

The TODO states in each version's Tasks section are the authoritative task tracker. The feature tables describe what each version delivers.

Feature releases increment the minor version (v0.X.0). Bugfix and hardening releases increment the patch version (v0.X.Y). This ensures that security patches and critical fixes are visible in the version number and can ship independently of feature work. No feature release ships without its prerequisite hardening releases resolved.

** File Update Checklist

When a version's state changes (DONE → tested → released), update these locations:

1. ~ROADMAP.org~ — mark item DONE, update LOGBOOK timestamp
2. ~README.org~ — update version badge (line 6), update Current Capabilities table (add new Stable rows for shipped features, remove Planned rows that have shipped)
3. ~~.env.example~ — update version references as needed
4. ~lisp/core-transport.lisp~ — update the ~make-hello-message~ version string
5. ~passepartout~ (bash entry point) — update version reference

On release:
1. Tag the release on GitHub
2. Extract DONE items from ROADMAP (all items with LOGBOOK timestamps since the last release tag) and use as the release notes body
3. If a ~CHANGELOG.md~ is needed for packaging tools, auto-generate it from ROADMAP DONE items

** v0.1.0: The Autonomous Foundation — RELEASED 2026-04-20
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon 19:05]
:END:

The secure, auditable Lisp kernel. All core infrastructure in place.

*** DONE Perceive-Reason-Act pipeline
:PROPERTIES:
:ID:       id-06f10b9a-4054-4dea-a927-b0935fbdcd2f
:CREATED:  [2026-03-22 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

This established the three-stage cognitive cycle that all later features plug into. The pipeline is the invariant — skills, gates, actuators, and clients all compose through it.

*** DONE Skills engine with jailed loading
:PROPERTIES:
:ID:       id-dc83944f-3923-4142-b324-c317dacd6b0b
:CREATED:  [2026-03-22 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

This made the "thin harness, fat skills" identity operational. Skills loading into jailed packages (v0.1.0) is the foundation for the skill sandbox mode (v0.3.2) and the Skill Creator (v0.8.0).

*** DONE Policy skill (6 invariants)
:PROPERTIES:
:ID:       id-929c84b7-d6ae-42b9-a8b5-d9df962db826
:CREATED:  [2026-03-22 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

This established the "explanation required" invariant that gates stack above. The policy gate (priority 500) runs first and sets the precedent that every action must justify itself.

*** DONE Memory (memory-object + Merkle hashing)
:PROPERTIES:
:ID:       id-3a96b384-cacf-4da0-8faa-1647739feba9
:CREATED:  [2026-03-22 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

The Merkle tree with content-addressed hashing made copy-on-write snapshots (v0.2.0) and MVCC concurrency (v0.6.1) possible. The hash-as-identity property also feeds directly into the foveal-peripheral model's semantic retrieval.

*** DONE Scribe + Gardener background workers
:PROPERTIES:
:ID:       id-3f618a38-ec23-4034-ba3c-ef272e212e2b
:CREATED:  [2026-03-22 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

These background workers established the heartbeat-driven maintenance pattern. The event orchestrator (v0.3.0) generalizes this into hooks and cron jobs.

*** DONE LLM gateway (OpenRouter, Ollama)
:PROPERTIES:
:ID:       id-f5d870e2-cbd2-4c00-a8d4-174ab4118afc
:CREATED:  [2026-04-11 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

The provider-agnostic cascade pattern established in v0.1.0 makes the model-tier router (v0.3.0), privacy-aware routing (v0.3.0), and consensus loop (v0.10.0) possible — they all build on the same ~backend-cascade-call~ abstraction.

*** DONE Shell actuator, Emacs bridge, credentials vault
:PROPERTIES:
:ID:       id-7ca3167f-8353-4bb7-8b97-c039017716b0
:CREATED:  [2026-04-11 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

The actuator registry pattern makes MCP tools (v0.7.0) possible — they register the same way.

*** DONE FiveAM test suite
:PROPERTIES:
:ID:       id-925d4180-764b-4219-8bdc-8e1849572da1
:CREATED:  [2026-04-11 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-20 Mon]
:END:

The test infrastructure established in v0.1.0 becomes the TDD runner (v0.7.1) and the SWE-bench harness (v0.9.0).

** v0.2.0: Interactive Refinement — RELEASED 2026-04-29
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed 20:17]
:END:

The "Brain" meets the "Machine." Standardization and professionalization of the user interface and environment.

*** DONE  Text User Interface (Croatoan-based, styled, scrollable)
:PROPERTIES:
:ID:       id-57cef382-fe14-42e6-aade-03e05e3e920b
:CREATED:  [2026-04-28 Tue]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed]
:END:

The Croatoan-based TUI with model-view separation and dirty-flag rendering is the foundation for all TUI improvements: word wrap in v0.3.3, gate trace in v0.4.0, tool visualization in v0.7.0, and streaming in v0.6.3.

*** DONE Self-editing (error detection, surgical fix, hot-reload)
:PROPERTIES:
:ID:       id-459b8275-9979-4d0f-8d61-a9af883930d4
:CREATED:  [2026-04-23 Wed]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed]
:END:

The surgical edit + tangle + hot-reload pipeline (text replace → tangle → compile → load) established the self-modification capability that makes the Skill Creator (v0.8.0) safe — skills are generated, tangled, loaded, and verified in the same loop.

*** DONE Enhanced utilities (structural Lisp/Org manipulation + REPL)
:PROPERTIES:
:ID:       id-23f37c0d-4e77-4dc3-ab43-52a5987eb426
:CREATED:  [2026-04-23 Wed]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed]
:END:

Structural Lisp/Org manipulation tools are the primitives the self-improve module (v0.2.0) and the programming skills (literate block extraction, syntax validation) build on.

*** DONE Onboarding wizard (modular Lisp setup for LLM providers)
:PROPERTIES:
:ID:       id-bd497de7-3533-4056-b89f-2c992d2ea28b
:CREATED:  [2026-04-28 Tue]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed]
:END:

The setup wizard established the "works out of the box" constraint that the gateway QA (v0.4.0) and Emacs bridge (v0.4.0) onboarding flows follow.

*** DONE Memory rollback (snapshot and restore)
:PROPERTIES:
:ID:       id-fd2fb6e3-03e7-4e22-b9e9-a7eecfd06718
:CREATED:  [2026-04-12 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-04-29 Wed]
:END:

Copy-on-write snapshots (deep-copying the memory hash table on every write) gave the pipeline crash recovery. The snapshot mechanism is the root of MVCC concurrency (v0.6.1).

** v0.3.0: Event Orchestration + HITL — RELEASED 2026-05-06
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-06 Wed 15:50]
:END:

Unified control plane, Human-in-the-Loop state management, and backfill remediation
for stubs and gaps from v0.1.0/v0.2.0. Security hardening followed as
v0.3.1–v0.3.3 point releases.

*** DONE Secret Exposure Gate, Shell Safety, Lisp Validation
:PROPERTIES:
:ID:       id-aa53c128-195b-42d4-9838-2def59faf7cf
:CREATED:  [2026-05-02 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat]
:END:

*** DONE Multi-distro deployment (Debian+Fedora, systemd, Docker)
:PROPERTIES:
:ID:       id-783df999-f7fe-45c8-896d-2fd07c604d64
:CREATED:  [2026-05-02 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat]
:END:

*** DONE Project rename to Passepartout (files, packages, env vars)
:PROPERTIES:
:ID:       id-91724874-aa0d-4804-9220-8bc5551f1366
:CREATED:  [2026-05-02 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat]
:END:

*** DONE 31 org files with full literate prose
:PROPERTIES:
:ID:       id-597b2a92-aac6-481a-b2c4-4f9842ced97c
:CREATED:  [2026-05-02 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat]
:END:

*** DONE Human-in-the-Loop (HITL)
CLOSED: [2026-05-03 Sun 14:00]
:PROPERTIES:
:ID:       id-hitl-complete
:CREATED:  [2026-05-02 Sat]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-03 Sun 14:00]
:END:
Continuation-based interaction. The agent can suspend its cognitive loop to ask for
permission or clarification and resume precisely where it left off. Builds on the
dispatcher's existing Flight Plan mechanism.

*** DONE Event Orchestrator (unified hooks+cron+routing)
:PROPERTIES:
:ID:       id-d35aea3d-2e5f-4a12-a9b0-1c2d3e4f5a6b
:CREATED:  [2026-05-02 Sat 14:00]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat 22:36]
:END:
Unified control plane for hooks, cron, and complexity-based routing.
- *hook-registry* + *cron-registry* + tier classifier
- Hooks via ~#+HOOK:~ Org-mode properties
- Three complexity tiers: ~:REFLEX~ (no LLM), ~:COGNITION~ (light LLM), ~:REASONING~ (full LLM)
- Hooked into heartbeat for cron processing
- Rule-based tier classifier (overrideable via ~*tier-classifier*~)

*** DONE Context Manager (project scoping)
CLOSED: [2026-05-05 Tue]
:PROPERTIES:
:ID:       id-context-manager-scoping
:CREATED:  [2026-05-05 Tue]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-05 Tue]
:END:
Stack-based project focusing with persistence.
- ~push-context~/~pop-context~/~with-context~ stack operations
- ~current-scope~ wired into perceive gate ~*scope-resolver*~
- ~/focus~/~/scope~/~/unfocus~ TUI commands
- Context stack persisted to ~~/.cache/passepartout/context.lisp~, auto-restores on boot

*** DONE Model-Tier Routing (cost optimization)
CLOSED: [2026-05-03 Sun 16:00]
:PROPERTIES:
:ID:       id-model-tier-routing
:CREATED:  [2026-05-02 Sat 23:00]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-03 Sun 16:00]
:END:
Extend ~*model-selector*~ for quadrant-based routing with per-slot provider cascades.
- Privacy filter (local-only for @personal content) — top priority
- Quadrant tagging (foreground/background × probabilistic/deterministic)
- Complexity classifier (code/plan/chat/background slots), each with its own provider cascade
- Model-selector skill registers into =*model-selector*= hook
Deferred to v0.5.0: budget tracking per request, per-session cost monitoring.
Deferred to v0.10.0: TUI /config command for cascade configuration (env vars for now).

*** DONE Memory Scope Segmentation
CLOSED: [2026-05-03 Sun 16:30]
:PROPERTIES:
:ID:       id-memory-scope-segmentation
:CREATED:  [2026-05-02 Sat 23:00]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-03 Sun 16:30]
:END:
Extend memory-object with ~:scope~ property.
- ~:memex~ (permanent knowledge), ~:session~ (ephemeral), ~:project~ (current work)
- Scope-aware retrieval in memory layer

*** DONE Asynchronous Embedding Gateway
CLOSED: [2026-05-05 Tue]
:PROPERTIES:
:ID:       id-async-embedding
:CREATED:  [2026-05-02 Sat 23:00]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-05 Tue]
:END:
Provider-agnostic vector generation (Ollama, OpenAI, hashing fallback).
- Three backends: local (Ollama-compatible), openai (/v1/embeddings), hashing (SHA-256)
- ~embeddings-compute~ and ~*embedding-backend*~ for runtime provider selection
- ~ingest-ast~ populates vectors at object creation time
- ~mark-vector-stale~ marks vectors as ~:pending~ and queues for re-embedding
- ~embed-all-pending~ drains queue, computes vectors, stores in ~*memory-store*~
- Cron job registered with orchestrator: runs every 10m on ~:reflex~ tier
- ~EMBEDDING_PROVIDER~ env var for provider selection
- Registered as proper skill (~defskill~~:passepartout-system-model-embedding~)

*Note:* The default ~:hashing~ backend uses SHA-256-derived vectors. SHA-256 is a
cryptographic hash with the avalanche property — one-bit input differences produce
entirely different outputs. This makes it a correct integrity check (Merkle tree)
but an incorrect similarity function (semantic retrieval). v0.4.0 replaces it with
a zero-dependency lexical similarity algorithm that actually captures textual
overlap while remaining offline-capable.

*** DONE TUI Experience (Daily Driver Quality)
CLOSED: [2026-05-05 Tue]
:PROPERTIES:
:ID:       id-tui-experience
:CREATED:  [2026-05-02 Sat 23:00]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-05 Tue]
:END:
All P0-P4 items implemented:
- P0: Chat scrollback (Page Up/Down), Input history (up/down arrows)
- P1: Status bar (connection, mode, msg count, scroll, activity indicator)
- P1: Message rendering (timestamps, colors, role icons)
- P2: Command palette (~/help~ command listing)
- P2: Multi-line input (~\ + Enter~ inserts newline)
- P3: Background activity indicator (~…thinking~ spinner)
- P4: Tab completion for all ~/~~ commands
- P4: Configurable theme (~*tui-theme*~ plist, ~~/theme~~ command)

*** DONE v0.2.x Backfill Remediation (stubs and gaps)
CLOSED: [2026-05-03 Sun]
:PROPERTIES:
:ID:       id-v02x-remediation
:CREATED:  [2026-05-03 Sun]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-03 Sun]
:END:
- P0: vault-get-secret / vault-set-secret wrappers (one-line delegation to vault-get/vault-set with ~:type :secret~)
- P0: system-archivist Scribe + Gardener (distill daily logs → atomic notes; scan broken links, orphaned memory-objects)
- P0: system-self-improve surgical edit + error fix (read → replace → snapshot → write → balance → tangle → reload)
- P0: programming-org org-modify + org-ast-render (locate node by ID, apply changes; convert plist AST → Org text)
- P0: programming-literate balance check + tangle sync (verify balanced parens in source blocks; verify .lisp matches tangled output)
- P1: system-event-orchestrator bootstrap (scan Org files for HOOK/CRON properties, register via existing registries)
- P1: system-memory introspection (structured statistics: object count by type, TODO distribution, orphans, snapshots)
- P1: path relic skills/ → lisp/ (update skill-initialize-all and context-skill-source to resolve against lisp/ directory)
- P2: core-context semantic retrieval (populate org-object-vector at ingest; fallback: TF-IDF bag-of-words)
- P2: core-context subtree-based skill source loading (context-skill-subtree for targeted retrieval by heading name)
- P3: Variable name drift normalization (*memory* vs *memory-store*, *skills-registry* vs *skill-registry*)
- P4: Eliminate STYLE-WARNINGs from setup output (reorder defuns for same-file forward references; accept cross-skill references)

*** DONE Project Renaming (Bouncer → Dispatcher)
:PROPERTIES:
:ID:       id-9e779580-287b-b3d1-37b9-bcefd750bf9e
:CREATED:  [2026-05-01 Fri 15:40]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-02 Sat 22:00]
:END:
The Dispatcher's role has evolved beyond security guard. It is the seed of the deterministic engine — it learns to execute procedures without invoking the neural net.


*** DONE Parser RCE elimination
:PROPERTIES:
:ID:       id-v031-parser-rce
:CREATED:  [2026-05-06 Wed]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-06 Wed 16:38]
:END:

Rationale: SBCL's default ~*read-eval* accessor is ~t~, enabling the ~#.~ reader macro to execute arbitrary Lisp forms during parsing. Three code paths in the current codebase process untrusted input with ~read-from-string~ or ~read~ without binding ~*read-eval*~ to ~nil~. Each represents a remote code execution vector that bypasses all deterministic safety gates — the Dispatcher's shell safety check, path protection, secret scanning, and network exfiltration detection never execute because the malicious form is evaluated during parsing, before the action plist is even constructed.

- Wrap ~read-from-string~ in ~think()~ (core-loop-reason.lisp:102) with ~(let ((*read-eval* nil)) ...)~ — LLM output is untrusted by definition; parsing it must never execute code. The markdown-strip regex already runs, so the fix immediately follows it.
- Wrap ~read~ in ~load-memory-from-disk~ (core-memory.lisp:143) with ~(let ((*read-eval* nil)) ...)~ — the ~memory.snap~ file lives in ~~/ by default and could be corrupted or planted.
- Wrap ~read-from-string~ in ~action-system-execute~ (core-loop-act.lisp:62) with ~(let ((*read-eval* nil)) ...)~ — the ~:system :eval~ path executes untrusted payload code. Explicitly assert that this path requires the Dispatcher's approval gate.
- Add FiveAM test: inject ~"(#.(shell \"echo pwned\"))"~ into the ~think()~ pipeline and assert no shell execution occurs.

*** DONE Shell safety & actuator sandboxing
:PROPERTIES:
:ID:       id-v032-shell-sandbox
:CREATED:  [2026-05-06 Wed]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-06 Wed 16:46]
:END:

Rationale: The ~:system :eval~ actuator path is currently unchecked by the Dispatcher's approval gate — only ~:shell~ and ~:tool "shell"~ trigger HITL. The shell actuator wraps commands through double ~bash -c~ nesting (~system-actuator-shell.lisp:10~), where Lisp's ~format~ with ~s~ produces S-expression-safe strings, not shell-safe strings. A command containing quotes or substitution characters can break out. Additionally, skill files loaded via ~skill-initialize-all~ execute arbitrary Lisp in jailed packages — a skill file containing ~(uiop:run-program "dangerous")~ executes immediately on load before any gate can inspect it.

- Fix shell double-wrapping: remove the outer ~bash -c~ in ~actuator-shell-execute~; pass the command string directly to ~uiop:run-program~ with ~:force-shell nil~. The timeout wrapping remains via the OS ~timeout~ binary.
- Extend the Dispatcher approval requirement to the ~:system :eval~ path (currently only ~:shell~ and ~:tool "shell"~ trigger HITL). An unbounded ~eval~ should require the same Flight Plan approval as a shell command.
- Add skill sandbox mode for ~skill-initialize-all~: load each skill's code into a temporary jailed package, run the registered trigger function in isolation, verify it imports no restricted symbols (from CL package: ~run-program~, ~shell~, ~run-shell-command~), then promote to the live registry on pass.
- Add FiveAM test: register a skill containing ~(uiop:run-program "echo test")~ in the body and verify the sandbox blocks its promotion.

*** DONE TUI Critical Fixes
:PROPERTIES:
:ID:       id-v033-tui-fixes
:CREATED:  [2026-05-06 Wed]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-06 Wed 17:59]
:END:

Rationale: The TUI is Passepartout's only interface. OpenClaw distributes across 25+ messaging channels with voice, Canvas, and macOS/iOS apps. Hermes Agent ships multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output in its TUI. Passepartout's Croatoan TUI must carry the product alone, and it currently lacks word wrap, cursor movement, resize handling, connection-loss feedback, a quit command, and persistent history. None of these fixes require daemon changes — they are pure client-side Croatoan work that closes the gap from "proof of concept" to "daily driver."

- Word wrap in ~view-chat~: every LLM response longer than the terminal width is silently truncated to one line. Croatoan supports multi-line rendering; ~view-chat~ must calculate per-message line height, adjust visible-message count accordingly, and scroll per message-line rather than per message. For very long messages, add a pager mode where pressing Enter on a message opens it in a scrollable overlay.
- Left/Right cursor in input: add ~:left~ and ~:right~ key handlers that move a cursor position index within the ~:input-buffer~ list. Characters are inserted at the cursor position, not always appended. Backspace deletes at the cursor position.
- SIGWINCH handler: register a terminal resize signal. On resize, re-measure the root window, destroy and recreate the three sub-windows (~sw~, ~cw~, ~iw~), set all dirty flags to ~t~, and force a full redraw.
- Connection-loss detection: the reader thread currently polls ~recv-daemon~ silently on EOF. On disconnection, queue a ~:disconnected~ event, set ~:connected~ to ~nil~, clear ~:busy~, add a red system message "Connection lost — run /reconnect to retry." The ~:disconnected~ event dirties the status bar to show the status indicator.
- ~/quit~ command + persistent history: on ~/quit~, save ~:input-history~ to ~~/.cache/passepartout/history~ (one line per entry, most recent first), send a goodbye handshake to the daemon, close the socket, and exit the main loop cleanly. On startup, load history from the save file if it exists.
- Scroll offset clamping: clamp ~:scroll-offset~ to ~(max 0 (- msg-count visible-lines))~. The status bar shows ~"msgs:12/45"~ (visible / total) rather than ~"msgs:45"~ (total only) so the user knows when they've scrolled past the oldest message.
- Message list storage: replace the O(n²) ~(nth i msgs)~ list indexing with a simple adjustable vector. ~add-msg~ appends; ~view-chat~ iterates with ~aref~. The vector is resized as needed. Same API surface, 100x speedup on message-heavy sessions.
- Add FiveAM tests: word-wrap produces correct line count for a 200-character string at 80-column width; cursor left/right wraps at buffer boundaries; SIGWINCH preserves message state; ~/quit~ saves and restores history.

** v0.4.0: Production Hardening — RELEASED 2026-05-06
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-06 Wed 20:56]
:END:

The features in this version were originally sequenced as v0.3.x patches but represent feature-level scope. They activate the architectural advantages designed in v0.1.0–v0.3.0, harden the self-build safety boundary, and expand Passepartout's interaction surfaces beyond the terminal TUI. Each feature depends on infrastructure already in place — the wiring, the sandbox, the gate trace — and activates it.

*** DONE Semantic retrieval activation
CLOSED: [2026-05-06 Tue]
- State "DONE" from "TODO" [2026-05-06 Tue]

Rationale: Two independent failures prevent the foveal-peripheral semantic retrieval path from ever firing. First, ~context-awareness-assemble~ never passes ~:foveal-vector~ to ~context-object-render~, so the renderer receives ~nil~ for ~foveal-vector~ and the similarity calculation always returns 0.0. Second, the default ~:hashing~ embedding backend uses SHA-256 (a cryptographic hash with the avalanche property) as a similarity function. SHA-256 is designed to produce entirely different outputs for nearly identical inputs — the property that makes it secure for integrity verification is precisely what makes it useless for semantic retrieval. A content-addressed Merkle tree correctly uses SHA-256 for identity; a retrieval engine needs a similarity function, not an identity function. The infrastructure for real embeddings (~local~ with Ollama, ~openai~ with the embeddings API) is fully implemented and working — this release activates the last-mile wiring and replaces the semantically blind default with a zero-dependency algorithm that actually captures textual overlap.

- Wire ~:foveal-vector~ into ~context-awareness-assemble~: pass ~(memory-object-vector (memory-object-get foveal-id))~ as the ~:foveal-vector~ argument to the ~context-object-render~ call (one line in ~core-context.lisp:148-150~).
- Replace ~:hashing~ default backend with character-trigram Jaccard similarity. Pure Lisp, zero external dependencies, works exactly as offline as SHA-256, but captures lexical overlap: "authentication" and "authenticate" share trigrams "aut," "uth," "the," "hen," "ent," etc. The vector is a bloom filter of trigrams; cosine similarity maps to Jaccard (intersection / union). This provides real if crude semantic signal without any server.
- Rename existing ~embedding-backend-hashing~ to ~embedding-backend-sha256~ and repurpose it as an explicit ~:sha256~ provider for environments where even trivial Lisp computation is undesirable (embedded, resource-constrained). Document it as "integrity-only, no semantic retrieval capability."
- Add ~EMBEDDING_PROVIDER~ guidance to the setup wizard: explain that ~:hashing~ is the default offline fallback, ~:local~ requires Ollama with ~nomic-embed-text~, and ~:openai~ uses the paid embeddings API.
- Add FiveAM test: ingest two semantically related nodes ("implement login form" and "add password authentication"), verify cosine similarity > 0.0 with the trigram backend.

*** DONE Self-build safety boundary
CLOSED: [2026-05-06 Tue]
- State "DONE" from "TODO" [2026-05-06 Tue]

Rationale: Self-building (the agent modifying its own source code) begins at v0.7.1 when the tool ecosystem and test runner are in place. But self-building without path-level write protection means the agent can modify the very pipeline code that is currently executing — the ~core-*~ files that implement the Perceive-Reason-Act cycle, the Merkle-tree memory, the skill engine loader, and the Dispatcher gate stack itself. A hallucination or a logic error during self-building that corrupts ~core-loop-reason.lisp~ destroys the agent's ability to reason about and fix the corruption. The "thin harness" is not privileged code in the architectural sense (homoiconicity means any code can be modified at runtime), but it must be *protected* code — modifications to the harness require a human in the loop, enforced by the Dispatcher's path-protection gate, not by convention.

This is the corollary to "thin harness, fat skills": the harness is thin enough to be auditable by a human, and the Dispatcher ensures it stays that way. Skills and system modules expand freely; the core contracts to a minimal, protected kernel.

- Add ~core-*~ patterns to ~*dispatcher-protected-paths*~: ~core-*.org~, ~core-*.lisp~, and their tangled equivalents. Any file write, file read-that-prefaces-a-write, or shell command targeting these paths triggers the Dispatcher's blocking gate.
- The blocked action produces a Flight Plan (HITL approval required). The human reviews the proposed core change in an Org buffer before approving. This is the same mechanism that governs shell commands and network exfiltration — the core protection is a path-specific instance of the existing gate, not a new gate.
- Implement a ~SELF_BUILD_MODE~ env var. When ~SELF_BUILD_MODE=true~ (default ~false~):
  - Core path protection is active (writes blocked, HITL required)
  - Non-core writes proceed through the standard Dispatcher gate (permissions table + policy + Dispatcher)
  - ~SELF_BUILD_MODE=false~ disables core protection entirely — useful during initial development when the human is manually editing core files and doesn't want every save to trigger a Flight Plan
- Telemetry: track self-build actions (core modifications proposed, core modifications approved, core modifications denied). This is the dataset that the Dispatcher's learning system uses in v3.0.0 to understand which core modifications are safe enough to automate.
- Add FiveAM test: simulate a write to ~core-loop.lisp~, verify the Dispatcher returns a ~:LOG~ rejection with ~"protected path"~ in the message.

*** DONE TUI Differentiator Visualization
CLOSED: [2026-05-06 Tue]
- State "DONE" from "TODO" [2026-05-06 Tue]

Rationale: Three architectural elements exist today in the daemon that no competitor can render — the Dispatcher gate trace, the foveal-peripheral focus map, and the rules-learned counter. All three run in pure Lisp with 0 LLM tokens. None are visible to the user. Making them visible turns Passepartout's architecture from an internal mechanism into a trust-building UX — the user sees exactly which safety gates passed, exactly what the agent is focusing on, and exactly how many rules the Dispatcher has learned from their decisions. No competitor can ship this because none has deterministic gates to trace, foveal-peripheral context to map, or a rule-synthesizing Dispatcher to count.

- Gate trace per action: extend the daemon's response plist to include ~:gate-trace~ — a list of ~(:gate <name> :result <:passed | :blocked | :approval>)~ entries produced by ~cognitive-verify~. The TUI renders each entry as a colored line below the corresponding agent message: green ~✓ Dispatcher: path allowed~, red ~✗ Dispatcher: blocked (shell safety)~, yellow ~→ HITL required: /approve HITL-ab12~. Gate trace lines are dim and collapsible (press Tab on a message to toggle trace visibility). This turns the invisible ten-vector safety gate into the user's primary trust mechanism.
- Focus map in status bar: add a second status bar line showing ~[Focus: core-loop.lisp:think()] [Scope: passepartout] [3 related nodes]~. The daemon already tracks ~foveal-id~ and ~*scope-resolver*~ in the signal plist; the TUI reads these from the most recent response and renders them. Related node count comes from the number of objects with cosine similarity ≥ threshold in the last context assembly. This shows the user *what the agent is looking at* — the single biggest trust gap in AI agents.
- Rule counter in status bar: ~[Rules: 47]~. The Dispatcher's ~*hitl-pending*~ hash table and approved/disallowed memory-object entries provide the count — every HITL decision that produces a rule increments it. The TUI reads the count from a new daemon response field ~:rule-count~. The user watches the counter tick up as they teach the agent their preferences.
- Expanded theme: replace the 7-flat-color ~*tui-theme*~ with a 25-color layered system organized by message category (roles, content types, tool visibility, gate states, status). See the design discussion for the full color mapping. Implement a ~/theme <name>~ command that swaps between named presets (~dark~, ~light~, ~solarized~, ~gruvbox~). Theme change persists to disk and reloads on next session.
- Add FiveAM tests: gate trace renders correctly for pass/block/approval states; focus map updates when ~foveal-id~ changes; rule counter increments on HITL approval.

*** DONE Gateway QA, Discord, Slack + Emacs Bridge
CLOSED: [2026-05-06 Tue]
- State "DONE" from "TODO" [2026-05-06 Tue]

Rationale: Passepartout currently has Telegram and Signal gateways in the codebase, both untested. The setup wizard has Slack as a configurable option with no implementation. Two messaging channels is not competitive — OpenClaw has 25+, Hermes Agent has 6+. But more critically: the Lisp crowd is Passepartout's natural audience, and they live in Emacs. An Emacs bridge that speaks the framed TCP protocol is trivial to implement (the protocol is ~200 lines of Lisp; porting to elisp is straightforward) and turns every Emacs buffer into a Passepartout interaction surface. This is not the deep Emacs integration of v0.10.2 (where the agent controls Emacs) — this is Emacs controlling the agent over TCP. The Emacs user selects a region, hits ~M-x passepartout-send-region~, and the agent responds in a dedicated buffer. They never leave their editor.

Gateway:
- Integration tests for Telegram gateway: mock the Telegram Bot API, verify message send (POST ~/sendMessage~) and receive (GET ~/getUpdates~) round-trip. Verify HITL commands (~/approve~, ~/deny~) are intercepted before injection.
- Integration tests for Signal gateway: mock ~signal-cli~ output, verify JSON message parsing and polling loop. Verify send path constructs correct ~signal-cli send~ arguments.
- Add Discord gateway: Discord Bot API (REST + Gateway WebSocket for real-time messages). Register bot, handle ~MESSAGE_CREATE~ events, send via ~POST /channels/{id}/messages~. Map Discord mentions to ~:user-input~ signals. HITL commands work identically to Telegram.
- Add Slack gateway: Slack Events API + Web API. Subscribe to ~message.im~ events, send via ~chat.postMessage~. Reuse the SLACK_TOKEN config key already present in the setup wizard.
- Each gateway is a skill under ~passepartout.skills.gateway-<platform>~ — jail-loaded, hot-reloadable, sandbox-verified.
- Gateway configuration surfaced in the setup wizard: after entering a token, offer "send a test message to yourself" as a connection verification step. Surface the result as a green ✓ or red ✗ with the error detail.
- Gateway status displayed in ~messaging-list~: platform, configured (yes/no), gateway active (yes/no), last message received (timestamp).

Emacs Bridge:
- Elisp package: ~passepartout.el~. Connects to daemon on localhost:9105 via ~make-network-process~ (TCP).
- Sends: framed plist protocol identical to the TUI (~frame-message~ ported to elisp — write hex length prefix, write prini'd plist). The daemon does not know or care whether the client is the Croatoan TUI, the CLI, or Emacs.
- Receives: daemon responses arrive in a ~passepartout-response~ buffer. Each response is rendered as an Org headline: role prefix, timestamp, content. Gate trace (from v0.4.0) is rendered as property drawer entries under the headline.
- ~M-x passepartout-send-region~: sends the selected region as a ~:user-input~ signal with the current buffer's file path as context.
- ~M-x passepartout-send-buffer~: sends the entire buffer.
- ~M-x passepartout-focus~: sets the foveal focus to the Org headline at point (extracts ~:ID:~ property, sends ~:point-update~ signal). Equivalent to the TUI's ~/focus~ command.
- ~M-x passepartout-approve~ / ~M-x passepartout-deny~: prompts for HITL token and sends approval/denial.
- Agent modifies an Org file → Emacs receives ~:buffer-update~ via the bridge → the buffer is refreshed (~revert-buffer~ or targeted replacement).
- The Emacs bridge is the daily driver for Lisp users. The TUI remains for non-Emacs users and for the differentiator visualizations. Emacs users get the gate trace and focus map as Org property drawers in the response buffer — same data, elisp-native rendering.

**** DONE Native embedding inference
CLOSED: [2026-05-07 Thu]

Implemented: in-process embedding inference via CFFI binding to llama.cpp.

- FFI binding to llama.cpp's current (non-deprecated) embedding API via a C wrapper library (~/usr/local/lib/libllama_wrap.so~) that bridges CFFI pointer params to llama.cpp struct-by-value calls
- Builds on ~/usr/local/lib/libllama.so~ (llama.cpp shared library)
- Ship nomic-embed-text-v1.5 (80MB Q4_K_M GGUF) as the bundled embedding model. 768-dimensional vectors (nomic-bert, 12 layers), CPU-friendly, <100ms per document on any modern CPU
- ~EMBEDDING_PROVIDER=native~ enables the native backend; model preloads at daemon startup (~30s)
- Lazy loading via ~*embedding-backend* :native~ also works (first call blocks ~45s for model init)
- C wrapper functions: ~llama_wrap_model_load~, ~llama_wrap_new_context~, ~llama_wrap_encode~, ~llama_wrap_batch_init/free~
- Struct sizes verified via C sizeof/offsetof: llama_model_params (72B), llama_context_params (136B), llama_batch (56B)
- BERT pooling: uses ~llama_get_embeddings_seq~ for sequence-level embedding
- ~sb-int:set-floating-point-modes :traps nil~ required before any llama.cpp call (FPU state conflict)
- ~llama_backend_init~ required before model load
- ~llama_model_get_vocab~ + ~llama_vocab_n_tokens~ replaces deprecated ~llama_n_vocab~
- ~llama_tokenize~ takes ~vocab*~ not ~model*~ (API change since earlier llama.cpp versions)
- Exports: ~embedding-backend-native~, ~embedding-native-load-model~, ~embedding-native-unload~, ~embedding-native-ensure-loaded~, ~embedding-native-get-dim~
- FiveAM tests: availability, loading, dimensions (768), self-similarity (1.0), semantic similarity ranking
- The trigram Jaccard backend remains as the default fallback for zero-config deployments

- State "DONE" from "TODO" [2026-05-07 Thu]

*** Competitive Advantage Analysis — v0.4.0 Summary

Production hardening is the process of turning architectural potential into operational strength. The semantic retrieval fix activates the foveal-peripheral model's full power: deep nodes that are topically related to the user's focus now surface automatically. Without this, the context model is "dumb truncation at depth 2." With it, it's genuine semantic awareness — and since the retrieval is deterministic (in-image vector math, zero LLM tokens), the cost advantage over competitors' LLM-assisted search compounds with every query.

The self-build safety boundary is a capability no competitor provides: the agent cannot modify its own brain stem without human review. The ~core-*~ path protection means the Dispatcher draws a line at the filesystem level, not the policy document level. Claude Code, OpenClaw, and Hermes all allow agents to modify their own source files without distinction between application code and runtime code. Passepartout's Dispatcher prevents modification of the very pipeline that implements the Perceive-Reason-Act cycle, the Merkle-tree memory, the skill engine loader, and the Dispatcher gate stack itself. This is the operational realization of "thin harness, fat skills" — the harness is thin enough to be auditable by a human, and the Dispatcher ensures it stays that way.

The TUI differentiator visualizations are Passepartout's permanent UX advantage. The gate trace, focus map, and rule counter are UX elements that only make sense in Passepartout's architecture — deterministic gates, foveal-peripheral context, and Dispatcher rule synthesis exist nowhere else. No competitor can ship this because none has deterministic gates to trace, foveal-peripheral context to map, or a rule-synthesizing Dispatcher to count. Combined with the TUI critical fixes from v0.3.3, the TUI is competitive on usability and uniquely informative on safety and context transparency.

The messaging gateways and Emacs bridge expand Passepartout's interaction surface from a single terminal TUI to four surfaces: terminal, Telegram/Signal/Discord/Slack messaging, Emacs, and voice (via the voice gateway in v0.7.3). The Emacs bridge is strategically critical — the Lisp crowd is Passepartout's natural audience, and they live in Emacs. An Emacs bridge that speaks the framed TCP protocol turns every Emacs buffer into a Passepartout interaction surface. Combined with the gate trace and focus map rendered as Org property drawers in the response buffer, Emacs users get the same differentiator visualizations as TUI users — same data, elisp-native rendering.

** v0.4.1: Design Cleanup

*** DONE Remove system-prompt-augment mechanism
:PROPERTIES:
:ID:       id-v041-augment-removal
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 13:13]
:END:

Rationale: The ~system-prompt-augment~ slot on the skill struct enables skills to inject always-on text into every LLM system prompt via a ~maphash~ over ~*skill-registry*~ in ~think()~ (core-loop-reason.lisp:83-92). Only one skill uses it — ~programming-repl~ — and it does so as a backdoor: the skill's trigger is hardcoded to ~nil~, so it never fires as an active skill. Its sole contribution is injecting a REPL-first mandate into every system prompt. The other ~24 skills have nil augments and are skipped by the ~when aug-fn~ guard. This is architecturally wrong: standing mandates (always-on rules) should live in a dedicated ~*standing-mandates*~ list, not piggyback on a skill that is never triggered. The mechanism also fuels a false claim in DESIGN_DECISIONS about 3,000-8,000 tokens of overhead — the actual overhead is ~40 tokens from the one active augment.

- Remove ~system-prompt-augment~ slot from the ~skill~ defstruct and ~defskill~ macro (core-skills.org:78, core-skills.org:121-133).
- Remove the ~maphash~ skill-augments collection block from ~think()~ and the associated ~(or skill-augments "")~ injection in the system-prompt ~format~ call (core-loop-reason.org:83-95, core-loop-reason.org:196-198).
- Remove ~:system-prompt-augment #'repl-mandate~ from ~programming-repl~'s ~defskill~ (programming-repl.org:269).
- Introduce ~*standing-mandates*~ (a list of function → string generators). Inject them into the IDENTITY section of the system prompt alongside ~assistant-name~. Move ~repl-mandate~ there: ~(push #'repl-mandate *standing-mandates*)~.
- Tangle the corresponding lisp/ files.

*** DONE Fix false token-overhead claims in docs
:PROPERTIES:
:ID:       id-v041-doc-fix
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 13:13]
:END:

Rationale: Two documents claim the ~system-prompt-augment~ mechanism can waste 3,000-8,000 tokens per think() call (DESIGN_DECISIONS line 435, ROADMAP line 504). This conflates the ~maphash~ iteration (cheap hash walk, no token cost) with the augments actually emitted (only ~programming-repl~ emits ~40 tokens; the ~when aug-fn~ guard skips the other 24 nil-augment skills). Once issue #1 above is resolved (removing the mechanism), these claims become doubly false.

- DESIGN_DECISIONS: Rewrite or remove bullet 2 under "Open Questions and Risks" (line 435). Replace with a corrected note on standing mandates via ~*standing-mandates*~.
- ROADMAP v0.5.0 intro (line 504): Remove or rewrite the claim that "system prompt overhead alone could reach 3,000-8,000 tokens per call before user input is even processed." The fixed overhead is not from skill augments — it is from the IDENTITY, TOOLS, CONTEXT, and LOGS sections, which prefix caching addresses.

*** DONE Update security vector count 9→10 in docs
:PROPERTIES:
:ID:       id-v041-vector-count
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 14:40]
:END:

Rationale: The current dispatcher runs 10 deterministic checks (11 counting the warning-only REPL lint), but the README, ARCHITECTURE.org, and the ~dispatcher-check~ docstring all say 9. The actual count: 0=REPL-lint (warn only), 1=lisp-validation, 2=secret-path, 2b=self-build-core, 3=secret-content, 4=vault-secrets, 5=privacy-tags, 6=privacy-text, 7=shell-safety, 8=network-exfil, 8b=high-impact-approval. Ten blocking/approval checks. The vector 2b (self-build safety) and the new count must be reflected accurately in all documentation.

- Update README.org "What Makes Passepartout Different" → "nine" becomes "ten".
- Update docs/ARCHITECTURE.org Dispatcher Gate Stack table — add self-build entry.
- Update security-dispatcher.lisp:196 docstring to list all 11 vectors.

*** DONE Rewrite README — add "What is an agent?" section, revise claims
:PROPERTIES:
:ID:       id-v041-readme-rewrite
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 14:40]
:END:

Rationale: The current README opens with competitive claims (downward cost curve, 2-3x fewer tokens) that are architecturally sound but not yet measured in the implementation. A non-engineer reader doesn't know what an AI agent is or why they'd want one. The README should lead with a short "What is an agent?" section (3-4 sentences, Wikipedia link), then "What Makes It Different" (safety, org-mode, offline — things that actually work today), then honest status of what's implemented vs planned.

- Add "What is an AI Agent?" section at top: 3-4 sentences + link to [[https://en.wikipedia.org/wiki/Software_agent][Software agent]].
- Move competitive cost/speed claims to docs/DESIGN_DECISIONS.org.
- Revise "The more you use it, the cheaper it gets" to reflect current state — architectural aspiration, not measured implementation yet.
- The Current Capabilities table and Quick Start sections stay intact.

*** DONE Register cognitive tools — 10 tools for codebase operations
:PROPERTIES:
:ID:       id-v041-cognitive-tools
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 14:40]
:END:

Rationale: The ~def-cognitive-tool~ macro and ~*cognitive-tool-registry*~ are fully implemented but the registry is empty. The LLM sees "No tools registered" in its tool belt prompt. The agent can chat and run shell commands, but cannot search codebases, find files, eval code, run tests, or manipulate Org files. Ten cognitive tools bridge this gap and are prerequisites for the TDD workflow, org-mode additions, and evaluation harness in v0.5.0.

- New skill: ~programming-tools.org~ (~programming-tools.lisp~).
- Register 10 tools via ~def-cognitive-tool~:
  1. ~search-files~ — regex search in file contents (uses ~cl-ppcre:scan~). Parameters: ~pattern~, ~path~ (dir), ~include~ (glob filter).
  2. ~find-files~ — glob file matching (uses SBCL ~directory~). Parameters: ~pattern~, ~path~.
  3. ~read-file~ — read file contents (uses ~uiop:read-file-string~). Parameters: ~filepath~.
  4. ~write-file~ — write content to file. Parameters: ~filepath~, ~content~.
  5. ~list-directory~ — list directory contents. Parameters: ~path~, ~pattern~ (optional).
  6. ~run-shell~ — execute shell command (through existing shell actuator). Parameters: ~cmd~.
  7. ~eval-form~ — evaluate Lisp expression in running image. Parameters: ~code~, ~package~ (optional).
  8. ~run-tests~ — run FiveAM tests. Parameters: ~test-name~ (optional, nil runs all).
  9. ~org-find-headline~ — find Org headline by ID or title. Parameters: ~id~ or ~title~, ~filepath~ (optional, searches memory store if not given).
  10. ~org-modify-file~ — surgical text replacement in Org file (reuses existing ~org-modify~). Parameters: ~filepath~, ~old-text~, ~new-text~.
- Descriptive names rather than Unix command names — the LLM reads these in a prompt, not a terminal.
- Each tool is ~20-60 lines. ~search-files~ iterates directory, reads files, scans lines.
- FiveAM tests: each tool gets a test verifying operation on a temp directory.

*** DONE Enforce NO-HARDCODED-CONSTANTS programming standard
:PROPERTIES:
:ID:       id-v041-no-hardcoded
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 14:40]
:END:

Rationale: Currently, several configurable values are hardcoded in source: the Dispatcher's rule threshold (not yet configurable), similarity thresholds, timeouts, shell max output. The user should control behavior through ~.env~, not by editing source code. This is rule #6 in the ~programming-standards.org~ skill. Each new TODO that introduces a configurable value must add it to ~.env.example~ with a documented default.

- Add ~DISPATCHER_RULE_THRESHOLD=3~ to ~.env.example~ (number of HITL approvals before a pattern becomes a permanent rule).
- Add ~RULES_FILE="$HOME/memex/system/rules.org"~ to ~.env.example~.
- Scan existing source for hardcoded configurable values — add to ~.env.example~ where missing.
- Any new TODO in v0.4.2+ that introduces a configurable value MUST include its ~.env.example~ entry.

** v0.4.2: Structured Output (LLM → JSON → plist)

The current ~think()~ function asks the LLM to produce raw S-expression plists. Four pieces of defensive infrastructure (~handler-case~ around ~read-from-string~, ~markdown-strip~, ~plist-keywords-normalize~, the RCE guard test) exist because LLMs cannot reliably produce balanced, keyword-prefixed plists. The fix: use the LLM API's native function calling / tool-use feature. The LLM always returns guaranteed-valid JSON. Convert to plist deterministically at the boundary.

*** DONE Implement function-calling / tool-use API in provider requests
:PROPERTIES:
:ID:       id-v042-function-calling
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 17:17]
:END:

Rationale: Every major provider API (OpenAI, Anthropic, Groq, DeepSeek, OpenRouter) supports function calling. The LLM is sent tool definitions as JSON Schema. It returns ~tool_calls~ with guaranteed-valid JSON arguments. This eliminates the fragile ~read-from-string~ plist parsing entirely — the probabilistic layer speaks JSON (what it was trained on), the deterministic layer speaks plists (what the code controls). Conversion happens at a narrow, well-defined boundary.

- Modify ~provider-openai-request~ in ~system-model-provider.lisp~: add optional ~:tools~ parameter. When tools are provided, include ~"tools": [...]~ and ~"tool_choice": "auto"~ in the request body.
- Parse ~tool_calls~ from the API response: extract ~function.name~ and ~function.arguments~ (guaranteed valid JSON).
- Return a new result shape: ~(:status :success :tool-calls ((:name "shell" :arguments (:cmd "echo hello"))))~ alongside or instead of ~:content~.
- For providers that don't support function calling (local Ollama): keep ~:content~ path as fallback. LLM can still return raw text.
- FiveAM test: send a request with a mock tool definition, verify the response shape.

*** DONE Wire structured tool calls into ~think()~ — JSON→plist at boundary
:PROPERTIES:
:ID:       id-v042-wire-tool-calls
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 17:17]
:END:

Rationale: Once the provider layer returns structured ~tool-calls~, the ~think()~ function must convert them to the internal plist format that ~cognitive-verify~ and ~loop-gate-act~ expect. This is a one-way, deterministic conversion at the architectural boundary.

- Add ~json-alist-to-plist~ helper in ~core-loop-reason.lisp~: convert JSON alist (from ~cl-json:decode-json-from-string~) to keyword-prefixed plist. String keys → keywords. Nested objects recurse. JSON null → ~nil~. ~25 lines.
- In ~think()~ after ~backend-cascade-call~: if result contains ~:tool-calls~, convert each tool call's ~:arguments~ JSON to plist via ~json-alist-to-plist~, wrap in ~(:TYPE :REQUEST :PAYLOAD (:TOOL <name> :ARGS <plist> :EXPLANATION "..."))~.
- Keep the existing ~read-from-string~ path as fallback for providers that return raw text (local Ollama, streaming).
- The ~read-from-string~ path remains guarded by ~*read-eval* nil~ from v0.3.1.
- FiveAM test: JSON ~{"action":"shell","cmd":"echo hello"}~ → plist ~(:ACTION "shell" :CMD "echo hello")~ round-trip verified.

** v0.4.3: Shell Sandboxing & Safety Classification

The current shell safety is regex-based pattern matching — a fast pre-filter that catches obvious attacks but cannot contain sophisticated or encoded payloads. This version adds actual sandbox isolation (bubblewrap Linux namespaces) as the enforcement layer, and introduces severity classification so the rule learning system in v0.5.0 can apply different thresholds to catastrophic vs harmless operations.

*** DONE Add ~bwrap~ sandbox to shell actuator
:PROPERTIES:
:ID:       id-v043-bwrap-sandbox
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 17:37]
:END:

Rationale: Regex-based shell safety catches obvious patterns (~rm -rf /~, ~dd if=~, ~mkfs.~) but is fundamentally bypassable with encoding (~base64 -d | bash~), indirection (~find / -exec rm {} \;~), or interpreter-based execution (~python3 -c "import os; os.system(...)"~). Bubblewrap (~bwrap~) is a 200KB unprivileged sandbox binary available on all modern Linux distributions. It creates transient Linux namespaces without root, without Docker, without daemon processes. Combined with the regex pre-filter, it provides defense-in-depth: the regex catches obvious attacks fast (no sandbox spawn), the sandbox contains sophisticated ones.

- In ~actuator-shell-execute~ (~system-actuator-shell.lisp~): detect if ~bwrap~ binary is available (~which bwrap~).
- If available: wrap command in ~bwrap --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind /bin /bin --ro-bind /etc /etc --bind ~/memex ~/memex --bind /tmp /tmp --unshare-net --unshare-ipc timeout ...~.
- ~--unshare-net~: no network access within sandbox. Makes regex-based network exfiltration check redundant for sandboxed commands.
- ~--unshare-ipc~: no shared memory, no semaphore injection.
- If ~bwrap~ is unavailable: log a warning, fall back to current behavior (regex-only safety).
- The regex checks remain as a fast pre-filter — they run before spawning the sandbox.
- FiveAM test: command that reads ~/etc/shadow~ inside sandbox fails with permission error; same command in unsandboxed fallback is at least caught by path protection.

*** DONE Shell safety severity classification system
:PROPERTIES:
:ID:       id-v043-severity-classification
:CREATED:  [2026-05-07 Thu]
:END:
:LOGBOOK:
- State "DONE" from "TODO" [2026-05-07 Thu 17:37]
:END:

Rationale: The current shell safety check treats all dangerous patterns equally — ~rm -rf /~ gets the same treatment as a backtick injection in ~echo~. But not all shell operations carry the same risk. A severity classification system enables the rule learning engine (v0.5.0) to apply different thresholds: catastrophic operations are always HITL regardless of approval count, moderate operations graduate to allowed after N approvals, harmless operations are allowed by default.

- Define four severity tiers as plist keywords: ~:catastrophic~ (mkfs, dd to devices, rm -rf /, shred /dev/), ~:dangerous~ (chmod -R /, writes outside ~/memex, curl to unwhitelisted domains), ~:moderate~ (npm install, pip install, git push, writes within ~/memex), ~:harmless~ (echo, ls, cat, find without exec, grep).
- Extend ~*dispatcher-shell-blocked*~ entries from simple ~(NAME REGEX)~ to ~(NAME REGEX :SEVERITY <tier>)~.
- Extend ~dispatcher-check-shell-safety~ to return the severity alongside the matched pattern name.
- ~:catastrophic~ severity always triggers HITL approval, regardless of rule count. ~:harmless~ operations are allowed by default (skip HITL and rule learning).
- The severity classification is the foundation that ~dispatcher-learn~ (v0.5.0) builds on — learning only applies to ~:dangerous~ and ~:moderate~ tiers.
- FiveAM test: ~echo hello~ returns ~:harmless~ severity and passes through; ~mkfs.ext4 /dev/sda~ returns ~:catastrophic~ and is always blocked.

** v0.5.0: File Reorganization & Token Economics

The foundation work: rename and restructure the codebase around the self-repair criterion, extract non-core fragments from core, then build the learning loop on clean foundations.

*** File Reorganization — self-repair criterion

Rationale: The current file naming scheme mixes three concerns: architectural role (core-* = harness, system-* = skill), domain (security-*, programming-*, gateway-*), and implementation nature (system-model-* is LLM infrastructure, not a "system"). Worse, two fragments that can be extracted from core (context assembly, heartbeat) currently live there because the criterion for "what is core" was never defined. This reorganization establishes the criterion and applies it.

The criterion: a file belongs in core if, when corrupted, the agent cannot fix it without human help. Corrupted core = dead brain, dead hands, or unreachable. Corrupted skill = degraded but self-repairable.

*** TODO Extract core-context → symbolic-awareness
:PROPERTIES:
:ID:       id-v050-reorg-awareness
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: ~core-context.lisp~ (224 lines) handles ~context-assemble-global-awareness~, ~context-object-render~, ~context-query~, and related functions. If corrupted, the LLM receives empty awareness. But the agent still has tools, identity, and user input. It can reason about "no awareness", edit the context source file, reload the skill, and awareness returns. Degraded, not dead. Safe to extract.

- Move ~core-context.lisp~ content to new ~symbolic-awareness.lisp~ (new ~org/symbolic-awareness.org~).
- Register as a skill via ~defskill :passepartout-symbolic-awareness~.
- In ~core-reason.lisp~'s ~think()~: wrap ~context-assemble-global-awareness~ and ~context-get-system-logs~ calls with ~fboundp~ guards. On skill failure, inject degraded awareness note.
- Remove ~core-context~ from ~passepartout.asd~ ~:components~.
- FiveAM: verify ~think()~ produces valid output when awareness skill is not loaded.

*** TODO Extract heartbeat generation → symbolic-events
:PROPERTIES:
:ID:       id-v050-reorg-heartbeat
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The heartbeat thread (~heartbeat-start~, ~*heartbeat-thread*~, auto-save counter) lives in ~core-loop.lisp~ (~50 lines). If heartbeat is corrupted or missing, the agent has no background ticks — no cron jobs, no auto-save. But the agent is fully functional: it perceives, reasons, and acts. It can detect missing ticks, reload the events skill, and heartbeat returns. Safe to extract.

- Move heartbeat generation (~heartbeat-start~, ~*heartbeat-thread*~, ~*heartbeat-save-counter*~, ~*memory-auto-save-interval*~) from ~core-pipeline.lisp~ to ~symbolic-events.lisp~.
- Rename ~heartbeat-start~ → ~events-start-heartbeat~.
- In ~core-pipeline.lisp~'s ~main()~: change ~(heartbeat-start)~ to ~(when (fboundp 'events-start-heartbeat) (events-start-heartbeat))~.
- ~symbolic-events~ already processes ~:heartbeat~ signals for cron dispatch (existing code). Now it also generates them.

*** TODO Relocate 6 utility fragments to correct files
:PROPERTIES:
:ID:       id-v050-reorg-utilities
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: Several functions live in core files not because they need core protection but because they were written there first. They are utility functions that can be extracted into skills.

- ~markdown-strip~ (core-reason.lisp:51) → new ~programming-markdown.lisp~ (~org/programming-markdown.org~).
- ~plist-keywords-normalize~ (core-reason.lisp:60) → ~programming-lisp.lisp~.
- ~cognitive-tool-prompt~ / ~generate-tool-belt-prompt~ (core-defpackage.lisp:214-231) → ~programming-tools.lisp~.
- ~lisp-syntax-validate~ (core-skills.lisp) → ~programming-lisp.lisp~.
- ~VAULT-MASK-STRING~ + ~*VAULT-MEMORY*~ (core-skills.lisp) → ~security-vault.lisp~.
- ~*backend-registry*~ dedup: merge with ~*probabilistic-backends*~ (core-reason.lisp:10-12), remove ~backend-register~ (core-reason.lisp:18-19), update ~backend-cascade-call~ to check only one hash table.

*** TODO Rename 6 core files — shorter, clearer names
:PROPERTIES:
:ID:       id-v050-reorg-core-names
:CREATED:  [2026-05-07 Thu]
:END:

Rename mapping:
- ~core-defpackage~ → ~core-package~
- ~core-communication~ → ~core-transport~
- ~core-loop~ → ~core-pipeline~
- ~core-loop-perceive~ → ~core-perceive~
- ~core-loop-reason~ → ~core-reason~
- ~core-loop-act~ → ~core-act~

Update: ASDF ~:components~, all ~:tangle~ headers in ~.org~ files, cross-file references, ~README.org~, ~ARCHITECTURE.org~, ~AGENTS.md~, ~*dispatcher-protected-paths*~ (wildcard ~core-*~ still matches — no change needed).

*** TODO Rename 13 system-* → symbolic-/neuro-/embedding-*
:PROPERTIES:
:ID:       id-v050-reorg-system-names
:CREATED:  [2026-05-07 Thu]
:END:

Rename mapping:
- ~system-config~ → ~symbolic-config~
- ~system-diagnostics~ → ~symbolic-diagnostics~
- ~system-archivist~ → ~symbolic-archivist~
- ~system-event-orchestrator~ → ~symbolic-events~
- ~system-self-improve~ → ~symbolic-self-improve~
- ~system-context-manager~ → ~symbolic-scope~
- ~system-memory~ → ~symbolic-memory~
- ~system-model-provider~ → ~neuro-provider~
- ~system-model-router~ → ~neuro-router~
- ~system-model-explorer~ → ~neuro-explorer~
- ~system-model-embedding~ → ~embedding-backends~
- ~system-model-embedding-native~ → ~embedding-native~
- ~system-actuator-shell~ → ~channel-shell~

*** TODO Delete ~system-model.lisp~ (16-line wrapper)

The file delegates to ~*probabilistic-backends*~ — dead code. No skill references it directly.

*** TODO Rename 4 gateway-* → channel-*
:PROPERTIES:
:ID:       id-v050-reorg-channel-names
:CREATED:  [2026-05-07 Thu]
:END:

Rename mapping:
- ~gateway-cli~ → ~channel-cli~
- ~gateway-tui-main~ → ~channel-tui-main~
- ~gateway-tui-model~ → ~channel-tui-state~
- ~gateway-tui-view~ → ~channel-tui-view~

Update TUI package name: ~passepartout.gateway-tui~ → ~passepartout.channel-tui~.

*** TODO Split ~gateway-messaging~ → 4 ~channel-*~ files
:PROPERTIES:
:ID:       id-v050-reorg-messaging-split
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: ~gateway-messaging.lisp~ (411 lines) bundles 4 independent platforms. A Telegram fix shouldn't touch Signal/Discord/Slack code. Each platform becomes its own skill — independently loadable, hot-reloadable, self-repairable.

- ~channel-telegram~: poll + send via Telegram Bot API. ~register-actuator :telegram~.
- ~channel-signal~: poll + send via ~signal-cli~ subprocess. ~register-actuator :signal~.
- ~channel-discord~: WebSocket events + REST POST. Replace hardcoded channel IDs with env vars. ~register-actuator :discord~.
- ~channel-slack~: Events API + ~chat.postMessage~. Replace hardcoded channel IDs. ~register-actuator :slack~.
- Delete ~gateway-messaging.lisp~. Update ~DEFSKILL-FROM-ORG~ references in ~system-config~ setup wizard.

*** TODO Document core/non-core self-repair criterion
:PROPERTIES:
:ID:       id-v050-reorg-docs
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The criterion is the architectural foundation for every discussion about "should this be core or a skill?" It must be documented where developers look.

- New section in ~docs/ARCHITECTURE.org~: "What Makes Core Different — The Self-Repair Criterion." Explain: core = can't self-repair when corrupted, needs human. Skill = agent degrades but self-repairs.
- Include the dependency-chain analysis: which files block self-repair.
- New section in ~docs/DESIGN_DECISIONS.org~: "The Self-Repair Criterion for Core Files." Explain why ~core-context~ and heartbeat were extracted.
- Update ~README.org~ architecture summary to reflect new file map.

*** TODO Update all cross-references after reorg
:PROPERTIES:
:ID:       id-v050-reorg-crossref
:CREATED:  [2026-05-07 Thu]
:END:

After all renames complete, update every remaining reference:
- ~passepartout.asd~: remove ~core-context~, rename 6 core entries.
- All ~#+PROPERTY: header-args:lisp :tangle ../lisp/<old>.lisp~ lines in ~.org~ files.
- All ~in-package~ / ~find-package~ / ~fboundp~ references to renamed packages.
- ~skill-initialize-all~ / ~context-skill-source~: resolve org files under new names.
- ~README.org~: Current Capabilities table, pipeline description, file references.
- ~ARCHITECTURE.org~: layer tables, pipeline flow, dispatcher gate stack.
- ~AGENTS.md~: Project Structure section, file path references.
- ~.env.example~: remove stale ~SAFETY_BLOCK_SHELL~ (unused), update skill paths if any.
- ~ROADMAP.org~: update v0.4.2 and v0.4.3 TODOs (system-model-provider → neuro-provider, core-loop-reason → core-reason, system-actuator-shell → channel-shell) to match new names.

*** Verify: ASDF compiles, FiveAM suite passes, integration tests pass.

*** Token Economics (foundation complete — now build features)

**Design insight: why token economics is the structural differentiator.** Passepartout's sparse-tree rendering and deterministic safety gates should produce 2–3x fewer tokens than competitors for equivalent coding tasks, and 13–24x fewer for knowledge management. But without caching and budget enforcement, the fixed overhead per call eats these savings. A coding session that touches 30 files with competent context management costs ~72K tokens (Passepartout) versus ~185K (Claude Code). Without caching, the Passepartout number climbs toward ~150K because every call retransmits the static prefix. The architectural advantage exists in theory but requires operational plumbing to materialize.

*** TODO Tokenizer integration
- Integrate a tokenizer for at minimum the model families used in the provider cascade (cl100k_base for OpenAI, claude-3 tokenizer for Anthropic). Options: FFI binding to tiktoken via CFFI, or a pure-Lisp port of the BPE tokenizer for cl100k_base (the encoding table is ~100KB, the algorithm is ~100 lines).
- Expose ~(count-tokens text &key model)~ as a core utility.
- Use for three purposes: context budget enforcement (reject assembly if over limit), cost estimation (tokens × provider price), and prompt optimization (measure which sections of the system prompt consume the most budget).

*** TODO Prompt prefix caching
- Split the system prompt into a static prefix (IDENTITY string, TOOLS section, LOGS format header) and a dynamic suffix (CONTEXT render, current log entries, standing mandates, user prompt).
- Track a hash of the static prefix; only retransmit when it changes (skill load/unload, identity config change). On cache hit, send the cached prefix with the dynamic suffix appended.
- Implement the Anthropic prompt-caching header protocol for providers that support it (claude-3-* models, up to 90% discount on cached tokens). For OpenAI, the automatic caching layer handles prefix detection without explicit headers.
- Log cache hit/miss rate to telemetry for cost tracking.

*** TODO Incremental context assembly
- Cache the last rendered ~context-awareness-assemble~ string with metadata: foveal-id at render time, scope, last memory modification timestamp.
- On ~think()~ invocation: if foveal-id, scope, and memory-modification-timestamp are unchanged since the cached render, return the cached string. This eliminates re-rendering on heartbeat ticks, tool-output feedback loops, and multi-turn conversations where the user hasn't changed focus.
- Invalidate the cache on any ~ingest-ast~ call, any ~org-modify~, or any focus change.
- For heartbeats specifically: skip context assembly entirely — the heartbeat sensor bypasses the reason gate (returns early in ~loop-gate-reason:154~), so building awareness for a signal that won't call the LLM is pure waste. Add an early return in ~think()~ for ~:heartbeat~ / ~:delegation~ sensors.

*** TODO Per-call token budget
- ~CONTEXT_MAX_TOKENS~ env var (default: 16384, half of a 32K context window to leave room for model response).
- In ~think()~: compute total token count (static prefix + dynamic context + user prompt). If over budget, progressively trim: first truncate system logs to 5 lines, then drop skill augments from non-triggered skills, then if still over, downgrade peripheral nodes to title-only (disable ~:foveal-vector~ path, render strict depth ≤ 2).
- Log budget violations to telemetry with the trimmed-token count for diagnostics.
- The goal: Passepartout never silently exceeds a model's context window. Silent truncation by the model API produces undefined behavior (mid-thought cutoff, lost instructions). A system that knows it's over budget can degrade intentionally.

*** TODO Cost tracking
- Per-provider pricing lookup table: input/output token costs for each model in the provider cascade (gpt-4o-mini, claude-3-5-sonnet, deepseek-chat, llama-3.1-70b, groq-llama, etc.).
- After each ~backend-cascade-call~: compute cost as (input_tokens × input_price + output_tokens × output_price), log to session accumulator, emit ~:cost-update~ telemetry event.
- Per-session cumulative cost stored in memory (~*session-cost*~ plist: ~(:total <float> :by-provider <alist> :by-task <alist>)~).
- TUI status bar shows current session cost (optional, off by default, toggled via ~/cost~ command). The cost counter renders as ~[Session: $0.12]~ in the status bar, updating after each ~backend-cascade-call~. Color: green when under 50% of daily budget, yellow at 50-90%, red above 90%.
- ~COST_BUDGET_DAILY~ env var with soft cap — warning injected into system prompt when approaching budget, HITL gate on any single action exceeding 25% of remaining budget.

**** TODO Self-configuring setup binary

Rationale: The current ~passepartout configure~ flow is a bash script that detects Debian or Fedora, installs packages, installs Quicklisp, tangles Org sources, and runs the setup wizard. It handles 2 distro families. It fails on everything else. A self-configuring setup with a small LLM expands coverage to "anything with a package manager" without shipping gigabytes of model data. The key constraint: the LLM follows a decision tree for setup, it does not improvise. This keeps setup reliable while expanding coverage.

- The setup binary (~passepartout-setup~) is a ~save-lisp-and-die~ executable (~100MB: SBCL runtime + core Lisp code + native embedding inference from v0.4.0 + 23MB embedding model). No SBCL install required. No Quicklisp. No bash script. The user runs one file.
- Deterministic path (default, always runs first): the same distro detection, package installation, and configuration logic from today's bash script, reimplemented in Lisp. Handles Debian and Fedora families. Covers the common case without touching an LLM.
- LLM-assisted path (optional, activates on deterministic failure): downloads Qwen2.5-0.5B (~500MB GGUF, pinned by hash, cached to ~~/.local/share/passepartout/models/~). The model reads command output, classifies success/failure/recoverable-error from a finite set of outcomes, and selects the next corrective action from a constrained decision tree. On unrecognized failures, generates a diagnostic for the user.
- Model hash verification: the GGUF file is pinned by SHA-256 hash. If the hash doesn't match (wrong version, corrupted download), fall back to deterministic setup with a warning. The bootstrap tool must not fail silently because of a model mismatch.
- After setup completes, the binary exits. The user runs ~passepartout daemon~ to start the full system (a live SBCL process, not a sealed binary — REPL, hot-reload, self-modification all available).
- The setup binary is a bridge. It gets the system installed and configured, then gets out of the way. The final system is a live Lisp image, not a sealed binary.
- Add FiveAM test: the deterministic path succeeds on a system with all dependencies pre-installed; the LLM-assisted path correctly classifies 10 common package-manager error messages.

*** TODO Resolution budget

Rationale: Without cost-aware routing, every request goes through the full deliberative pipeline. A "what's my calendar?" query costs the same overhead as a multi-file refactor. The resolution budget prevents the agent from spending $5 of compute on a $0.01 question.

- Lightweight pre-routing layer classifies complexity before the Reason stage: simple lookup (deterministic, 0 LLM tokens), standard interaction (cached context, tiered model), deep reasoning (full deliberative path with all gates).
- Simple lookups take the fast path: query memory, check file, list TODOs — all in-process function calls, no LLM invocation, sub-second response.
- Tasks exceeding their assigned complexity budget are flagged, reclassified by the tier router, or escalated to the user with a cost estimate.
- The resolution budget is a skill — reloadable, tunable per user preference (~RESOLUTION_BUDGET~ env var with per-tier caps).
- This complements the tier classifier (v0.8.1) which handles safety routing. The resolution budget handles cost routing. Together, cheap simple tasks take the cheap fast path, dangerous complex tasks take the expensive safe path.

*** TODO Symbolic induction

Rationale: The Dispatcher currently learns from blocked and approved actions — it accumulates rules about what to allow and what to deny. Symbolic induction extends this: the agent extracts patterns from successful interaction sequences and converts them into reusable Lisp functions. When the agent successfully completes a multi-step task (e.g., "find all TODOs tagged @urgent, sort by deadline, and create a summary"), it extracts the interaction pattern as a ~defun~ that replaces future LLM calls for similar tasks. This is the mechanism by which the system genuinely needs the LLM less over time — not just by blocking fewer dangerous actions, but by replacing probabilistic reasoning with deterministic functions. The Dispatcher learns what to prevent. Symbolic induction learns what to automate.

- Scan successful interaction sequences (user request → agent actions → successful outcome) and extract reusable patterns: what was asked, what tools were called, what the verification chain looked like, and what the final result was.
- When a pattern repeats across 3+ sessions with consistent outcomes, trigger induction: the LLM proposes a Lisp function implementing the pattern, the REPL verifies it against historical inputs, and if it passes, the function is registered as a skill.
- Induced functions live in ~passepartout.skills.induced-<name>~ — jailed packages, same loading sandbox as user-written skills. They can be inspected, modified, or removed by the user.
- The rule counter in the TUI status bar gains a second counter: ~[Rules: 47 | Induced: 12]~ — rules learned from HITL decisions vs functions learned from successful sessions.
- Induced functions are proposed, not automatically applied. The next time a similar request arrives, the agent checks: "I have an induced function for this. Use it?" The user approves the first invocation, and subsequent invocations of the same function are automatic.
- Add FiveAM test: replay a historical interaction sequence, verify the induced function produces the same outcome.

*** TODO TDD workflow skill — language-agnostic test runner
:PROPERTIES:
:ID:       id-v050-programming-tdd
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The REPL-TDD-Literate workflow described in AGENTS.md lives entirely outside the agent's cognitive loop. The agent should be able to write tests, run them, observe red/green, and iterate — without the user manually managing the cycle. This is the Lisp advantage made operational: redefine a function, re-run a single test, get results in <100ms. Claude Code cannot do this — it has no REPL. The skill is language-agnostic: it dispatches to the REPL skill for Lisp, shells out to ~pytest~ for Python, ~go test~ for Go, etc.

- New skill: ~programming-tdd.org~. Depends on REPL skill for Lisp, falls back to shell for other languages.
- Cognitive tools: ~deftest~ (define a test), ~run-test~ (run a specific test), ~list-tests~ (list all defined tests).
- ~run-test~ dispatches on ~:language~ parameter:
  - ~:lisp~ → ~(fiveam:run 'test-name)~ via REPL eval
  - ~:python~ → shell ~python3 -m pytest test_file.py::test_name~
  - ~:go~ → shell ~go test -run TestName ./...~
  - ~:rust~ → shell ~cargo test test_name~
  - ~:default~ → shell command template from env ~TEST_RUNNER_<LANG>~
- The TDD loop: write test → ~run-test~ (expect RED) → write implementation → ~run-test~ (expect GREEN) → report.
- ~#+DEPENDS_ON: org-skill-utils-repl~ for Lisp TDD; no dependency for other languages (shell fallback).
- FiveAM tests: ~run-test~ on a known-failing test returns RED status; ~run-test~ on a known-passing test returns GREEN.

*** TODO Expand literate programming skill — persist after TDD
:PROPERTIES:
:ID:       id-v050-literate-persist
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: After the TDD loop confirms green, the agent must persist the working code into its Org source file and tangle to ~.lisp~. Currently ~self-improve-edit~ can do surgical text replacement but doesn't integrate with the TDD confirmation step. The literate skill should provide a ~persist-verified-block~ tool that takes TDD-confirmed code and writes it to the appropriate ~#+begin_src lisp~ block.

- Add ~persist-verified-block~ cognitive tool: accepts ~filepath~, ~block-name~, ~code~, ~test-result~. Only writes if ~test-result~ is GREEN.
- Verifies the written Org file passes ~literate-block-balance-check~ before tangling.
- Tangles via existing ~org-tangle-file~.
- FiveAM test: persist a verified block, verify it appears in the tangled ~.lisp~ file, verify the Org file passes balance check.

*** TODO Org-mode productivity additions — agenda, clock, checklist, table
:PROPERTIES:
:ID:       id-v050-org-additions
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: Passepartout bets on Org-mode as the universal format for human and machine. But current Org support is thin: headlines, tags, property drawers, source blocks. Missing are the features that make Org a productivity tool: agenda views, clock-in/out, checklists, tables. Adding these turns the agent from a chat partner into a productivity assistant — it can answer "what should I work on today?" with 0 LLM tokens.

- Extend ~programming-org.lisp~ (~programming-org.org~) with five new functions:
  1. ~org-agenda-today~ — walk memory (or file tree) for headlines with ~SCHEDULED~ ≤ today or ~DEADLINE~ within N days. Returns list of memory-objects. ~60 lines.
  2. ~org-clock-in~ / ~org-clock-out~ — set ~:CLOCK-START~ property; on clock-out, compute duration, append to ~:LOGBOOK:~ drawer. ~80 lines.
  3. ~org-checklist-toggle~ — parse ~- [ ]~ / ~- [X]~ checkboxes in headline content, toggle state, return completed/total count. ~50 lines.
  4. ~org-table-parse~ / ~org-table-render~ — parse ~| a | b |~ tables into list-of-lists, render back. ~70 lines.
  5. ~org-agenda-view~ — compose agenda + clock state + TODO headlines into single Org-formatted string. Used by ~/agenda~ TUI command. ~50 lines.
- ~org-agenda-today~ and ~org-agenda-view~ operate on memory store (zero file I/O, zero LLM tokens).
- FiveAM test for each function.

*** TODO Vault encryption — Ironclad AES + PBKDF2
:PROPERTIES:
:ID:       id-v050-vault-encryption
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The vault (~*VAULT-MEMORY*~) stores API keys and credentials in plaintext in a hash table. ~VAULT-MASK-STRING~ always returns ~"[MASKED]"~ ignoring input — it's a stub. Ironclad is already a dependency (used for SHA-256 in Merkle hashing) and provides AES-256-GCM, ChaCha20, and PBKDF2. Encryption makes the vault go from security theater to actual security.

- Add ~vault-encrypt~ / ~vault-decrypt~ using Ironclad AES-256-GCM. Master key derived via PBKDF2 from ~VAULT_MASTER_PASSPHRASE~ env var or ~~/.config/passepartout/.key~ file.
- Store ciphertext instead of plaintext in ~*VAULT-MEMORY*~.
- ~VAULT-MASK-STRING~ actually masks (replaces all chars with ~*~, preserving length).
- ~dispatcher-vault-scan~ searches plaintext after decrypt (still catches leaks before they reach the LLM).
- FiveAM test: round-trip encrypt/decrypt; wrong passphrase fails; masked string has same length as original.

*** TODO Deterministic gate growth — ~dispatcher-learn~ + ~rules.org~
:PROPERTIES:
:ID:       id-v050-dispatcher-learn
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: This is the "cheaper over time" claim made operational. Every HITL approval or denial becomes data. After N approvals of the same pattern, it becomes a permanent deterministic rule. The LLM no longer asks permission. 0 LLM tokens spent on what used to be a human decision. The user watches the rule counter tick up as they teach the agent.

- ~dispatcher-learn~ function in ~security-dispatcher.lisp~: called from ~hitl-approve~ and ~hitl-deny~. Extracts pattern (~:tool~ + ~:filepath~ glob + ~:cmd~ pattern). Tracks count per pattern in memory store.
- When count passes ~DISPATCHER_RULE_THRESHOLD~ (from ~.env~, default 3), writes a rule to ~RULES_FILE~ (~~/memex/system/rules.org~).
- Each rule is an Org headline with ~:EXPLANATION:~ property explaining what the rule does and why it was created.
- ~dispatcher-check~ consults ~RULES_FILE~ before its blocking vectors — allowed rules pass through, blocked rules are denied.
- Rules are loaded from ~rules.org~ at daemon startup (survive restarts).
- ~dispatcher-severity-allowed-p~: uses severity classification from v0.4.3 — ~:catastrophic~ always HITL regardless of rule count. ~:harmless~ always allowed.
- Severity thresholds: ~:dangerous~ = 5 approvals, ~:moderate~ = 3 approvals (configurable via ~.env~).
- ~DISPATCHER_RULE_THRESHOLD~ and ~RULES_FILE~ env vars already added in v0.4.1's NO-HARDCODED-CONSTANTS TODO.
- ~DISPATCHER_SEVERITY_DANGEROUS_THRESHOLD~ and ~DISPATCHER_SEVERITY_MODERATE_THRESHOLD~ in ~.env.example~.
- FiveAM test: approve same pattern 3 times → rule appears in ~rules.org~ → pattern passes through ~dispatcher-check~ without approval.

*** TODO Rule visibility — TUI ~/rules~ commands
:PROPERTIES:
:ID:       id-v050-rule-visibility
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The user must know what rules the Dispatcher has learned and must be able to undo bad learning. The rules live in ~~/memex/system/rules.org~ (editable in any text editor), but the TUI should provide live access.

- TUI commands:
  - ~/rules~ — list all rules sorted by recency (most recent first). Shows pattern, decision (allowed/blocked), severity, approval count, explanation.
  - ~/rules blocked~ — show only blocked patterns.
  - ~/rules allowed~ — show only allowed patterns.
  - ~/rule delete <id>~ — remove a rule (undoes the learning). Deletes the headline from ~rules.org~.
  - ~/rule allow <id>~ — flip a blocked rule to allowed (user overrides the learning).
- On rule creation, daemon sends ~:rule-created~ event. TUI adds system message: ~[Rules: 47 → 48] New rule: shell commands targeting ~/memex/projects/* are now allowed. /rule delete rule-48 to undo.~
- Rules are visible in the TUI status bar via the rule counter (already implemented in v0.4.0 gate trace).
- FiveAM test: ~/rules~ returns expected rules; ~/rule delete~ removes a rule and it no longer passes through ~dispatcher-check~.

*** TODO Merkle learning — memory-find-similar, outcome recording
:PROPERTIES:
:ID:       id-v050-merkle-learning
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The Merkle tree provides content-addressed storage. Combined with embedding vectors (populated at ingest time since v0.4.0), it can answer "what happened the last 3 times I asked something like this?" This is retrieval-augmented generation from the user's own history — the agent learns what approaches succeeded and failed, not from the LLM's training data but from the user's actual sessions.

- ~memory-find-similar~ in ~core-memory.lisp~: given a vector, return N memory objects with highest cosine similarity. Uses ~memory-object-vector~ (already populated via ~ingest-ast~ → ~embeddings-compute~ since v0.4.0). ~30 lines.
- ~memory-outcome-record~: store an outcome (success/failure plist) against a signal. Keyed by Merkle hash of the signal. ~25 lines.
- ~memory-find-outcomes~: given a signal (current context), find similar past signals and their outcomes. Uses ~memory-find-similar~ on the signal's foveal vector. Returns ranked list of past approaches with success/failure labels. ~40 lines.
- Outcome data feeds into ~symbolic-awareness~ (formerly core-context, extracted from core): when the foveal node has similar past interactions, include them in the context as "Historical: last 3 times you asked this, approach X succeeded, Y failed."
- FiveAM test: record 3 outcomes for similar signals, verify ~memory-find-outcomes~ returns them ranked by similarity.

*** TODO Merkle learning documentation in Design Decisions
:PROPERTIES:
:ID:       id-v050-merkle-docs
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The Merkle tree was designed for integrity, not learning. Its second life as a learning substrate — content-addressed history + vector similarity → retrospective knowledge — deserves architectural documentation explaining the data flow, the similarity gating, and how it feeds the "cheaper over time" thesis.

- New section in ~docs/DESIGN_DECISIONS.org~: "The Merkle Tree as Learning Substrate."
- Explain: Merkle hash → content identity. Memory-object-vector → content similarity. Together → "find what worked last time."
- Include data flow diagram (ASCII art) showing ingest → embed → query → retrieve → inform cycle.
- Distinguish from symbolic induction (v0.5.0): Merkle learning answers "what happened last time?" Symbolic induction answers "can I automate this next time?"

*** TODO Internal evaluation harness — ~deftask~, ~run-eval-suite~
:PROPERTIES:
:ID:       id-v050-eval-harness
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: Without an evaluation harness, there is no way to know if the agent's capabilities improve or regress across releases. SWE-bench (v0.9.0) measures competitive ranking against other agents. The internal suite measures regression detection — it catches when v0.5.1 breaks something v0.5.0 could do. The suite starts with 10 tasks and grows with the codebase.

- New skill: ~symbolic-evaluation.org~ (~symbolic-evaluation.lisp~).
- ~deftask~ macro: define an eval task with ~:setup~ (create test environment), ~:prompt~ (what to ask the agent), ~:verify~ (function that checks the output), ~:teardown~ (cleanup). Similar to ~defskill~ but for agent capabilities, not code.
- ~run-eval-task~: inject ~:prompt~ as ~:user-input~ signal via ~stimulus-inject~, wait for completion (poll ~*memory-store*~ or signal status), run ~:verify~ on the result, return ~(:passed)~ or ~(:failed :reason ...)~.
- ~run-eval-suite~: run all registered eval tasks, produce score (pass count / total), per-task diagnostics, summary.
- ~eval-score~: return current score as a number. Logged to telemetry.
- Initial 10 tasks covering: find TODOs, create Org note, modify file, search codebase, run shell command (safe), list projects, query memory, find definition, run test, set TODO state.
- Task suite grows with codebase: every bug fix adds a regression task. Every new feature adds a capability task.
- FiveAM test: a task that should pass passes; a task that should fail fails with the expected reason.

*** TODO Evaluation workflow in AGENTS.md
:PROPERTIES:
:ID:       id-v050-eval-agentsmd
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: The AGENTS.md "Development Workflow" section describes how to develop code with REPL → TDD → Literate. A parallel "Evaluation Workflow" section should describe how to verify agent capabilities with eval tasks. Together they form the full quality cycle: TDD verifies the code the agent writes, eval verifies the agent itself.

- New section in AGENTS.md: "## Evaluation Workflow (Must Follow)".
- Mirror the Development Workflow structure: define task → prove BLANK (fresh agent fails) → implement capability → prove COMPLETE → track regression.
- Include ~deftask~ example and ~run-eval-suite~ usage.
- Rule: every new cognitive tool or skill MUST include an eval task before shipping.

*** TODO TDD + Eval + Merkle learning integration into ~.env.example~
:PROPERTIES:
:ID:       id-v050-env-vars
:CREATED:  [2026-05-07 Thu]
:END:

Rationale: All new configurable values from v0.5.0 must be documented in ~.env.example~ per the NO-HARDCODED-CONSTANTS standard (v0.4.1). This task ensures no env var is forgotten.

- Add to ~.env.example~:
  - ~DISPATCHER_RULE_THRESHOLD=3~ (if not already added in v0.4.1 cleanup)
  - ~RULES_FILE="$HOME/memex/system/rules.org"~
  - ~DISPATCHER_SEVERITY_DANGEROUS_THRESHOLD=5~
  - ~DISPATCHER_SEVERITY_MODERATE_THRESHOLD=3~
  - ~VAULT_MASTER_PASSPHRASE=""~ (empty = prompt on startup, or read from ~/.key file)
  - ~EVAL_TASKS_DIR="$HOME/memex/system/eval/"~
  - ~EVAL_TIMEOUT=120~ (seconds before a task is considered failed)
  - ~TEST_RUNNER_PYTHON="python3 -m pytest"~
  - ~TEST_RUNNER_GO="go test -run"~
  - ~TEST_RUNNER_RUST="cargo test"~
- Document each with a comment explaining its purpose and default.

*** Competitive Advantage Analysis — v0.5.0 Summary

Token economics is the dimension where the architecture's theoretical advantage becomes operationally real. The foveal-peripheral model and deterministic gates reduce the tokens *needed* per task; prompt caching and incremental assembly reduce the tokens *spent* per task. Combined, the 2–3x coding savings and 13–24x knowledge management savings in the DESIGN_DECISIONS token analysis become achievable rather than aspirational. Symbolic induction extends this downward cost curve into new territory: the agent doesn't just block fewer dangerous actions — it automates away entire categories of LLM calls by learning reusable Lisp functions from successful interaction patterns.

The cost tracking and budget enforcement are defensive advantages: no competitor gives the user visibility into per-task LLM cost. Claude Code and Copilot obscure cost behind flat-rate subscriptions. Passepartout's transparent cost model is a sovereignty feature — the user knows what the agent spends on their behalf and can cap it.

The minimum viable local model advantage is structural: at 2,000–4,000 effective tokens (foveal-peripheral + caching), a 7–8B parameter model on consumer hardware is a daily driver. Competitors at 32K+ effective tokens require 70B+ parameter models and 16–32 GB VRAM. Passepartout runs on a laptop GPU where competitors need a data center card or cloud API.

** v0.6.0: Signal Pipeline, Concurrency & Streaming

The current pipeline is strictly sequential — one signal traverses Perceive → Reason → Act before the next signal begins. Background tasks (heartbeat, embedding cron, gardener scans) compete with foreground interactions. A heartbeat that fires during a long tool chain is queued. A Telegram message during a multi-step planning cycle is queued. The system feels sluggish under concurrent load even though the symbolic operations are near-instant (SBCL hash table lookups are microseconds) — the bottleneck is the single-pipeline architecture, not the hardware.

*Design insight: why concurrency matters for an agent that is "one brain."* Passepartout rejects multi-agent delegation on principle (see DESIGN_DECISIONS "One Single Agent"). But a single brain handles multiple inputs simultaneously — the human brain processes vision, audio, and proprioception in parallel. Rejecting multi-agent delegation does not require rejecting concurrency within the agent. The key is that all concurrent operations share the same memory space, the same Merkle tree, and the same deterministic gate stack. They are threads of one cognition, not separate agents.

*** TODO Priority-queue signal processing
- Replace the linear ~process-signal~ call chain with a priority-ordered signal queue. The queue is a sorted plist-list consumed by the main loop. Priority tiers:
  - ~:user-input~ / ~:chat-message~ — highest priority (the user is waiting)
  - ~:approval-required~ — high (HITL re-injections need quick resolution)
  - ~:tool-output~ — medium (feedback from tool execution, needs LLM assessment)
  - ~:interrupt~ — medium-high (shutdown signal)
  - ~:heartbeat~ / ~:cron~ / ~:delegation~ — low (background maintenance)
- Coalesce duplicate heartbeats: if the queue already contains a ~:heartbeat~ signal when a new one arrives, discard the older one (no value in processing stale ticks). Keep at most one pending heartbeat at any time.
- The main loop drains the highest-priority signal from the queue, processes it through the pipeline, and repeats. If the pipeline produces feedback (tool-output → think), the feedback is enqueued at its appropriate priority — it may preempt background signals but won't interrupt the current signal mid-processing.
- Add telemetry: average queue depth by priority tier, max wait time per tier.
- TUI ~/reconnect~ command: when the connection-loss detection from v0.3.3 fires, the user can reconnect without restarting the TUI. The command closes the stale socket, re-runs ~connect-daemon~ with its retry backoff, and restores the ~:connected~ state on success.

*** TODO MVCC memory concurrency
- Replace ~*memory-store*~ (mutable global hash table) with a versioned Merkle-root pointer. The root is an ~(or null merkle-node)~ struct containing the tree and a monotonic version counter.
- Read threads snapshot the root before beginning their pipeline cycle. All object lookups dereference through the snapshot — they see a consistent view of memory regardless of concurrent writes. Reads never block.
- Write threads (ingest-ast, org-modify, snapshot-memory) build new object hashes, construct a new Merkle root, and CAS-replace the global root pointer. If another thread won the CAS race (root version changed), the loser re-reads the new root, replays its changes on the updated tree, and retries the CAS.
- Conflict probability is near-zero because concurrent signals almost never touch the same Org headline. The replay-on-conflict path exists for correctness but is rarely exercised. Lock contention is eliminated — the only atomic operation is the CAS on the root pointer.
- Remove the single-threaded pipeline assumption: previously, ~process-signal~ was safe because nothing else wrote to ~*memory-store*~ during its execution. With MVCC, multiple signals can process concurrently because each has its own snapshot. The ~*loop-interrupt-lock*~ becomes ~*signal-queue-lock*~ (protecting only the queue, not the memory).
- Test: concurrent ingest-ast from two threads writing to different memory objects, verify both commits succeed without corruption.

*** TODO Structured output enforcement
- Add a plist validation step between ~markdown-strip~ and ~read-from-string~ in ~think()~. Before attempting to parse, validate: (a) the output starts with ~(~ or ~[~, (b) it contains balanced delimiters (count opens vs closes), (c) it doesn't contain ~#.~ (redundant after v0.3.1 ~*read-eval* nil~ but defense-in-depth).
- On validation failure: construct a rejection trace (similar to the existing deterministic gate rejection feedback) and re-inject into the LLM prompt. The trace includes the raw output and a diagnostic ("Your response did not produce a valid plist. Ensure it starts with ( and has balanced parentheses.").
- Configurable ~LLM_OUTPUT_RETRIES~ (default 2). After exhausting retries, fall through with the raw text as a ~:MESSAGE~ action (current behavior).
- Track parse-failure rate per provider in telemetry. Use to guide provider cascade ordering: a provider with 20% parse-failure rate falls behind one with 2%.
- If retries are exhausted without a parseable plist, the TUI renders the raw LLM output in a dimmed, collapsible region labeled "Parse failure — could not interpret this response." The user can inspect what the model produced.

*** v0.6.3 — TODO Streaming responses

Rationale: Every competitor streams — Hermes Agent specifically lists "streaming tool output" as a feature, OpenClaw streams via messaging channels, Claude Code streams via terminal. A spinner followed by a wall of text is v0.1-era UX for an LLM chat interface. Streaming was originally sequenced in the evaluation release (after evaluation harness and computer use), but it depends only on the daemon protocol (chunked frames) and TUI rendering — neither require tools, planning, evaluation, or vision. Moving it to v0.6.3 means Passepartout streams before it ships tools, because streaming makes the existing chat experience competitive.

- Add a new frame type (~:type :stream-chunk~) to the daemon-TUI protocol. Chunks are variable-length strings carrying partial LLM output. The final chunk is an empty string, signalling end-of-stream.
- ~provider-openai-request~: for providers that support streaming (OpenRouter, OpenAI, Anthropic, Groq, local), send ~"stream": true~ in the request body. Read the SSE stream, extract ~delta.content~ from each chunk, and call a new ~*stream-callback*~ function with the partial text.
- The TUI renders partial output in the chat window as it arrives, appending characters to the in-progress agent message. The "…thinking" spinner is replaced by live, building text.
- Interrupt-and-redirect: the user pressing a key (Esc or any printable char) during streaming injects an interrupt signal. The partial response is captured as the agent's message, the LLM call is cancelled (HTTP connection closed), and the user's keystroke becomes new input. This replaces the current full-process ~SIGINT~ with a graceful mid-response redirect.
- The TUI message for a streamed response shows a ~[streaming]~ indicator that changes to a timestamp when the stream completes. If interrupted, the indicator changes to ~[interrupted]~.
- Add FiveAM tests: stream-chunk framing round-trips correctly; interrupt during streaming produces a valid partial message; the TUI correctly renders progressive chunks vs a completed message.

*** Competitive Advantage Analysis — v0.6.0 Summary

The priority queue eliminates the perception of sluggishness that concurrent load creates. A user typing a query never waits for a heartbeat tick to finish — their signal jumps the queue. The coalescing of duplicate heartbeats eliminates wasted processing. This is table-stakes UX for a daily-driver agent.

MVCC concurrency on the Merkle tree is genuinely novel for an AI agent. Most agents use either a single-threaded event loop (Claude Code) or process-level isolation (OpenClaw's subprocess model). Passepartout's approach — concurrent threads sharing a versioned content-addressable tree — combines the coherence of a single-agent memory with the throughput of concurrent execution. The Merkle tree, originally designed for integrity verification, gets a second life as the concurrency control primitive. This is the kind of architectural synergy that single-purpose databases can't match.

Structured output enforcement bridges the gap between "Passepartout uses plists, not JSON" and "LLMs sometimes produce malformed syntax." It gives the system the same reliability guarantee that JSON mode gives competitors — the output will parse — without introducing JSON into the architecture.

Streaming responses (v0.6.3) close the last remaining table-stakes UX gap with Hermes Agent and Claude Code. The "…thinking" spinner is replaced with live text. Interrupt-and-redirect means the user can course-correct mid-response instead of waiting for a wrong answer to complete. Combined with the TUI critical fixes (v0.3.3) and differentiator visualizations (v0.4.0), the TUI is competitive on responsiveness and uniquely informative on safety and context transparency.

** v0.7.0: Tool Ecosystem (MCP-Native) + Voice Gateway

The original roadmap placed MCP at v0.8.0 and planned "10+ cognitive tools" built from scratch for v1.0.0. This is inverted: the ecosystem already provides 50+ tools (filesystem, git, postgres, slack, github, web search, memory servers). Building bespoke tools from scratch duplicates work the community has already done and tested. Passepartout's advantage is not in tool *implementation* but in tool *orchestration* — the deterministic gate stack that verifies every tool invocation before execution.

*Why MCP matters for competitive positioning:* Claude Code's native tools (Read, Write, Edit, Bash, Grep, Glob, WebSearch) are implemented in TypeScript within the Claude Code runtime. They are not extensible — you cannot add a tool without modifying the runtime. OpenClaw's tools are similarly baked into the Node.js process. By building a native MCP client, Passepartout gains tool breadth that exceeds both competitors (50+ tools via the MCP ecosystem versus ~10 native tools) without building a single tool implementation. The tool quality is maintained by the ecosystem; the safety verification is maintained by Passepartout's gate stack. This division of labor is the right architecture for a small team building a competitor to well-funded commercial agents.

*** TODO MCP native client
- Pure Common Lisp MCP client: parse JSON-RPC messages from MCP servers over stdio or SSE. No Python bridge, no Node.js subprocess. The client runs in the same Lisp image as the agent — zero serialization overhead between the agent and the MCP layer.
- Implement the MCP protocol lifecycle: initialize handshake, list tools, call tool, handle notifications. Each MCP server registers its tools as entries in Passepartout's ~*cognitive-tool-registry*~ at connection time — the LLM's tool belt prompt automatically expands to include them.
- ~MCP_SERVERS~ env var: comma-separated paths to MCP server config files (JSON). Each config specifies the server command, args, and env vars. Example: =MCP_SERVERS=~/.config/passepartout/mcp/filesystem.json,~/.config/passepartout/mcp/git.json=.
- Tool invocation route: LLM proposes a tool call → Dispatcher verifies against permission table → MCP client serializes call as JSON-RPC → server executes → result deserialized back to plist → returned to LLM as tool output. The Dispatcher does not distinguish between native tools and MCP tools — the gate stack is uniform.
- Register the MCP client as a skill (~defskill~~:passepartout-mcp-client~) so it can be hot-reloaded. The MCP client is not core infrastructure — it is a skill that extends the tool ecosystem.

*** TODO Core MCP tools (from existing roadmap items)
- Git Steward (deferred from old v0.5.0): status, diff, commit, push, branch via the MCP Git server. Policy gate enforces commit-before-modify: any file write to a git-tracked directory must be preceded by a diff review.
- Web Research (deferred from old v0.7.0): headless browser via Puppeteer/Playwright MCP server. Text extraction, screenshot capture, page interaction.
- Interactive PTY (deferred from old v0.6.0): stream long-running process output to context window, async interrupt control.

*** TODO TUI tool visualization
- Tool invocation rendering: when the agent invokes a tool, the TUI renders a color-coded, collapsible region. Pre-execution: ~[Running: bash "npm test"...]~ in magenta with a dim spinner. Post-execution: ~✓ bash: tests passed (1.2s)~ in green, or ~✗ bash: exit code 1~ in red with the error output expanded below.
- Tool output is collapsed by default (single line summary). Pressing Enter on a tool invocation row toggles expansion to show the full output.
- Diff display: when a file write or git diff is involved, render the diff with standard ~+~ (green) / ~-~ (red) coloring. The diff is shown as a compact inline block with 3 lines of context around each change.
- Gate trace for tool invocations: each tool call shows its Dispatcher gate results inline (gate trace from v0.4.0), so the user sees both the tool execution and which safety gates allowed or blocked it.

*** TODO Environment Steward
- Detect "command not found" in shell actuator output.
- Search system PATH and package manager registries for the missing command.
- Propose installation command and retry the failed action on user approval.
- Cache resolved dependency paths to avoid repeated searches.

*** v0.7.3 — TODO Voice Gateway

Rationale: OpenClaw ships voice wake words and talk mode on macOS/iOS/Android via ElevenLabs. Hermes Agent has voice memo transcription. Both treat voice as a first-class channel. Passepartout's daemon already handles text — voice is an I/O format conversion. Speech-to-text turns audio into ~:user-input~ signals. Text-to-speech turns agent responses into audio. The architecture requires no changes; the voice gateway is a skill that wraps existing REST APIs.

- Speech-to-text: POST audio to OpenAI Whisper API (~/v1/audio/transcriptions~) or local Whisper via Ollama. Receive text. Inject as a ~:user-input~ signal into the pipeline. The daemon processes it identically to a typed message.
- Text-to-speech: POST text to ElevenLabs REST API (~/v1/text-to-speech/{voice-id}~) with stream response. Also support system ~say~ (macOS) / ~espeak~ (Linux) as zero-dependency fallbacks.
- TUI voice toggle: ~/voice on~ enables voice capture, shows a ~🎤~ (listening) indicator in the status bar. ~/voice off~ returns to text-only. The microphone capture runs in a dedicated thread that feeds audio chunks to the speech-to-text backend.
- Voice mode in messaging gateways: on Telegram and Discord, the voice gateway transcribes voice messages into text and injects them as ~:user-input~ signals. Agent responses can be optionally spoken back via text-to-speech if the user's message included a voice note (reply in kind).
- The voice gateway is a skill (~defskill~~:passepartout-gateway-voice~). No core daemon changes required. The daemon receives text signals whether they originated from a keyboard, a messaging app, or a microphone.

*** Competitive Advantage Analysis — v0.7.0 Summary

MCP-native tool architecture gives Passepartout a tool breadth advantage that no single team could achieve through bespoke implementation. The MCP ecosystem is growing faster than any individual agent's tool set. By connecting to it rather than competing with it, Passepartout's tool count scales with the ecosystem — every new MCP server is a new Passepartout tool.

The Dispatcher's tool permission table (allow/ask/deny) applies uniformly to MCP tools, giving Passepartout tool-level security granularity that competitors lack. Claude Code's tools are binary: available or not. Passepartout can conditionally allow filesystem writes to ~/projects/*~ while requiring HITL for writes to ~~/.config/*~ — per-path, per-tool, per-session. This is the deterministic gate stack's natural application domain.

The Git policy gate (commit-before-modify) is a safety feature no competitor provides. It prevents the most common agent failure mode: modifying files without preserving the prior state. Combined with memory snapshots (v0.2.0), this gives every action a dual audit trail: the git history and the memory object history.

v0.7.1 is also the threshold at which Passepartout can safely self-build — modify its own source files outside the core pipeline. The ~core-*~ path protection from v0.4.0 ensures the agent cannot destroy its own brain stem during self-building; the TDD runner catches regressions before commit; the Git policy gate preserves every state change. Together, these four releases (v0.4.0, v0.5.0, v0.6.2, v0.7.1) form the safety, economic, reliability, and tool stack that makes self-hosting viable.

The voice gateway (v0.7.3) adds parity with OpenClaw's voice features without architectural changes — speech-to-text and text-to-speech are thin REST wrappers that feed text signals into the existing pipeline. Combined with the Emacs bridge (v0.4.0) and messaging gateways (v0.4.0), Passepartout supports four interaction surfaces by v0.7.3: terminal (TUI), messaging apps, Emacs, and voice. Each surface is a thin client speaking the same framed TCP protocol to the same daemon.

** v0.8.0: Planning, Self-Modification & Deterministic Routing

*Design insight: the inverted tier classifier.* The current tier classifier routes "rm", "write-file", and "shell" to ~:REFLEX~ (no LLM). This routes the most dangerous operations to the path with the least oversight. It should be inverted: ~:REFLEX~ handles deterministic lookups (list TODOs, check file existence, query memory), ~:COGNITION~ handles text processing and summarization, ~:REASONING~ handles planning and code generation. Dangerous operations should always route through ~:REASONING~ where the full LLM cycle and Dispatcher gate stack apply. v0.8.1 fixes this.

*** TODO Long-horizon planning (task tree DAG)
- Decompose complex tasks into Org-mode headline trees. Each task node is a memory-object with terminal states: ~:todo~ → ~:next-action~ → ~:in-progress~ → ~:done~ / ~:blocked~ / ~:stuck~.
- The LLM generates the initial task tree from the user's request. The REASONING tier processes each leaf task sequentially, updating node states as it progresses.
- Parent nodes summarise child results: when all children of a node reach ~:done~, the parent is promoted to ~:done~ with a synthesised summary. When any child reaches ~:stuck~, the parent is promoted to ~:blocked~ with the blocking child's diagnostic.
- Branch pruning: if a child is ~:stuck~ after three retries with different LLM providers, the parent re-plans the branch — the LLM generates alternative decomposition paths for the blocked sub-task.
- Task trees persist as Org headlines in ~/memex/system/tasks/~. Survive restarts. Visible to the user as editable Org files.
- TUI task tree visualization: a collapsible Org headline tree rendered in the chat area. Each node shows its terminal state with a colored indicator (~○~ todo, ~▶~ next-action, ~◉~ in-progress, ~✓~ done, ~✗~ blocked, ~⏸~ stuck). Nodes expand/collapse on Enter. The tree updates in real time as the agent progresses through subtasks. This is visible in the TUI as an async status region that appears when the agent is executing a long-horizon plan and collapses to a single summary line when complete.

*** TODO Tier classifier fix
- Invert the current classifier: ~:REFLEX~ = deterministic lookups only (memory query, file-exists-p, check time, list TODOs by tag). ~:COGNITION~ = text processing, summarization, simple Q&A, note formatting. ~:REASONING~ = planning, code generation, multi-step task execution, dangerous operations.
- Track classifier accuracy via telemetry: for each classified action, record whether the classification was appropriate (did the ~:REFLEX~ action actually succeed without LLM? did a ~:REASONING~ action turn out to be a simple lookup?).
- The classifier function is overrideable via ~*tier-classifier*~, allowing users or skills to customize routing.
- The classifier should be a skill, not core infrastructure — reloadable and replaceable without restart.

*** TODO Skill Creator
- LLM drafts complete skill org-file from natural language description.
- Mandatory pipeline: (a) syntax validation via ~lisp-syntax-validate~, (b) sandbox-load in temporary jailed package (v0.3.2), (c) run registered trigger function against mock contexts, (d) run registered deterministic gate against mock proposals, (e) on pass, promote to live registry under ~passepartout.skills.<name>~.
- Required ~:repl-verified~ flag on all ~defun~ forms — the existing Dispatcher lint check (core-loop-act.lisp:152–161) warns on writes without verification. The Skill Creator enforces this at creation time.
- Skills are the primary extension mechanism for users. The Skill Creator makes skill authoring accessible to non-Lisp-programmers: describe what you want in English, the LLM drafts the Org file, the system verifies it, and the skill is live. This is how Passepartout grows its capability surface without requiring the user to learn Common Lisp.

*** Competitive Advantage Analysis — v0.8.0 Summary

The task tree DAG with terminal states and branch pruning is Passepartout's planning primitive — analogous to Claude Code's TODO list but structural (Org headlines with parent-child relationships) rather than flat. The advantage: subtask dependencies are explicit in the tree structure, so the agent knows that task C depends on tasks A and B without having to rediscover this from context. Parent summarisation means the LLM can check high-level progress without re-reading every child's output — a token savings multiplier on long-running tasks.

The tier classifier fix is a safety correctness issue. The current inverted classifier (dangerous ops → no-LLM path) is actively harmful — it reduces oversight on the operations that need it most. Fixing this means "dangerous by default → maximal oversight" becomes the routing rule, which is the correct security posture.

The Skill Creator is the mechanism by which Passepartout escapes the "team of Lisp programmers" constraint. Most agent frameworks require Python/TypeScript to extend. Passepartout's extension language is English — the LLM writes the Lisp, the system verifies it. The sandbox-load and verification pipeline (from v0.3.2) make this safe: a skill that fails verification never enters the running image.

** v0.9.0: Evaluation & Vision

With tools (v0.7.0) and planning (v0.8.0) in place, the agent can execute complex multi-step tasks. v0.9.0 answers two questions: (1) how do we *prove* it works? (SWE-bench evaluation harness), and (2) can the agent interact with visual interfaces? (computer use / vision). Streaming has been moved to v0.6.3 — it depends only on the daemon protocol, not on evaluation or vision.

*** TODO SWE-bench harness
- Automated pipeline: clone a repository from SWE-bench dataset, parse the GitHub issue, feed the issue description into Passepartout's cognitive loop, track the resolution trajectory as an Org headline tree, apply the generated patch, run the repository's test suite, score success (tests pass yes/no).
- Trajectory persistence: each benchmark run produces an Org file under ~/memex/system/benchmarks/~ recording every ~think()~ call, every tool invocation, every Dispatcher decision, and the final test result. The trajectory is auditable — a human can read why the agent made each decision and where it went wrong on failures.
- Regression mode: run the same benchmark after each version release. Track score trends. A version that regresses on SWE-bench does not ship.
- Target: competitive score with Claude Code and OpenClaw on SWE-bench-verified by v1.0.0. The evaluation harness ships in v0.9.0 so there are two full version cycles to iterate and improve before v1.0.0 ships.

*** TODO Computer Use / Vision
- Screenshot capture: X11 (~xwd~ / ~import~) and Wayland (~grim~) bridge. The agent requests a screenshot of a specific window or the full desktop.
- Vision model integration: send screenshot to a vision-capable model (GPT-4V, Claude 3.5, Gemini 2.0 Flash). The model analyzes UI elements and returns structured descriptions.
- Coordinate-based interaction: ~xdotool~ / ~ydotool~ for click and type commands at specific screen coordinates. Dispatcher approval gate applies — screen interaction requires HITL by default, overridable per-application via permission table.
- Use case: the user says "open Firefox, search for the Passepartout GitHub repo, and star it." The agent captures screenshots, identifies UI elements via the vision model, and issues click/type commands. Each step is verified by a follow-up screenshot to confirm the action succeeded.

*** Competitive Advantage Analysis — v0.9.0 Summary

SWE-bench evaluation is the industry standard for coding agent capability claims. Without it, "SOTA parity" is a marketing claim. With it, "SOTA parity" is a number. The harness's trajectory persistence is a differentiator: most evaluation harnesses produce a pass/fail score. Passepartout's produces a complete Org-mode audit trail showing exactly where the reasoning succeeded or failed. This turns benchmarking into a debugging tool — failed trajectories point directly to the skill, gate, or model that needs improvement.

Vision + screen interaction is table stakes for competing with Claude Code's computer use feature. The Passepartout advantage: every screen interaction passes through the Dispatcher gate stack. A vision model might hallucinate a UI element that doesn't exist — the follow-up screenshot verification catches this deterministically. Competitors' computer use features lack this verification step — they trust the vision model's output.

** v0.10.0: Consensus, GTD & Deep Emacs Integration

Near-SOTA. The agent has tools, planning, evaluation, and streaming. v0.10.0 adds reliability (consensus), productivity methodology (GTD), and environment depth (Emacs integration).

*** TODO Consensus loop
- Multi-provider parallel inference for critical decisions. When the action's impact score exceeds a threshold (file writes outside home directory, shell commands that touch /etc, git pushes to main), the system sends the same prompt to 2–3 independent providers.
- Disagreement detection: compare the structured outputs (actions proposed by each provider). If all providers propose the same action (or semantically equivalent actions), proceed with the highest-confidence result. If providers disagree, flag the action for HITL approval and present the user with each provider's proposal and confidence score.
- Confidence scoring: when providers agree, use the agreement level as a confidence metric for telemetry. Track which provider combinations produce the highest agreement rates for which task types.
- Cost-aware: consensus mode doubles/triples cost for the action. Only trigger when the action's impact exceeds the cost threshold. Configurable via ~CONSENSUS_THRESHOLD~ — actions below the threshold use single-provider mode.
- TUI consensus display: when consensus mode fires, the TUI shows a collapsible region listing each provider, its model, its proposal, and its confidence score. Agreement is rendered as ~✓ 3/3 providers agree~ in green; disagreement as ~✗ 2/3 providers agree (1 disagrees)~ in yellow with the dissenting proposal expanded for review. The user can accept the majority or inspect the dissent before approving.

*** TODO GTD integration
- Full GTD cycle: capture (inbox → process), clarify (what is this? is it actionable?), organize (project, next action, reference, someday/maybe, trash), reflect (weekly review), engage (context-appropriate action lists).
- Org properties: ~:TRIGGER:~ (what context makes this actionable — @home, @office, @computer, @phone), ~:BLOCKER:~ (what task must complete first).
- Weekly review: the agent scans all projects and tasks, surfaces stalled items, suggests next actions, and generates a review Org file for the user. The review is produced deterministically (no LLM — pure Org tree traversal) and takes zero tokens.
- TUI agenda view: a ~/agenda~ command renders the user's Org-agenda (scheduled items, deadlines, habits) as a formatted scrollable region within the chat area. The agent can reference agenda context in its responses without the user having to paste their schedule.

*** TODO Deep Emacs integration

Rationale: The Emacs bridge (v0.4.0) treats Emacs as a Passepartout client — the user sends text, Emacs displays responses. This is the first direction: Emacs → Passepartout. The deep integration is the second direction: Passepartout → Emacs. The agent reads the user's agenda, clocks time on tasks, refiles headlines, and archives completed work. This builds on the TCP bridge already in place from v0.4.0 — the agent now initiates commands to Emacs, not just responds to user input.

- Org-agenda awareness: the agent queries the user's agenda view (scheduled items, deadlines, habits) and incorporates agenda context into planning decisions. "What should I work on today?" considers the agenda, not just the task tree.
- Clock time tracking: the agent starts/stops clocks on Org headlines. Produces clock tables for time reporting. This enables the agent to answer "how long did I spend on that feature?"
- Refile and archive: the agent refiles headlines between Org files and archives completed items to ~/memex/archives/~. Archive decisions are proposed by the LLM and verified by the Dispatcher (archive policy: DONE items older than 30 days, DONE items with no open child tasks).

*** Competitive Advantage Analysis — v0.10.0 Summary

The consensus loop is not unique (OpenClaw has a similar feature), but Passepartout's implementation benefits from the structured output enforcement in v0.6.2 — comparing plists for semantic equivalence is simpler and more reliable than comparing free-text responses.

The GTD integration and Emacs integration are Passepartout's "unfair advantages" — no competitor has either. Claude Code and Copilot are development tools, not life management tools. Org-mode is the bridge: the same format that holds the agent's memory holds the user's tasks, calendar, and notes. The GTD cycle operates on the same Org trees that the foveal-peripheral model renders into LLM context. There is no import/export, no separate task database, no format conversion. The agent's world model IS the user's Org files. This is the unified format thesis from the DESIGN_DECISIONS document made operational — and it's a capability that JSON-based agents structurally cannot replicate.

** v1.0.0: SOTA Parity (verified)

Feature-complete, benchmark-verified, production-hardened. All capabilities from v0.3.0 through v0.10.0 integrated and tested end-to-end.

v1.0.0 is not a feature release — it is a verification release. Every feature from the v0.x series is tested under concurrent load, resource starvation, adversarial input, and benchmark scoring. The evaluation harness (v0.9.0) provides the scoring apparatus; v1.0.0 is the scored release.

| Area              | Parity Target                               | Verification Method                   |
|-------------------+---------------------------------------------+---------------------------------------|
| Self-improvement  | Skill Creator + self-edit + hot-reload      | Skill regression suite (v0.3.x)       |
| Planning          | Task tree DAG with terminal states           | Multi-step integration tests          |
| Tool ecosystem    | 15+ MCP tools + native shell + git          | MCP protocol compliance tests         |
| Context window    | Semantic search + foveal-peripheral + caching| Token budget vs competitor audit      |
| Safety            | 10-vector Dispatcher + policy + permissions | Chaos testing (v0.9.0)                |
| Multi-step tasks  | Task trees with terminal states              | SWE-bench score (v0.9.0 harness)      |
| Code editing      | Full file read/write via MCP + Org           | SWE-bench-verified subset             |
| Memory            | Vector recall + Merkle integrity + MVCC     | Concurrency stress test (v0.6.1)      |
| Emacs integration | Full org-mode control (exceeds Claude Code)  | Org-agenda round-trip test            |
| Streaming         | Partial output + early termination           | TUI UX latency benchmark              |
| TUI                | Word wrap, cursor, gate trace, focus map,   | TUI integration test suite (v0.3.3, v0.4.0) |
|                    | rule counter, cost counter, streaming       |                                       |
| Packaging         | Source install (primary) + save-lisp-and-die | Install test matrix across distros        |
|                    | binary for constrained platforms              |                                           |
| Offline           | 100% local capable (7-13B model)            | Air-gapped integration test           |
| Cost              | 2-3x fewer tokens than competitors           | SWE-bench token audit                 |
| Concurrency       | Priority queue + MVCC + parallel signals     | Concurrent load test (3 users + bg)   |

**Performance projection at v1.0.0:**

| Scenario                      | Passepartout v1.0.0              | Claude Code                        | OpenClaw                           |
|-------------------------------+----------------------------------+------------------------------------+------------------------------------|
| Single-turn chat (local 8B)   | 2-4s, ~1,500 tok                 | N/A (cloud-only)                   | N/A (cloud-only)                   |
| Single-turn chat (cloud)      | 1-3s, ~1,500 tok                 | 1-3s, ~3,000 tok                   | 1-3s, ~3,500 tok                   |
| Multi-step coding (5 files)   | 15-30s, ~30,000 tok              | 10-20s, ~65,000 tok                | 20-40s, ~85,000 tok                |
| Knowledge base query (500 nodes)| <1s (in-image vector), 0 LLM tok | 3-5s, ~5,000 tok (LLM-assisted)   | 3-5s, ~5,000 tok (LLM-assisted)    |
| Background maintenance         | 0 LLM tok (deterministic cron)    | Variable or skipped                 | Variable or skipped                |
| Offline operation              | Full capability                   | None                               | None                               |
| Cost per coding session        | ~$0.15 (gpt-4o-mini)             | ~$0.45 (gpt-4o-mini)               | ~$0.55 (gpt-4o-mini)               |

Passepartout wins on cost (2-3x savings from sparse trees + deterministic gates + caching), offline capability (unique), and knowledge management (10-40x savings from in-image vector lookup + Org-native format). It is competitive on single-turn latency and slightly behind on multi-step latency (the single-pipeline architecture adds ~5s overhead per tool execution versus competitors' parallel tool dispatch).

The key insight at v1.0.0: Passepartout does not beat competitors at everything. It wins decisively where the architecture's structural advantages apply (safety, cost, offline operation, knowledge management) and is competitive where they don't (raw LLM inference speed, parallel tool dispatch). This is a defensible position — the niches Passepartout dominates are exactly the niches that matter for a sovereign, local-first AI assistant.

But it is still fundamentally probabilistic at its core. The symbolic engine verifies and constrains, but the generative engine is still the primary reasoning source. The architectural transition to symbolic-first reasoning happens in v3.0.0.

** v2.0.0: Lisp Machine Emergence

v2.0.0 is where Passepartout stops being a daemon with clients and becomes the environment. The agent's cognitive loop, the user's editor, the user's shell, and the user's browser run in the same Common Lisp image. The Dispatcher gate stack verifies every action regardless of who initiated it — user or agent. The distinction between "tool" and "self" dissolves.

*Why this version matters for UX parity.* v0.4.0 through v1.0.0 give Passepartout four interaction surfaces (TUI, messaging apps, Emacs, voice). v2.0.0 inverts the problem: instead of building more clients, it builds a platform where the agent's environment and the user's environment are the same process, separated not by a sandbox but by the Dispatcher gate stack. The editor IS the agent's prompt. The shell IS the agent's actuator. The browser IS the agent's web research tool. There are no clients — there is one Lisp image, one address space, one Org-mode file system.

*Architectural principle: Browser inside Lisp, not Lisp inside browser.* Lisp is the parent process. It owns the window, the memory, and the input loop. The rendering engine (WebKit/Blink) is a library that paints pixels inside a Lisp buffer. The user can redefine functions while browsing without restarting. Keybinding lookups happen in microseconds (SBCL machine code) — the browser cannot "steal" shortcuts.

*** Qt/QML via EQL5 — the rendering surface

- Qt/QML (via EQL5) is the UI framework. EQL5 exposes the full Qt C++ API from Common Lisp. QML is declarative — it matches Lisp's generation model.
- Desktop: native look and feel on Linux, macOS, and Windows.
- Mobile: Qt runs natively on iOS and Android. Android uses F-Droid for the unrestricted version and Play Store for sandboxed. iOS uses Guideline 4.7 ("Educational/Developer Tool" loophole, no JIT compilation).
- Safety Bridge for mobile: Lisp code can manipulate browser/files but cannot touch hardware (GPS, camera, contacts) without standard permission pop-ups.
- The minibuffer: a universal command line at the bottom of the screen. Not an Emacs modeline. Not a VS Code command palette. A single command surface for every action — edit files, navigate web, run Lisp expressions, invoke agent commands. ~M-x~ for everything.

*** Lish — the Common Lisp editor

Not elisp. Not Emacs. A multi-threaded Common Lisp editor rendered via Qt/QML. The complete system prompt lives in an Org buffer — the agent's identity, its skill registry, its memory, and its reasoning are visible and editable as Org text. The user modifies the agent's prompt and the agent reflects the change immediately — the prompt is a file in memory, not a hidden string in a config.

Org-babel for interactive evaluation: source blocks in Org files are executable. The user evaluates a ~#+begin_src lisp~ block and the result appears inline. The agent evaluates blocks to verify code before writing. The REPL is not a separate window — it is the Org buffer in which the agent and user both work.

The editor and the agent share the same Lisp image. The editor is not a client that connects to a daemon — it IS the daemon process. The TUI from v0.3.6 (with word wrap, streaming, gate trace, focus map) is the editor's rendering surface.

*** Nyxt — the Common Lisp browser (three erosion stages)

The browser is not a one-time feature. It is a multi-year erosion of the rendering stack toward pure Lisp:

*Stage 1 — Qt + WebKit.* Qt provides window management and native widgets. WebKit renders web content inside a Lisp buffer. Network requests via dexador (pure Lisp). HTML parsed via Plump (pure Lisp). Layout via Yoga (C-based Flexbox, wrapped via FFI). JavaScript via embedded QuickJS. This stage delivers a working browser in months, not years.

*Stage 2 — S-expression DOM.* Lisp builds its own DOM representation as native S-expressions. WebKit is reduced to pixel painting only — it receives rendered layouts from Lisp, not raw HTML. The agent can traverse and manipulate the DOM as Lisp data structures without serialization. This makes web content natively queryable and modifiable by the agent's cognitive loop.

*Stage 3 — Pure Lisp layout.* WebKit turned off entirely. Lisp-native layout engine (12-18 months of focused development). CSS subset sufficient for the modern web's 95% use case. JavaScript via QuickJS remains for interactive content. The browser is now a Lisp application that happens to speak HTTP, not a web engine wrapped in a Lisp process.

*** Lish — the Lisp shell

Bash is a text-stream protocol. Passepartout speaks plists. The Lish shell replaces text streams with structured data — every command returns a plist, not a byte stream. Pipe becomes function composition. Scripts become Lisp functions that operate on memory objects directly.

The agent and the user share the same shell. The user types ~(list-todos :tag "@urgent")~. The agent proposes ~(shell "npm run build")~. The Dispatcher verifies both. The shell is not a separate process — it is a REPL connected to the same Lisp image as the agent's cognitive loop.

Org-mode buffers become the file system. The user's memex (~/memex/) is browsable as a tree of Org headlines. File operations (read, write, list, search) operate on Org AST nodes, not byte streams. A "directory listing" is a tree of headlines. A "file read" is a subtree rendered as text.

Bash remains available as a backend for running external commands, but it is not the primary interface.

*** Emacs migration — three phases

The Emacs bridge (v0.4.0) is Phase I. The deep integration is three phases, not one:

*Phase I — Parasite (v0.4.0).* Emacs is a client. The elisp TCP bridge sends text and receives responses. The agent does not control Emacs. Emacs users get a native chat experience alongside the TUI.

*Phase II — Interpreter (v2.0.0).* An ELisp compatibility layer runs inside Passepartout's Common Lisp image. Key Emacs packages (Org-mode, Magit) run natively without an Emacs process. The compatibility layer does not aim for 100% coverage — it targets the packages the agent's workflows depend on.

*Phase III — Successor (v2.0.0 and beyond).* Native Common Lisp implementations of Org-mode workflows and Git integration read/write the same file formats. Total independence from Emacs. Emacs users who prefer Emacs keep the bridge. New users get the native experience.

*** Strategic timeline

v0.4.0 Emacs bridge (Phase I Parasite) → v1.0.0 SOTA parity → v2.0.0 Lish editor + Nyxt browser (Stage 1) + Emacs Phase II/III + mobile. The Qt/QML surface enables gradual erosion of the rendering stack without rewriting the application logic. The three-phase Emacs migration ensures Lisp users are never abandoned — the bridge works from day one, the native experience grows under it.

** v3.0.0: Neurosymbolic Maturity

Deterministic planner takes the wheel. LLM relegated to semantic translation.

*Architectural approach: Stitching, not building.* The symbolic engine is not a from-scratch reasoner. It is an integration of existing Common Lisp libraries connected by macros and DSLs. The Lisp advantage is the macro system — it transforms human-readable rules into formal logic queries without requiring a new engine.

*** Open-source Lisp stack

- *Knowledge Graph:* VivaceGraph v3 — Lisp-native graph database with a Prolog-like query language built in. Stores facts, relationships, and rules as native Lisp objects in the same image as the agent.
- *Constraint Solver:* Screamer — non-deterministic backtracking. Given a set of constraints, finds all valid solutions or proves none exist. Used to verify that proposed actions do not violate invariants.
- *Formal Verifier:* ACL2 — a theorem prover for Common Lisp, BSD licensed. Proves properties about functions before they are committed to the running image. Used for skill verification and Dispatcher rule validation.

*** The 10-80-10 architecture

Ten percent neural for input translation, eighty percent symbolic for reasoning against a knowledge graph, ten percent neural for output formatting.

- *10% Input:* The LLM translates natural language into structured queries (Prolog facts, knowledge graph lookups). The neural translator is trained via EGGROLL (low-rank evolution strategies) on the reward signal from the symbolic verifier — it learns to produce queries that the symbolic engine accepts.
- *80% Reasoning:* Pure Lisp. Task graphs generated by the deterministic planner against the knowledge graph. Formal verification via ACL2. Constraint checking via Screamer. Fact retrieval via VivaceGraph. Zero LLM tokens. Zero hallucinations.
- *10% Output:* The LLM formats symbolic results back into natural language. The neural formatter is structurally identical to the translator — same training loop, reversed direction.

*** The auto-formalizer bootstrap

The symbolic engine needs a populated knowledge graph. The auto-formalizer populates it:

1. Feed unstructured data (documentation, manuals, logs, session histories) to the LLM in ~auto-formalizer~ mode.
2. The LLM extracts facts, relationships, and rules as structured S-expressions.
3. The symbolic verifier (Screamer + ACL2) checks each extracted fact for consistency with the existing knowledge graph.
4. Consistent facts are added. Conflicting facts are flagged for human review.
5. Over time, the knowledge graph grows without manual ontology engineering.

*** DSL approach over engine building

Domain-specific languages, not general-purpose reasoners:

- Lisp macros transform human-readable rules into Prolog queries that run against VivaceGraph.
- ~(defrule check-privacy :when (contains-tag payload "@personal") :then :block)~ expands to a VivaceGraph query with Screamer constraint checking.
- Users write rules in a domain-specific DSL. The macros handle the translation to formal logic.
- The Skill Creator (v0.8.0) generates DSL rules from English descriptions. The auto-formalizer verifies them.
- ~(macroexpand-1 '(defrule ...))~ shows exactly how the rule compiles — 100% auditable.

*** Self-correcting gates

Gates learn from the full history of outcomes — did the plan succeed? Where did it fail? The symbolic engine updates its own rules based on results:

- Induced functions from v0.5.0 feed into the symbolic engine as candidate rules.
- The symbolic verifier checks each candidate against the knowledge graph for consistency.
- Rules that pass verification are promoted to the active gate stack.
- Rules that fail verification are discarded with a diagnostic — the agent learns why the pattern doesn't generalize.

*** Implications

Hallucination becomes structurally impossible because the symbolic engine will not accept a fact that contradicts its knowledge graph. Safety becomes provable because ACL2 can prove properties about the system's behavior. Self-improvement becomes stable because the agent modifies skills that are then verified before execution. The 80% of computation that happens in the symbolic middle layer costs zero LLM tokens.

** v4.0.0: Native Inference

LLM inference moves in-process. No external servers. No API keys required for inference.

*Lisp as Sovereign Governor, not as Math Engine.* The weights themselves are not stored as Lisp objects — this would waste 50% memory on type tags and destroy cache locality through pointer-chasing. Instead, the entire tensor is tagged as a single Lisp object (~macro-tag~). The Lisp image holds a pointer to optimized flat binary (GPU-friendly, FPGA-compatible). The tag is checked once. After that, all math happens in the optimized backend.

*** Native inference (FFI binding to llama.cpp)

- FFI binding to llama.cpp via CFFI: load GGUF models, run inference, manage KV cache. Single SBCL image, zero process boundaries. The agent and the model share memory.
- Speculative safety: the Dispatcher gate stack intercepts token generation in real time. A token that would produce a blocked action is preemptively suppressed before generation. No external inference API supports this.
- Foveal-peripheral compute: the model skips pruned context nodes during attention computation. External APIs compute full attention regardless of what you send. In-process inference makes the sparse-tree rendering pay off at the compute level, not just the token level.

*** Live surgery on cognition

With in-process inference, the agent's internal state becomes inspectable:

- Pause inference mid-stream. Inspect hidden states and activations as Lisp variables.
- Modify a vector, change a sampling parameter, resume.
- Detect when the agent is likely to hallucinate by comparing current activation patterns against historical baselines.
- The REPL becomes a surgical instrument for the agent's own cognition — not just for verifying code, but for inspecting and correcting the neural process that generates it.

*** DSL-compiled model architectures

Model architectures are described as Lisp DSL:

- ~(defmodel passepartout-reasoning :type 'transformer :heads 32 :dim 4096 :layers 32)~
- The DSL compiles to machine code for the target backend (GPU via CUDA, FPGA via VexRiscv, CPU via llama.cpp).
- Python interprets at runtime. Lisp compiles once. Model architecture changes are treated the same as code changes — edited, verified, hot-reloaded.

** v5.0.0: Hardware — Tagged Lisp Architecture

The Lisp machine becomes physical. RISC-V with tagged architecture, hardware-enforced type checking, and FPGA prototype for the symbolic core.

*Not a from-scratch processor.* Use RISC-V as the skeleton, add custom Lisp extensions. RISC-V provides the carrier architecture (standard instruction set, existing toolchain, LLVM support). Lisp extensions provide tagged computation (type checking in hardware, parallel garbage collection, S-expression traversal as atomic operations).

*** The macro-tag approach

- Top 4–8 bits of every memory word = Type Tag. Hardware checks tags in parallel with ALU operations. Trap on type mismatch.
- A tensor (70B weights) is one macro-tagged Lisp object — a pointer to flat binary. The tag is checked once. Math happens at native speed. This replaces "weights as sexps" (which wastes 50% memory on per-weight tags and destroys cache locality).
- Custom instructions: TADD (tagged add), LISP.CAR, LISP.CDR — Lisp primitives as single-cycle hardware operations.

*** Phase migration: Host → Co-processor → Self-hosted

1. *Parasitic.* Lisp card (FPGA) is a PCIe co-processor. Host CPU (Intel/AMD, Linux/Windows) handles "dirty" I/O — networking, display, file systems. Lisp card handles tagged computation and the agent's cognitive loop. If Lisp crashes, host survives. Reset card, reload. Memory mapping: the card can see the host's memory. The Lisp environment reaches out and inspects data.

2. *Functional Hijacking.* Lisp UI runs on the card, displays through the PC's GPU. The agent indexes Linux files into Lisp objects. The host becomes an I/O server for the Lisp card.

3. *Driver Cannibalization.* Point the agent at C drivers. Ask it to generate native Lisp drivers for the hardware the card controls directly. PCIe Passthrough for direct hardware access.

4. *Self-Hosting.* Replace the Linux bootloader with Stage0 Lisp (a bootstrap from 500 bytes of hex to a self-hosting Lisp). Cut the umbilical cord. The Lisp machine runs on bare metal.

*** Concrete prototyping milestones

| Stage | Hardware | Cost | What it delivers |
|-------+----------+------+-----------------|
| TinyTapeout | Custom silicon (130nm) | ~$500–1,000 | 8-bit tagged toy processor with Lisp primitives |
| Shuttle | Multi-project wafer | ~$10,000–20,000 | Tagged RISC-V core at 100–300MHz |
| FPGA | Terasic DE10-Nano / Xilinx KCU105 | ~$200–500 | VexRiscv with custom Lisp extensions, PCIe card form factor |
| Industrial | Commercial foundry (5nm) | ~$10M–100M+ | Competes with modern CPUs on tagged workloads |

Start at TinyTapeout. Validate the tagged architecture works. Move to FPGA. Validate at speed. Only then consider silicon.

*** Garbage collection in hardware

Dedicated bus master (Scavenger) runs background garbage collection while the main CPU executes code. No "GC pause." The scavenger traverses the heap in parallel with computation, freeing unreachable objects without stopping the agent.

*** Persistent single-address-space memory

NVRAM for the entire heap. Turn on the machine — state is exactly where you left it. No "booting." No "loading memory from disk." The agent's Merkle-tree memory, skill registry, knowledge graph, and induced functions survive restarts as a contiguous hardware state.

*** Why this is not "Lisp inside browser"

Most Lisp-on-hardware attempts fail because they try to compete with Intel on raw math. That's the wrong axis. The tagged architecture doesn't need to beat a GPU at matrix multiplication. It needs to beat a CPU at symbolic computation — graph traversal, constraint solving, theorem proving, garbage collection. These are the v3.0.0 symbolic engine's workload. Hardware that makes them single-cycle is the differentiator, not hardware that runs matrix math faster.

** v6.0.0: True Agency

World models, temporal reasoning, goal persistence across restarts.

- World models: Predictive models of user behavior, project dynamics, system state.
- Temporal reasoning: Scheduling, deadlines, elapsed duration awareness.
- Goal persistence: Goals survive restarts. Long-term projects in memory-objects.