diff --git a/README.org b/README.org index 8d2cf94..ebd7cf2 100644 --- a/README.org +++ b/README.org @@ -53,14 +53,14 @@ See [[file:docs/USER_MANUAL.org][User Manual]] for the full guide. * Project Documentation -| Document | What it answers | -|----------|----------------| +| Document | Answers | +|----------|---------| | [[file:docs/USER_MANUAL.org][User Manual]] | How do I use it? | -| [[file:docs/ROADMAP.org][Roadmap]] | Where is it going? | -| [[file:docs/DESIGN_DECISIONS.org][Design Decisions]] | Why is it built this way? | -| [[file:docs/CONTRIBUTING.org][Contributing]] | How do I add a skill? | -| [[file:TODO.org][TODO]] | What is being worked on? | -| [[file:docs/CHANGELOG.org][Changelog]] | What changed between versions? | +| [[file:docs/ARCHITECTURE.org][Architecture]] | How does it work inside? | +| [[file:docs/DESIGN_DECISIONS.org][Design Decisions]] | Why was it built this way? | +| [[file:docs/ROADMAP.org][Roadmap]] | Where is it going? When? | +| [[file:docs/TODO.org][TODO]] | Who is doing what? | +| [[file:docs/CONTRIBUTING.org][Contributing]] | How do I contribute? | * License diff --git a/docs/ARCHITECTURE.org b/docs/ARCHITECTURE.org new file mode 100644 index 0000000..ff91246 --- /dev/null +++ b/docs/ARCHITECTURE.org @@ -0,0 +1,90 @@ +#+TITLE: Passepartout Architecture +#+AUTHOR: Agent +#+STARTUP: content + +* The Four Quadrants + +Passepartout divides cognition along two axes: **Foreground vs Background** (initiated by the user vs running autonomously) and **Probabilistic vs Deterministic** (LLM-driven vs pure Lisp logic). + +| | Probabilistic (LLM) | Deterministic (Lisp) | +|----------------|--------------------|---------------------| +| **Foreground** | Chat responses, task execution, code generation | Shell execution, file I/O, safety gates, dispatcher checks | +| **Background** | Scribe distillation, vector embedding, autonomous decisions | Heartbeat, cron jobs, memory auto-save, gateway polling | + +The Probabilistic engine proposes. The Deterministic engine verifies and executes. No proposal from the LLM touches a file, runs a command, or sends a message without passing through at least one deterministic gate. + +* Code Map + +The project is organized into ~org/~ (source of truth) and ~lisp/~ (generated by tangle). + +** Core pipeline (loaded by ASDF, committed to git) + +| File | Purpose | +|------|---------| +| ~org/core-defpackage.org~ | Package definition and export list | +| ~org/core-skills.org~ | Skill engine: ~defskill~ macro, topological sorter, jailed loading | +| ~org/core-communication.org~ | Framed TCP protocol, actuator registry, daemon server | +| ~org/core-memory.org~ | ~memory-object~ struct, Merkle hashing, snapshots, persistence | +| ~org/core-context.org~ | Foveal-peripheral rendering, context assembly for LLM | +| ~org/core-loop-perceive.org~ | Stage 1: normalize raw signals into pipeline format | +| ~org/core-loop-reason.org~ | Stage 2: LLM proposal + deterministic verification | +| ~org/core-loop-act.org~ | Stage 3: dispatch approved actions to actuators | +| ~org/core-loop.org~ | Orchestration: process-signal, heartbeat, main entry point | +| ~org/system-diagnostics.org~ | Boot-time health check, doctor CLI | + +** Skills (loaded at runtime by the skill engine) + +| Category | Files | Purpose | +|----------|-------|---------| +| **gateway-** | ~gateway-cli~, ~gateway-llm~, ~gateway-manager~, ~gateway-provider~, ~gateway-tui~ | External communication channels | +| **security-** | ~security-dispatcher~, ~security-policy~, ~security-permissions~, ~security-vault~, ~security-validator~ | Safety and authorization | +| **programming-** | ~programming-lisp~, ~programming-org~, ~programming-standards~, ~programming-literate~, ~programming-repl~ | Lisp and Org tooling | +| **system-** | ~system-config~, ~system-archivist~, ~system-self-improve~, ~system-memory~, ~system-actuator-shell~, ~system-event-orchestrator~ | Background services | + +* Pipeline Flow + +Every signal moves through three stages: + +``` +Signal → Perceive (normalize) → Reason (think + verify) → Act (dispatch) +``` + +The signal is a plist: ~(:TYPE :EVENT :META (...) :PAYLOAD (:SENSOR :user-input :TEXT "..."))~ + +1. **Perceive** normalizes raw input from any gateway into a uniform signal +2. **Reason** calls the LLM to generate a proposal, then runs the proposal through all registered deterministic gates (sorted by priority). If a gate rejects the proposal, the rejection trace feeds back to the LLM for self-correction (up to 3 retries) +3. **Act** dispatches the approved action to the registered actuator (~:cli~, ~:tool~, ~:system~, ~:shell~, ~:telegram~, ~:signal~) + +Each stage can produce feedback signals that loop back to Perceive (e.g., a tool-execute action produces a ~:tool-output~ event that becomes the next perception). + +** Depth limiting + +A depth counter prevents infinite loops. If a signal's depth exceeds 10, it is silently dropped. This is the circuit breaker for runaway recursive cycles. + +* Skill Lifecycle + +1. **Discovery:** ~skill-initialize-all~ scans the skills directory, globs for ~*.lisp~ files (excluding ~core-*~ files which are loaded by ASDF) +2. **Sorting:** ~skill-topological-sort~ orders skills by their ~#+DEPENDS_ON:~ declarations +3. **Loading:** Each skill is loaded into a jailed package (~passepartout.skills.~). The loader removes ~in-package~ forms, evaluates the remaining code in the jailed package, and exports symbols matching the skill's short name to ~passepartout~ +4. **Registration:** The skill's ~defskill~ call creates a ~skill~ struct in ~*skill-registry*~, registering its trigger function, probabilistic prompt generator, deterministic gate, and system-prompt augment +5. **Triggering:** On each cognitive cycle, ~skill-triggered-find~ iterates the registry and returns the highest-priority skill whose trigger matches the context +6. **Hot-reload:** A skill can be replaced at runtime by loading a new version into its jailed package — no restart needed + +* Protocol Format + +All communication between the daemon and its gateways (TUI, CLI, Emacs) uses length-prefixed plists over TCP: + +``` +00002C(:TYPE :EVENT :PAYLOAD (:ACTION :handshake :VERSION "0.2.0")) +``` + +The 6-character hex prefix encodes the payload length. The payload is a ~prin1~-serialized plist. ~*read-eval*~ is bound to nil on the receiving end to prevent code injection. + +** Standard message envelope: + +| Key | Value | Meaning | +|-----|-------|---------| +| ~:TYPE~ | ~:REQUEST~, ~:EVENT~, ~:RESPONSE~, ~:LOG~, ~:STATUS~ | Message category | +| ~:META~ | plist | ~:SOURCE~, ~:SESSION-ID~, ~:reply-stream~ | +| ~:PAYLOAD~ | plist | Action-specific data (~:SENSOR~, ~:ACTION~, ~:TEXT~) | +| ~:DEPTH~ | integer | Recursion counter for loop prevention | diff --git a/TODO.org b/docs/TODO.org similarity index 100% rename from TODO.org rename to docs/TODO.org