passepartout/org/core-loop-perceive.org

Stage 1: Perceive (perceive.lisp)

• Overview: Architectural Intent
  • Why the Async/Sync Split?
• Implementation
  • Package Context
  • Interrupt Flag
  • Sensor Configuration
  • loop-focus-id
  • Stimulus Injection (stimulus-inject)
  • Perceive Gate (loop-gate-perceive)
• Test Suite

Overview: Architectural Intent

The Perceive stage is the sensory cortex of Passepartout. It receives raw stimuli from diverse sources — terminal input, Emacs buffers, Telegram messages, Signal chats, heartbeat clocks, shell command outputs — and normalizes them into a single Signal format that the rest of the pipeline can process.

Each source has its own format and protocol. The CLI sends raw text. Emacs sends AST diffs. Telegram sends JSON. Without normalization, every downstream component (Reason, Act) would need to understand every input format. With normalization:

1. The gateway layer (CLI, Emacs, Telegram, Signal) just sends raw messages
2. Perceive transforms them into Signals regardless of origin
3. Reason and Act work with a single, consistent plist format
4. Adding a new gateway requires gateway code only — no changes to core

This is the "thin harness, fat skills" principle applied to input processing. The harness does the minimal normalization needed to produce a uniform Signal; the actual interpretation is left to skills.

Why the Async/Sync Split?

Perceive handles two kinds of stimuli:

• Synchronous (user input, chat messages) — these must be processed in order, one at a time, because each depends on the state left by the previous one
• Asynchronous (heartbeats, background sensor readings, delegation results) — these can be processed in parallel because they don't depend on user intent

The `*loop-async-sensors*` list defines which sensor types are processed in dedicated threads. Everything else goes through the main synchronous pipeline.

The depth limit prevents runaway recursive loops. A signal that generates another signal that generates another signal can infinite-loop. If depth exceeds a threshold (10), the signal is silently dropped rather than processed. This is the metabolic loop's circuit breaker.

Implementation

Package Context

(in-package :passepartout)

Interrupt Flag

A global interrupt flag that can be set by any signal. When set, the metabolic loop should stop processing and clean up. This is used for graceful shutdown: a SIGINT or /exit command sets the flag, and the loop exits at the next cycle boundary.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-interrupt* nil)

Sensor Configuration

*loop-async-sensors* lists the sensor types that should be processed in their own threads. Currently, :chat-message, :delegation, and :user-command are async because they don't block the main reasoning loop — the agent can process a Telegram message while waiting for the user's next input.

*loop-focus-id* tracks what the user is currently looking at in Emacs. When the user moves their cursor to a different Org headline, the buffer-update signal updates this ID. The Reason stage uses it to build the foveal-peripheral context model: the current headline gets full detail, everything else gets a skeletal outline.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-async-sensors* '(:chat-message :delegation :user-command)
  "Sensors that are processed in dedicated threads.")

loop-focus-id

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-focus-id* nil
  "The Org ID of the node the user is currently interacting with.")

#+end_src

Stimulus Injection (stimulus-inject)

This is the entry point that gateways call to send a message into the cognitive pipeline. It sets metadata (source, session ID, reply stream), decides whether the stimulus should be processed synchronously or on a background thread, and wraps the whole thing in error recovery so that no single bad stimulus can crash the system.

The error recovery uses Common Lisp's restart system. If any error occurs during processing, a `skip-event` restart is available. The handler displays the error, then invokes `skip-event` which drops the stimulus and continues. This is the "fail open" safety model — better to drop one message than to crash the entire agent.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun stimulus-inject (raw-message &key stream (depth 0))
  "Inject a raw message into the signal processing pipeline."
  (let* ((payload (getf raw-message :payload))
         (sensor (getf payload :sensor))
         (meta (getf raw-message :meta))
         (async-p (or (getf payload :async-p)
                     (member sensor *loop-async-sensors*))))

    (unless meta
      (setf meta (list :SOURCE :SYSTEM :SESSION-ID "internal")))

    (when stream
      (setf (getf meta :reply-stream) stream))

    (setf (getf raw-message :meta) meta)
    (setf (getf raw-message :depth) depth)

    (if async-p
        (bt:make-thread
         (lambda ()
           (restart-case (process-signal raw-message)
             (skip-event () nil)))
         :name "passepartout-async-task")
        
        (restart-case
            (handler-bind ((error (lambda (c)
                                    (log-message "SYSTEM ERROR: ~a" c)
                                    (invoke-restart 'skip-event))))
              (process-signal raw-message))
          (skip-event ()
            (log-message "SYSTEM RECOVERY: Stimulus dropped."))))))

Perceive Gate (loop-gate-perceive)

The perceive gate is the first stage of the metabolic pipeline. It receives a normalized signal and routes it based on the event type:

• :EVENT with :buffer-update — an Emacs buffer changed (new Org headline created, text edited). The change is ingested into memory so the agent has the latest state.
• :EVENT with :point-update — the user moved their cursor to a different headline. The foveal focus is updated, and the node at the cursor is ingested at higher priority.
• :EVENT with :interrupt — the user requested an interrupt. The interrupt flag is set.
• :RESPONSE — an action completed. The gate logs the result status.

All signals get tagged with their processing stage (`:status :perceived`) and the current foveal focus before being passed to the Reason stage.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun loop-gate-perceive (signal)
  "Stage 1 of the metabolic pipeline: Normalize sensory input."
  (let* ((payload (getf signal :payload))
         (type (getf signal :type))
         (meta (getf signal :meta))
         (sensor (getf payload :sensor)))

    (log-message "GATE [Perceive]: ~a (~a) [Source: ~s]"
                 type (or sensor "no-sensor") (getf meta :source))

    (cond ((eq type :EVENT)
            (case sensor
              (:buffer-update
               (let ((ast (getf payload :ast)))
                 (when ast
                   (snapshot-memory)
                   (ingest-ast ast))))
              (:point-update
               (let ((element (getf payload :element)))
                 (when element
                   (snapshot-memory)
                   (setf *loop-focus-id* (getf element :id))
                   (ingest-ast element))))
              (:interrupt
               (setf *loop-interrupt* t))
              ;; HITL: re-injected approved action from dispatcher-approvals-process
              (:approval-required
               (when (getf payload :approved)
                 (log-message "GATE [Perceive]: Approved Flight Plan re-injected")
                  (setf (getf signal :approved) t)
                 (setf (getf signal :approved-action) (getf payload :action))))
              ;; Default sensor: pass through without requiring user-input processing
              (otherwise
               (log-message "GATE [Perceive]: Unknown sensor ~a, passing through" sensor))))
          ((eq type :RESPONSE)
           (log-message "GATE [Perceive]: Act Result -> ~a" (getf payload :status))))

    (setf (getf signal :status) :perceived)
    (setf (getf signal :foveal-focus) *loop-focus-id*)
    signal))

Test Suite

Verifies that the perceive gate correctly ingests AST nodes into memory and that the depth limiter prevents runaway recursive signals.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload :fiveam :silent t))

(defpackage :passepartout-pipeline-perceive-tests
  (:use :cl :fiveam :passepartout)
  (:export #:pipeline-perceive-suite))

(in-package :passepartout-pipeline-perceive-tests)

(def-suite pipeline-perceive-suite :description "Test suite for Perceive pipeline")
(in-suite pipeline-perceive-suite)

(test test-loop-gate-perceive
  (clrhash passepartout::*memory-store*)
  (let* ((signal (list :type :EVENT :payload (list :sensor :buffer-update :ast (list :type :HEADLINE :properties (list :ID "test-node" :TITLE "Test") :contents nil))))
         (result (loop-gate-perceive signal)))
    (is (eq :perceived (getf result :status)))
    (is (not (null (gethash "test-node" passepartout::*memory-store*))))))

(test test-depth-limiting
  (let ((runaway-signal (list :type :EVENT :depth 11 :payload (list :sensor :heartbeat))))
    (is (null (process-signal runaway-signal)))))