passepartout/org/core-perceive.org

Stage 1: Perceive (perceive.lisp)

• Overview: Architectural Intent
  • Why the Async/Sync Split?
  • Contract
• Implementation
  • Package Context
  • Interrupt Flag
  • Scope Resolver
  • Sensor Configuration
  • loop-focus-id
  • Pre-Reason Handler Registry
    • Pre-Reason Handler Hash Table
    • register-pre-reason-handler
  • Stimulus Injection (stimulus-inject)
  • Perceive Gate (loop-gate-perceive)
    • loop-gate-perceive
    • perceive-gate (backward-compatibility alias)
• Test Suite

Overview: Architectural Intent

The Perceive stage is the sensory cortex of Passepartout. It receives raw stimuli from diverse sources — terminal input, Emacs buffers, Telegram messages, Signal chats, heartbeat clocks, shell command outputs — and normalizes them into a single Signal format that the rest of the pipeline can process.

Each source has its own format and protocol. The CLI sends raw text. Emacs sends AST diffs. Telegram sends JSON. Without normalization, every downstream component (Reason, Act) would need to understand every input format. With normalization:

1. The gateway layer (CLI, Emacs, Telegram, Signal) just sends raw messages
2. Perceive transforms them into Signals regardless of origin
3. Reason and Act work with a single, consistent plist format
4. Adding a new gateway requires gateway code only — no changes to core

This is the "thin harness, fat skills" principle applied to input processing. The harness does the minimal normalization needed to produce a uniform Signal; the actual interpretation is left to skills.

Why the Async/Sync Split?

Perceive handles two kinds of stimuli:

• Synchronous (user input, chat messages) — these must be processed in order, one at a time, because each depends on the state left by the previous one
• Asynchronous (heartbeats, background sensor readings, delegation results) — these can be processed in parallel because they don't depend on user intent

The `*loop-async-sensors*` list defines which sensor types are processed in dedicated threads. Everything else goes through the main synchronous pipeline.

The depth limit prevents runaway recursive loops. A signal that generates another signal that generates another signal can infinite-loop. If depth exceeds a threshold (10), the signal is silently dropped rather than processed. This is the metabolic loop's circuit breaker.

Contract

1. (loop-gate-perceive signal): normalizes sensory input. Routes by sensor type (:buffer-update, :point-update, :interrupt, :approval-required) and signal type (:EVENT, :RESPONSE). Sets :status :perceived on completion. Returns the signal.
2. (perceive-gate signal): thin alias for loop-gate-perceive.

Implementation

Package Context

(in-package :passepartout)

Interrupt Flag

A global interrupt flag that can be set by any signal. When set, the metabolic loop should stop processing and clean up. This is used for graceful shutdown: a SIGINT or /exit command sets the flag, and the loop exits at the next cycle boundary.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-interrupt* nil)

Scope Resolver

A hook for the context-manager skill to register its current-scope function. When set, the perceive gate passes the current context scope to ingest-ast so ingested objects are tagged and queryable by scope. Defaults to nil meaning all objects are ingested as :memex.

;; REPL-VERIFIED: 2026-05-03T14:00:00

(defvar *scope-resolver* nil
  "If set, function returning current scope keyword. Used by perceive gate.")

Sensor Configuration

*loop-async-sensors* lists the sensor types that should be processed in their own threads. Currently, :chat-message, :delegation, and :user-command are async because they don't block the main reasoning loop — the agent can process a Telegram message while waiting for the user's next input.

*loop-focus-id* tracks what the user is currently looking at in Emacs. When the user moves their cursor to a different Org headline, the buffer-update signal updates this ID. The Reason stage uses it to build the foveal-peripheral context model: the current headline gets full detail, everything else gets a skeletal outline.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-async-sensors* '(:chat-message :delegation :user-command)
  "Sensors that are processed in dedicated threads.")

loop-focus-id

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *loop-focus-id* nil
  "The Org ID of the node the user is currently interacting with.")

Pre-Reason Handler Registry

Skills register handlers for custom sensors here. When a signal arrives with a registered sensor, the handler is called in the perceive gate, before the signal reaches the LLM. The handler receives the full signal and returns T if the signal was consumed (don't continue to reason) or nil if processing should proceed normally.

Pre-Reason Handler Hash Table

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defvar *pre-reason-handlers* (make-hash-table :test 'eq)
  "Pre-reason handler registry: sensor keyword → handler function.")

register-pre-reason-handler

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun register-pre-reason-handler (sensor fn)
  "Registers FN to handle signals with SENSOR in the perceive gate.
FN receives (signal) and returns T if consumed, nil to continue."
  (setf (gethash sensor *pre-reason-handlers*) fn))

Stimulus Injection (stimulus-inject)

This is the entry point that gateways call to send a message into the cognitive pipeline. It sets metadata (source, session ID, reply stream), decides whether the stimulus should be processed synchronously or on a background thread, and wraps the whole thing in error recovery so that no single bad stimulus can crash the system.

The error recovery uses Common Lisp's restart system. If any error occurs during processing, a `skip-event` restart is available. The handler displays the error, then invokes `skip-event` which drops the stimulus and continues. This is the "fail open" safety model — better to drop one message than to crash the entire agent.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun stimulus-inject (raw-message &key stream (depth 0))
  "Inject a raw message into the signal processing pipeline."
  (let* ((payload (getf raw-message :payload))
         (sensor (getf payload :sensor))
         (meta (getf raw-message :meta))
         (async-p (or (getf payload :async-p)
                     (member sensor *loop-async-sensors*))))

    (unless meta
      (setf meta (list :SOURCE :SYSTEM :SESSION-ID "internal")))

    (when stream
      (setf (getf meta :reply-stream) stream))

    (setf (getf raw-message :meta) meta)
    (setf (getf raw-message :depth) depth)

    (if async-p
        (bt:make-thread
         (lambda ()
           (restart-case (process-signal raw-message)
             (skip-event () nil)))
         :name "passepartout-async-task")
        
        (restart-case
            (handler-bind ((error (lambda (c)
                                    (log-message "SYSTEM ERROR: ~a" c)
                                    (invoke-restart 'skip-event))))
              (process-signal raw-message))
          (skip-event ()
            (log-message "SYSTEM RECOVERY: Stimulus dropped."))))))

Perceive Gate (loop-gate-perceive)

The perceive gate is the first stage of the metabolic pipeline. It receives a normalized signal and routes it based on the event type:

• :EVENT with :buffer-update — an Emacs buffer changed (new Org headline created, text edited). The change is ingested into memory so the agent has the latest state.
• :EVENT with :point-update — the user moved their cursor to a different headline. The foveal focus is updated, and the node at the cursor is ingested at higher priority.
• :EVENT with :interrupt — the user requested an interrupt. The interrupt flag is set.
• :RESPONSE — an action completed. The gate logs the result status.

All signals get tagged with their processing stage (`:status :perceived`) and the current foveal focus before being passed to the Reason stage.

loop-gate-perceive

The main perceive pipeline stage.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun loop-gate-perceive (signal)
  "Stage 1 of the metabolic pipeline: Normalize sensory input."
  (let* ((payload (getf signal :payload))
          (type (getf signal :type))
          (meta (getf signal :meta))
          (sensor (getf payload :sensor)))
    ;; HITL: intercept approval/denial commands before LLM processing
    (when (and (eq sensor :user-input)
               (stringp (getf payload :text)))
      (let ((text (getf payload :text)))
        (when (ignore-errors (hitl-handle-message text (getf meta :source)))
          (log-message "GATE [Perceive]: HITL command processed — ~a" text)
          (return-from loop-gate-perceive signal))))
    ;; Pre-reason handlers: dispatch custom sensors to registered skill handlers
    (let ((handler (gethash sensor *pre-reason-handlers*)))
      (when handler
        (when (funcall handler signal)
          (return-from loop-gate-perceive signal))))

    (log-message "GATE [Perceive]: ~a (~a) [Source: ~s]"
                 type (or sensor "no-sensor") (getf meta :source))

    (cond ((eq type :EVENT)
            (case sensor
              (:buffer-update
               (let ((ast (getf payload :ast)))
                 (when ast
                   (snapshot-memory)
                   (ingest-ast ast :scope (if *scope-resolver* (funcall *scope-resolver*) :memex)))))
              (:point-update
               (let ((element (getf payload :element)))
                 (when element
                   (snapshot-memory)
                   (setf *loop-focus-id* (getf element :id))
                   (ingest-ast element :scope (if *scope-resolver* (funcall *scope-resolver*) :memex)))))
               (:interrupt
                (setf *loop-interrupt* t))
               ;; v0.7.2 undo/redo
               (:undo
                (log-message "GATE [Perceive]: undo requested")
                (undo "perceive"))
               (:redo
                (log-message "GATE [Perceive]: redo requested")
                (redo "perceive"))
              ;; HITL: re-injected approved action from dispatcher-approvals-process
              (:approval-required
               (when (getf payload :approved)
                 (log-message "GATE [Perceive]: Approved Flight Plan re-injected")
                  (setf (getf signal :approved) t)
                 (setf (getf signal :approved-action) (getf payload :action))))
              ;; Default sensor: pass through without requiring user-input processing
              (otherwise
               (log-message "GATE [Perceive]: Unknown sensor ~a, passing through" sensor))))
          ((eq type :RESPONSE)
           (log-message "GATE [Perceive]: Act Result -> ~a" (getf payload :status))))

    (setf (getf signal :status) :perceived)
    (setf (getf signal :foveal-focus) *loop-focus-id*)
    signal))

perceive-gate (backward-compatibility alias)

The pipeline gate was originally named perceive-gate. Code that still uses the old name can call this alias. New code should call loop-gate-perceive.

;; REPL-VERIFIED: 2026-05-03T13:00:00

(defun perceive-gate (signal)
  (loop-gate-perceive signal))

Test Suite

Verifies that the perceive gate correctly ingests AST nodes into memory and that the depth limiter prevents runaway recursive signals.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload :fiveam :silent t))

(defpackage :passepartout-pipeline-perceive-tests
  (:use :cl :fiveam :passepartout)
  (:export #:pipeline-perceive-suite))

(in-package :passepartout-pipeline-perceive-tests)

(def-suite pipeline-perceive-suite :description "Test suite for Perceive pipeline")
(in-suite pipeline-perceive-suite)

(test test-loop-gate-perceive
  "Contract 1: :buffer-update ingests AST and sets :perceived status."
  (clrhash passepartout::*memory-store*)
  (let* ((signal (list :type :EVENT :payload (list :sensor :buffer-update :ast (list :type :HEADLINE :properties (list :ID "test-node" :TITLE "Test") :contents nil))))
         (result (loop-gate-perceive signal)))
    (is (eq :perceived (getf result :status)))
    (is (not (null (gethash "test-node" passepartout::*memory-store*))))))

(test test-depth-limiting
  "Edge: depth 11 signals are rejected by the pipeline."
  (let ((runaway-signal (list :type :EVENT :depth 11 :payload (list :sensor :heartbeat))))
    (is (null (process-signal runaway-signal)))))

(test test-loop-gate-perceive-unknown-sensor
  "Contract 1: unknown sensors pass through and reach :perceived."
  (let* ((signal (list :type :EVENT :depth 0 :payload (list :sensor :custom-metric)))
         (result (loop-gate-perceive signal)))
    (is (eq :perceived (getf result :status)))))

(test test-loop-gate-perceive-no-ast
  "Contract 1: :buffer-update without AST doesn't crash, reaches :perceived."
  (clrhash passepartout::*memory-store*)
  (let* ((signal (list :type :EVENT :depth 0 :payload (list :sensor :buffer-update)))
         (result (loop-gate-perceive signal)))
    (is (eq :perceived (getf result :status)))))

(test test-depth-limiting-normal
  "Contract 1: signals at normal depth pass through without rejection."
  (let ((normal-signal (list :type :EVENT :depth 5 :payload (list :sensor :heartbeat))))
    (is (not (eq :rejected (getf normal-signal :status)))
        "Signal at normal depth should not be rejected")))