Amr Gharbeia
3f51a772d4
Some checks failed
Deploy (Gitea) / deploy (push) Failing after 3s
Adds prose sections before every code block to satisfy the prose-before-code discipline. Each backward-compatibility alias (process-signal, perceive-gate, reason-gate, act-gate, inject-stimulus) now has its own subsection explaining why it exists and what new code should use instead. Also: - Fixes double #+end_src in core-loop-perceive.org - Renames inject-stimulus → stimulus-inject in heartbeat-start and client-handle-connection (both already had aliases) - Adds HITL interception prose to gateway-manager.org telegram/signal sections - Splits Pre-Reason Handler Registry into two code blocks (defvar + defun) for one-per-block compliance
252 lines
11 KiB
Org Mode
252 lines
11 KiB
Org Mode
#+TITLE: Stage 1: Perceive (perceive.lisp)
|
|
#+AUTHOR: Agent
|
|
#+FILETAGS: :harness:perceive:
|
|
#+STARTUP: content
|
|
#+PROPERTY: header-args:lisp :tangle ../lisp/core-loop-perceive.lisp
|
|
|
|
* Overview: Architectural Intent
|
|
|
|
The Perceive stage is the sensory cortex of Passepartout. It receives raw stimuli from diverse sources — terminal input, Emacs buffers, Telegram messages, Signal chats, heartbeat clocks, shell command outputs — and normalizes them into a single Signal format that the rest of the pipeline can process.
|
|
|
|
Each source has its own format and protocol. The CLI sends raw text. Emacs sends AST diffs. Telegram sends JSON. Without normalization, every downstream component (Reason, Act) would need to understand every input format. With normalization:
|
|
|
|
1. The gateway layer (CLI, Emacs, Telegram, Signal) just sends raw messages
|
|
2. Perceive transforms them into Signals regardless of origin
|
|
3. Reason and Act work with a single, consistent plist format
|
|
4. Adding a new gateway requires gateway code only — no changes to core
|
|
|
|
This is the "thin harness, fat skills" principle applied to input processing. The harness does the minimal normalization needed to produce a uniform Signal; the actual interpretation is left to skills.
|
|
|
|
** Why the Async/Sync Split?
|
|
|
|
Perceive handles two kinds of stimuli:
|
|
- **Synchronous** (user input, chat messages) — these must be processed in order, one at a time, because each depends on the state left by the previous one
|
|
- **Asynchronous** (heartbeats, background sensor readings, delegation results) — these can be processed in parallel because they don't depend on user intent
|
|
|
|
The `*loop-async-sensors*` list defines which sensor types are processed in dedicated threads. Everything else goes through the main synchronous pipeline.
|
|
|
|
The depth limit prevents runaway recursive loops. A signal that generates another signal that generates another signal can infinite-loop. If depth exceeds a threshold (10), the signal is silently dropped rather than processed. This is the metabolic loop's circuit breaker.
|
|
|
|
* Implementation
|
|
|
|
** Package Context
|
|
#+begin_src lisp
|
|
(in-package :passepartout)
|
|
#+end_src
|
|
|
|
** Interrupt Flag
|
|
|
|
A global interrupt flag that can be set by any signal. When set, the metabolic loop should stop processing and clean up. This is used for graceful shutdown: a SIGINT or /exit command sets the flag, and the loop exits at the next cycle boundary.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defvar *loop-interrupt* nil)
|
|
#+end_src
|
|
|
|
** Sensor Configuration
|
|
|
|
~*loop-async-sensors*~ lists the sensor types that should be processed in their own threads. Currently, ~:chat-message~, ~:delegation~, and ~:user-command~ are async because they don't block the main reasoning loop — the agent can process a Telegram message while waiting for the user's next input.
|
|
|
|
~*loop-focus-id*~ tracks what the user is currently looking at in Emacs. When the user moves their cursor to a different Org headline, the buffer-update signal updates this ID. The Reason stage uses it to build the foveal-peripheral context model: the current headline gets full detail, everything else gets a skeletal outline.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defvar *loop-async-sensors* '(:chat-message :delegation :user-command)
|
|
"Sensors that are processed in dedicated threads.")
|
|
|
|
#+end_src
|
|
** *loop-focus-id*
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defvar *loop-focus-id* nil
|
|
"The Org ID of the node the user is currently interacting with.")
|
|
#+end_src
|
|
|
|
** Pre-Reason Handler Registry
|
|
|
|
Skills register handlers for custom sensors here. When a signal arrives
|
|
with a registered sensor, the handler is called in the perceive gate,
|
|
before the signal reaches the LLM. The handler receives the full signal
|
|
and returns T if the signal was consumed (don't continue to reason)
|
|
or nil if processing should proceed normally.
|
|
|
|
*** Pre-Reason Handler Hash Table
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defvar *pre-reason-handlers* (make-hash-table :test 'eq)
|
|
"Pre-reason handler registry: sensor keyword → handler function.")
|
|
#+end_src
|
|
|
|
*** register-pre-reason-handler
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defun register-pre-reason-handler (sensor fn)
|
|
"Registers FN to handle signals with SENSOR in the perceive gate.
|
|
FN receives (signal) and returns T if consumed, nil to continue."
|
|
(setf (gethash sensor *pre-reason-handlers*) fn))
|
|
#+end_src
|
|
|
|
** inject-stimulus backward-compatibility alias
|
|
|
|
Skills and external code that still call ~inject-stimulus~ (the previous
|
|
name for the pipeline injection function) can use this alias. New code
|
|
should call ~stimulus-inject~ directly.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defun inject-stimulus (raw-message &key stream (depth 0))
|
|
(stimulus-inject raw-message :stream stream :depth depth))
|
|
#+end_src
|
|
|
|
** Stimulus Injection (stimulus-inject)
|
|
|
|
This is the entry point that gateways call to send a message into the cognitive pipeline. It sets metadata (source, session ID, reply stream), decides whether the stimulus should be processed synchronously or on a background thread, and wraps the whole thing in error recovery so that no single bad stimulus can crash the system.
|
|
|
|
The error recovery uses Common Lisp's restart system. If any error occurs during processing, a `skip-event` restart is available. The handler displays the error, then invokes `skip-event` which drops the stimulus and continues. This is the "fail open" safety model — better to drop one message than to crash the entire agent.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defun stimulus-inject (raw-message &key stream (depth 0))
|
|
"Inject a raw message into the signal processing pipeline."
|
|
(let* ((payload (getf raw-message :payload))
|
|
(sensor (getf payload :sensor))
|
|
(meta (getf raw-message :meta))
|
|
(async-p (or (getf payload :async-p)
|
|
(member sensor *loop-async-sensors*))))
|
|
|
|
(unless meta
|
|
(setf meta (list :SOURCE :SYSTEM :SESSION-ID "internal")))
|
|
|
|
(when stream
|
|
(setf (getf meta :reply-stream) stream))
|
|
|
|
(setf (getf raw-message :meta) meta)
|
|
(setf (getf raw-message :depth) depth)
|
|
|
|
(if async-p
|
|
(bt:make-thread
|
|
(lambda ()
|
|
(restart-case (process-signal raw-message)
|
|
(skip-event () nil)))
|
|
:name "passepartout-async-task")
|
|
|
|
(restart-case
|
|
(handler-bind ((error (lambda (c)
|
|
(log-message "SYSTEM ERROR: ~a" c)
|
|
(invoke-restart 'skip-event))))
|
|
(process-signal raw-message))
|
|
(skip-event ()
|
|
(log-message "SYSTEM RECOVERY: Stimulus dropped."))))))
|
|
#+end_src
|
|
|
|
** Perceive Gate (loop-gate-perceive)
|
|
|
|
The perceive gate is the first stage of the metabolic pipeline. It receives a normalized signal and routes it based on the event type:
|
|
|
|
- ~:EVENT~ with ~:buffer-update~ — an Emacs buffer changed (new Org headline created, text edited). The change is ingested into memory so the agent has the latest state.
|
|
- ~:EVENT~ with ~:point-update~ — the user moved their cursor to a different headline. The foveal focus is updated, and the node at the cursor is ingested at higher priority.
|
|
- ~:EVENT~ with ~:interrupt~ — the user requested an interrupt. The interrupt flag is set.
|
|
- ~:RESPONSE~ — an action completed. The gate logs the result status.
|
|
|
|
All signals get tagged with their processing stage (`:status :perceived`) and the current foveal focus before being passed to the Reason stage.
|
|
|
|
*** loop-gate-perceive
|
|
|
|
The main perceive pipeline stage.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defun loop-gate-perceive (signal)
|
|
"Stage 1 of the metabolic pipeline: Normalize sensory input."
|
|
(let* ((payload (getf signal :payload))
|
|
(type (getf signal :type))
|
|
(meta (getf signal :meta))
|
|
(sensor (getf payload :sensor)))
|
|
;; HITL: intercept approval/denial commands before LLM processing
|
|
(when (and (eq sensor :user-input)
|
|
(stringp (getf payload :text)))
|
|
(let ((text (getf payload :text)))
|
|
(when (ignore-errors (hitl-handle-message text (getf meta :source)))
|
|
(log-message "GATE [Perceive]: HITL command processed — ~a" text)
|
|
(return-from loop-gate-perceive signal))))
|
|
;; Pre-reason handlers: dispatch custom sensors to registered skill handlers
|
|
(let ((handler (gethash sensor *pre-reason-handlers*)))
|
|
(when handler
|
|
(when (funcall handler signal)
|
|
(return-from loop-gate-perceive signal))))
|
|
|
|
(log-message "GATE [Perceive]: ~a (~a) [Source: ~s]"
|
|
type (or sensor "no-sensor") (getf meta :source))
|
|
|
|
(cond ((eq type :EVENT)
|
|
(case sensor
|
|
(:buffer-update
|
|
(let ((ast (getf payload :ast)))
|
|
(when ast
|
|
(snapshot-memory)
|
|
(ingest-ast ast))))
|
|
(:point-update
|
|
(let ((element (getf payload :element)))
|
|
(when element
|
|
(snapshot-memory)
|
|
(setf *loop-focus-id* (getf element :id))
|
|
(ingest-ast element))))
|
|
(:interrupt
|
|
(setf *loop-interrupt* t))
|
|
;; HITL: re-injected approved action from dispatcher-approvals-process
|
|
(:approval-required
|
|
(when (getf payload :approved)
|
|
(log-message "GATE [Perceive]: Approved Flight Plan re-injected")
|
|
(setf (getf signal :approved) t)
|
|
(setf (getf signal :approved-action) (getf payload :action))))
|
|
;; Default sensor: pass through without requiring user-input processing
|
|
(otherwise
|
|
(log-message "GATE [Perceive]: Unknown sensor ~a, passing through" sensor))))
|
|
((eq type :RESPONSE)
|
|
(log-message "GATE [Perceive]: Act Result -> ~a" (getf payload :status))))
|
|
|
|
(setf (getf signal :status) :perceived)
|
|
(setf (getf signal :foveal-focus) *loop-focus-id*)
|
|
signal))
|
|
#+end_src
|
|
|
|
*** perceive-gate (backward-compatibility alias)
|
|
|
|
The pipeline gate was originally named ~perceive-gate~. Code that still
|
|
uses the old name can call this alias. New code should call
|
|
~loop-gate-perceive~.
|
|
|
|
;; REPL-VERIFIED: 2026-05-03T13:00:00
|
|
#+begin_src lisp
|
|
(defun perceive-gate (signal)
|
|
(loop-gate-perceive signal))
|
|
#+end_src
|
|
|
|
* Test Suite
|
|
Verifies that the perceive gate correctly ingests AST nodes into memory and that the depth limiter prevents runaway recursive signals.
|
|
#+begin_src lisp :tangle ../lisp/core-loop-perceive.lisp
|
|
(eval-when (:compile-toplevel :load-toplevel :execute)
|
|
(ql:quickload :fiveam :silent t))
|
|
|
|
(defpackage :passepartout-pipeline-perceive-tests
|
|
(:use :cl :fiveam :passepartout)
|
|
(:export #:pipeline-perceive-suite))
|
|
|
|
(in-package :passepartout-pipeline-perceive-tests)
|
|
|
|
(def-suite pipeline-perceive-suite :description "Test suite for Perceive pipeline")
|
|
(in-suite pipeline-perceive-suite)
|
|
|
|
(test test-loop-gate-perceive
|
|
(clrhash passepartout::*memory-store*)
|
|
(let* ((signal (list :type :EVENT :payload (list :sensor :buffer-update :ast (list :type :HEADLINE :properties (list :ID "test-node" :TITLE "Test") :contents nil))))
|
|
(result (loop-gate-perceive signal)))
|
|
(is (eq :perceived (getf result :status)))
|
|
(is (not (null (gethash "test-node" passepartout::*memory-store*))))))
|
|
|
|
(test test-depth-limiting
|
|
(let ((runaway-signal (list :type :EVENT :depth 11 :payload (list :sensor :heartbeat))))
|
|
(is (null (process-signal runaway-signal)))))
|
|
#+end_src |