DOCS: Systematic overhaul of Literate source (Granularity & Technical Reasoning)

literate/loop.org

@@ -4,25 +4,42 @@
 #+STARTUP: content
 
 * The Metabolic Loop (loop.lisp)
+
 ** Architectural Intent: The Heartbeat
-The Metabolic Loop is the high-level coordinator of the OpenCortex. It orchestrates the flow of energy (information) through the system by calling the three metabolic stages in sequence:
-1. **Perceive:** Sensory intake.
-2. **Reason:** Cognitive processing.
-3. **Act:** Physical side-effects.
+The Metabolic Loop is the high-level coordinator of the OpenCortex. It orchestrates the flow of energy (information) through the system by recursively calling the metabolic stages: Perceive, Reason, and Act.
 
-** Package and Variables
-The loop requires thread-safe interrupt handling to ensure that the agent can be stopped gracefully without leaving the Lisp image in an inconsistent state.
+Inspired by biological metabolism, the loop ensures that every stimulus is processed until it reaches "stasis" (no further actions required) or an error occurs. This recursive design allows the agent to chain multiple thoughts and tool calls together into a single cohesive cognitive session.
 
+** Pipeline Initialization
 #+begin_src lisp :tangle ../src/loop.lisp
 (in-package :opencortex)
-
-(defvar *interrupt-flag* nil)
-(defvar *interrupt-lock* (bt:make-lock "harness-interrupt-lock"))
-(defvar *heartbeat-thread* nil)
 #+end_src
 
-** The Metabolic Pipeline
-The `process-signal` function is the core metabolic processor. It iterates through the Perceive-Reason-Act gates until the signal is fully processed or an error state is reached. We have refined the error handling to ensure that memory rollbacks only occur on critical system failures, preventing transient tool errors from wiping short-term cognitive state.
+* Concurrency and Interrupts
+
+** Metabolic Interrupt Flag
+The harness must be able to stop gracefully. We use a thread-safe flag to signal the daemon to exit its primary loop.
+
+#+begin_src lisp :tangle ../src/loop.lisp
+(defvar *interrupt-flag* nil
+  "Thread-safe signal to halt the metabolic pipeline and daemon.")
+#+end_src
+
+#+begin_src lisp :tangle ../src/loop.lisp
+(defvar *interrupt-lock* (bt:make-lock "harness-interrupt-lock")
+  "Protects the interrupt flag from concurrent access.")
+#+end_src
+
+** Heartbeat Thread Reference
+#+begin_src lisp :tangle ../src/loop.lisp
+(defvar *heartbeat-thread* nil
+  "Reference to the background thread driving autonomous reflection.")
+#+end_src
+
+* The Metabolic Pipeline
+
+** Signal Processor (process-signal)
+The primary cognitive processor. It takes a normalized signal and pushes it through the gates. If a gate generates "Feedback" (e.g., a tool result), the function recursively processes that feedback as a new stimulus.
 
 #+begin_src lisp :tangle ../src/loop.lisp
 (defun process-signal (signal)
@@ -31,69 +48,83 @@ The `process-signal` function is the core metabolic processor. It iterates throu
     (loop while current-signal do
       (let ((depth (getf current-signal :depth 0))
             (meta (getf current-signal :meta)))
+        ;; Safety: Prevent infinite cognitive recursion.
         (when (> depth 10) (harness-log "METABOLISM ERROR: Max depth reached.") (return nil))
+        
+        ;; Check for graceful shutdown.
         (when (bt:with-lock-held (*interrupt-lock*) *interrupt-flag*)
           (harness-log "METABOLISM: Interrupted.")
           (bt:with-lock-held (*interrupt-lock*) (setf *interrupt-flag* nil))
           (return nil))
+        
         (handler-case
             (progn
+              ;; Stage 1: Ingest and Normalize
               (setf current-signal (perceive-gate current-signal))
+              ;; Stage 2: Cogitate and Verify
               (setf current-signal (reason-gate current-signal))
+              ;; Stage 3: Actuate and Generate Feedback
               (let ((feedback (act-gate current-signal)))
-                ;; feedback generation
                 (if feedback
                     (progn
-                      ;; Inherit meta from trigger signal
+                      ;; Inheritance: Metadata must persist across recursive cycles.
                       (unless (getf feedback :meta) (setf (getf feedback :meta) meta))
                       (setf current-signal feedback))
                     (setf current-signal nil))))
           (error (c)
             (let ((sensor (ignore-errors (getf (getf current-signal :payload) :sensor))))
               (harness-log "METABOLISM CRASH [~a]: ~a" (or sensor :unknown) c)
-              ;; Only rollback on critical errors, not standard tool or loop errors
+              ;; Resilience: Only rollback on critical system errors.
               (unless (member sensor '(:loop-error :tool-error :syntax-error))
                 (harness-log "CRITICAL ERROR: Initiating Micro-Rollback.")
                 (rollback-memory 0))
+              ;; If recursion is shallow, attempt to notify the user of the error.
               (if (or (> depth 2) (member sensor '(:loop-error :tool-error)))
                   (setf current-signal nil)
                   (setf current-signal (list :type :EVENT :depth (1+ depth) :meta meta
                                              :payload (list :sensor :loop-error :message (format nil "~a" c) :depth depth)))))))))))
 #+end_src
 
-** Heartbeat Mechanism
-The heartbeat ensures the agent remains "alive" even in the absence of external stimuli, allowing for latent reflection and periodic maintenance. The interval is externalized to the `HEARTBEAT_INTERVAL` environment variable.
+* Autonomous Reflection
+
+** Heartbeat Mechanism (start-heartbeat)
+The heartbeat ensures the agent remains "alive" even in the absence of external stimuli. It allows background workers like the Scribe and Gardener to trigger periodically.
 
 #+begin_src lisp :tangle ../src/loop.lisp
 (defun start-heartbeat ()
-  "Starts the background heartbeat thread. Interval is loaded from HEARTBEAT_INTERVAL."
+  "Starts the background heartbeat thread. Interval is loaded from HEARTBEAT_INTERVAL (default: 60s)."
   (let ((interval (or (ignore-errors (parse-integer (uiop:getenv "HEARTBEAT_INTERVAL"))) 60)))
     (setf *heartbeat-thread* 
           (bt:make-thread 
            (lambda () 
              (loop 
                (sleep interval) 
-               ;; inject-stimulus is synchronous for heartbeats, preventing accumulation.
+               ;; Note: inject-stimulus is synchronous for heartbeats to prevent task accumulation.
                (inject-stimulus (list :type :EVENT :payload (list :sensor :heartbeat :unix-time (get-universal-time)))))) 
            :name "opencortex-heartbeat"))))
 #+end_src
 
-** Main Entry Point
-The `main` function initializes the environment, loads skills, and starts the heartbeat. It now includes a graceful shutdown handler for `SIGINT` (Ctrl+C) and uses `DAEMON_SLEEP_INTERVAL` to control its idle rhythm.
+* Lifecycle Management
+
+** Main Daemon Entry Point (main)
+Initializes the image, boots the gateways, and enters the primary idle loop.
 
 #+begin_src lisp :tangle ../src/loop.lisp
 (defun main ()
-  "Entry point for the Skeleton MVP. Handles initialization and graceful shutdown."
+  "Primary entry point for the OpenCortex daemon."
+  ;; 1. Environment Hydration
   (let* ((home (uiop:getenv "HOME"))
          (env-file (uiop:merge-pathnames* ".local/share/opencortex/.env" (uiop:ensure-directory-pathname home))))
     (when (uiop:file-exists-p env-file) (cl-dotenv:load-env env-file)))
   
+  ;; 2. System Bootstrap
   (initialize-actuators)
   (initialize-all-skills)
 
+  ;; 3. Wake up the heart.
   (start-heartbeat)
   
-  ;; Graceful shutdown handler for SBCL
+  ;; 4. OS Signal Handling (SBCL specific)
   #+sbcl
   (sb-sys:enable-interrupt sb-unix:sigint 
                            (lambda (sig code scp) 
@@ -101,6 +132,7 @@ The `main` function initializes the environment, loads skills, and starts the he
                              (harness-log "SHUTDOWN: SIGINT received. Exiting...")
                              (uiop:quit 0)))
 
+  ;; 5. Primary Idle Loop
   (let ((sleep-interval (or (ignore-errors (parse-integer (uiop:getenv "DAEMON_SLEEP_INTERVAL"))) 3600)))
     (loop 
       (when (bt:with-lock-held (*interrupt-lock*) *interrupt-flag*) (return))