DOCS: Systematic overhaul of Literate source (Granularity & Technical Reasoning)

literate/reason.org

@@ -4,29 +4,59 @@
 #+STARTUP: content
 
 * Stage 2: Reason (reason.lisp)
+
 ** Architectural Intent: Unified Cognition
-The Reason stage is the cognitive engine of the OpenCortex. It bridges the gap between raw sensory data (Perceive) and physical side-effects (Act). 
+The Reason stage is the cognitive engine of the OpenCortex. Its primary responsibility is to bridge the gap between raw sensory data (Perceive) and physical side-effects (Act). 
 
-* Cognition Engine (reason.lisp)
+Cognition is split into two distinct modes:
+1. **Probabilistic Reasoning:** Utilizing LLMs to generate creative proposals and understand natural language intent.
+2. **Deterministic Verification:** Utilizing native Lisp logic to verify and constrain the neural proposals against security and physics invariants.
 
-** Package Context
+This hybrid approach ensures the agent is both intelligent and mathematically safe.
+
+** Pipeline Initialization
 #+begin_src lisp :tangle ../src/reason.lisp
 (in-package :opencortex)
 #+end_src
 
-** Neural Backend Registry
-#+begin_src lisp :tangle ../src/reason.lisp
-(defvar *probabilistic-backends* (make-hash-table :test 'equal))
-(defvar *provider-cascade* nil)
-(defvar *model-selector-fn* nil)
-(defvar *consensus-enabled-p* nil)
+* Probabilistic Engine Infrastructure
 
+** Neural Backend Registry
+OpenCortex is provider-agnostic. All neural backends (OpenRouter, Ollama, etc.) register themselves here.
+
+#+begin_src lisp :tangle ../src/reason.lisp
+(defvar *probabilistic-backends* (make-hash-table :test 'equal)
+  "A global mapping of provider identifiers (keywords) to their respective execution functions.")
+#+end_src
+
+** Provider Cascade Configuration
+#+begin_src lisp :tangle ../src/reason.lisp
+(defvar *provider-cascade* nil
+  "An ordered list of providers to attempt if the primary one fails.")
+#+end_src
+
+#+begin_src lisp :tangle ../src/reason.lisp
+(defvar *model-selector-fn* nil
+  "A hook for dynamic model selection based on context complexity.")
+#+end_src
+
+#+begin_src lisp :tangle ../src/reason.lisp
+(defvar *consensus-enabled-p* nil
+  "Flag to enable parallel multi-model voting (not implemented in MVP).")
+#+end_src
+
+** Backend Registration Helper
+#+begin_src lisp :tangle ../src/reason.lisp
 (defun register-probabilistic-backend (name fn)
-  "Registers a neural provider (e.g., :gemini, :anthropic) with its calling function."
+  "Registers a neural provider with its calling function."
   (setf (gethash name *probabilistic-backends*) fn))
 #+end_src
 
-** Probabilistic Reasoning (probabilistic-call)
+* The Cognitive Cycle
+
+** Probabilistic Call (probabilistic-call)
+The primary interface for neural reasoning. It iterates through the cascade until a successful response is achieved or the cascade is exhausted.
+
 #+begin_src lisp :tangle ../src/reason.lisp
 (defun probabilistic-call (prompt &key (system-prompt "You are the Probabilistic engine.") (cascade nil) (context nil))
   "Dispatches a neural request through the provider cascade. Returns a Lisp plist or a failure log."
@@ -46,10 +76,12 @@ The Reason stage is the cognitive engine of the OpenCortex. It bridges the gap b
         (list :type :LOG :payload (list :text "Neural Cascade Failure: All providers exhausted.")))))
 #+end_src
 
-** Cognitive Proposal (Think)
+** LLM Output Sanitization (strip-markdown)
+Modern LLMs often wrap Lisp code in markdown backticks. This helper ensures the code is clean before the Lisp reader touches it.
+
 #+begin_src lisp :tangle ../src/reason.lisp
 (defun strip-markdown (text)
-  "Strips common markdown code block markers from text."
+  "Strips common markdown code block markers from text to ensure valid S-expression parsing."
   (if (and text (stringp text))
       (let ((cleaned text))
         (setf cleaned (cl-ppcre:regex-replace-all "^```[a-z]*\\n" cleaned ""))
@@ -57,7 +89,12 @@ The Reason stage is the cognitive engine of the OpenCortex. It bridges the gap b
         (setf cleaned (cl-ppcre:regex-replace-all "```" cleaned ""))
         (string-trim '(#\Space #\Newline #\Tab) cleaned))
       text))
+#+end_src
 
+** The Thought Process (Think)
+The core logic that prepares the "mind" for reasoning. It assembles the global awareness (Memex status, recent logs, active tasks) and provides a strict protocol template for the LLM to follow.
+
+#+begin_src lisp :tangle ../src/reason.lisp
 (defun think (context)
   "Generates a Lisp action proposal based on current context."
   (let* ((active-skill (find-triggered-skill context))
@@ -104,9 +141,11 @@ PROVIDER RULE: Always use the default cascade provider unless a specific model o
 #+end_src
 
 ** Deterministic Verification
+The final safety check. It iterates through all active skills to verify that the proposed neural action does not violate any invariants.
+
 #+begin_src lisp :tangle ../src/reason.lisp
 (defun deterministic-verify (proposed-action context)
-  "Iterates through all skill deterministic-gates sorted by priority."
+  "Iterates through all skill deterministic-gates sorted by priority. Ensures absolute safety of the neural proposal."
   (let ((current-action proposed-action)
         (skills nil))
     (maphash (lambda (name skill) (declare (ignore name)) (when (skill-deterministic-fn skill) (push skill skills))) *skills-registry*)
@@ -127,13 +166,18 @@ PROVIDER RULE: Always use the default cascade provider unless a specific model o
     current-action))
 #+end_src
 
-** Reasoning Gate (The Pipeline Stage)
+* The Reasoning Pipeline Stage
+
+** Reasoning Gate (reason-gate)
+The stage that ties it all together. It filters stimuli that don't require cognition (like internal heartbeat pulses) and executes the hybrid neural-logical loop.
+
 #+begin_src lisp :tangle ../src/reason.lisp
 (defun reason-gate (signal)
   "Unified Stage: Combines Probabilistic proposals and Deterministic verification."
   (let* ((type (proto-get signal :type))
          (payload (proto-get signal :payload))
          (sensor (proto-get payload :sensor)))
+    ;; Optimization: Only reason about user input or chat messages.
     (unless (and (eq type :EVENT) (member sensor '(:user-input :chat-message)))
       (return-from reason-gate signal))
     (let ((candidate (think signal)))