PSF: Standardize core gates and refine skill loading mechanism

- Improved decide-gate to normalize candidates (wrap strings in RESPONSE)
- Refined load-skill-from-org to skip tangled blocks and Org properties
- Updated system definition and test suites for v1.0

docs/quickstart.org

@@ -0,0 +1,55 @@
+#+TITLE: Quickstart Guide: The Road to Sovereignty
+#+AUTHOR: Amr
+#+DATE: [2026-04-11 Sat]
+#+FILETAGS: :quickstart:onboarding:guide:
+
+* 1. Introduction
+Welcome to ~org-agent~, the "Executive Soul" of your personal OS. This guide will help you set up and interact with your first neurosymbolic agent.
+
+* 2. Prerequisites
+Before launching the kernel, ensure your host environment has:
+- **Docker & Docker Compose**: The primary enclosure for the Lisp Machine.
+- **LLM API Keys**: At least one key for Gemini, Anthropic, or OpenAI.
+- **Emacs (Optional)**: For the full literate experience via ~org-agent.el~.
+
+* 3. Installation & Enclosure
+** Step 1: Clone the Sovereignty
+#+begin_src bash
+git clone https://github.com/amr/org-agent.git
+cd org-agent
+#+end_src
+
+** Step 2: Secret Configuration
+Copy the example environment file and add your keys.
+#+begin_src bash
+cp .env.example .env
+# Edit .env with your favorite editor
+#+end_src
+
+** Step 3: Launch the Image
+This will build the SBCL environment and start the Micro-Loader.
+#+begin_src bash
+docker-compose up --build -d
+#+end_src
+
+* 4. Interaction Gateways
+Once the kernel is "Ready", you can interact with it via multiple sensors.
+
+** Gateway A: Emacs (OACP)
+If you have configured the ~org-agent~ package in Emacs:
+1. Open a chat buffer: ~M-x org-agent-chat-open~.
+2. Send: "Are you online, agent?"
+
+** Gateway B: External Sensors
+If you enabled Signal or Telegram in ~.env~, send a message directly to your bot.
+
+* 5. Verification (The Chaos Check)
+To ensure the kernel is fully healthy, check the logs for the Micro-Loader summary:
+#+begin_src bash
+docker-compose logs -f org-agent
+#+end_src
+Look for: ~LOADER: Boot Complete. [Ready: 34] [Failed: 0]~
+
+* 6. Next Steps
+- **Extend the Brain**: Read the [[file:skill-creation.org][Skill Creation Guide]] to add custom Lisp skills.
+- **Deep Dive**: Explore the [[file:../literate/][literate/]] directory to understand the kernel's architecture.

literate/core.org

@@ -172,7 +172,7 @@ Invokes the neural System 1 engine to generate intuition-based proposals. If par
     (return-from neuro-gate signal))
   (kernel-log "GATE [Neuro]: Consulting System 1...")
   (let ((thoughts (think signal)))
-    (setf (getf signal :proposals) (if (and thoughts (listp thoughts) (listp (car thoughts))) 
+    (setf (getf signal :proposals) (if (and (listp thoughts) (listp (car thoughts))) 
                                        thoughts 
                                        (if thoughts (list thoughts) nil)))
     (setf (getf signal :status) :thought)
@@ -216,16 +216,22 @@ Compares multiple proposals (from parallel backends) and selects the most consis
 *** Decide Gate
 The System 2 safety gate. Validates the candidate action against formal rules and PSF invariants.
 
+**** Phase A: Demand
+- *Need:* Ensure that malformed candidates (e.g., raw strings from System 1) do not crash the `decide` or `safety-harness` logic, which expect property lists.
+- *Success:* Coerce non-list candidates into valid `:RESPONSE` property lists before validation.
+
+**** Phase B: Blueprint
+Before passing the candidate to `decide`, the gate checks its type. If it's a string, it wraps it in `(:type :RESPONSE :payload (:text <string>))`.
+
+**** Phase D: Build
 #+begin_src lisp :tangle ../src/core.lisp
 (defun decide-gate (signal)
   "System 2: Safety and validation."
   (let ((candidate (getf signal :candidate)))
     (if candidate
-        (let ((decision (decide candidate signal)))
+        (let* ((normalized-candidate (if (listp candidate) candidate (list :type :RESPONSE :payload (list :text candidate))))
-          ;; If decision is different from candidate, it's an interception (EVENT or LOG)
+               (decision (decide normalized-candidate signal)))
-          (setf (getf signal :approved-action) decision)
+          (setf (getf signal :approved-action) decision))
-          (unless (equal decision candidate)
-            (kernel-log "GATE [Decide]: Intercepted/Rejected by System 2")))
         (setf (getf signal :approved-action) nil))
     (setf (getf signal :status) :decided)
     signal))
@@ -504,3 +510,5 @@ Following the PSF mandates, the Reactive Signal Pipeline must be empirically ver
     (is (not (null obj)))
     (is (equal (getf (org-object-attributes obj) :TITLE) "State A"))))
 #+end_src
+ual (getf (org-object-attributes obj) :TITLE) "State A"))))
+#+end_src

literate/package.org

@@ -124,3 +124,60 @@ The `package.lisp` file defines the public API of the `org-agent` kernel. It exp
    #:set-llm-model
    #:get-llm-model))
 #+end_src
+
+** Package Implementation
+#+begin_src lisp :tangle ../src/package.lisp
+(in-package :org-agent)
+#+end_src
+
+** Kernel Logging State
+#+begin_src lisp :tangle ../src/package.lisp
+(defvar *system-logs* nil)
+(defvar *logs-lock* (bt:make-lock "kernel-logs-lock"))
+(defvar *max-log-history* 100)
+#+end_src
+
+** Skills Registry
+#+begin_src lisp :tangle ../src/package.lisp
+(defvar *skills-registry* (make-hash-table :test 'equal)
+  "Global registry of all loaded skills.")
+#+end_src
+
+** Skill Telemetry State
+#+begin_src lisp :tangle ../src/package.lisp
+(defvar *skill-telemetry* (make-hash-table :test 'equal))
+(defvar *telemetry-lock* (bt:make-lock "kernel-telemetry-lock"))
+#+end_src
+
+** Cognitive Tool Registry
+#+begin_src lisp :tangle ../src/package.lisp
+(defvar *cognitive-tools* (make-hash-table :test 'equal))
+
+(defstruct cognitive-tool
+  name
+  description
+  parameters
+  guard
+  body)
+
+(defmacro def-cognitive-tool (name description parameters &key guard body)
+  `(setf (gethash (string-downcase (string ',name)) *cognitive-tools*)
+         (make-cognitive-tool :name (string-downcase (string ',name))
+                              :description ,description
+                              :parameters ',parameters
+                              :guard ,guard
+                              :body ,body)))
+#+end_src
+
+** Kernel Logging Implementation
+#+begin_src lisp :tangle ../src/package.lisp
+(defun kernel-log (msg &rest args)
+  "Centralized logging for the kernel."
+  (let ((formatted-msg (apply #'format nil msg args)))
+    (bt:with-lock-held (*logs-lock*)
+      (push formatted-msg *system-logs*)
+      (when (> (length *system-logs*) *max-log-history*)
+        (setq *system-logs* (subseq *system-logs* 0 *max-log-history*))))
+    (format t "~a~%" formatted-msg)
+    (finish-output)))
+#+end_src

literate/skills.org

@@ -164,8 +164,16 @@ Calculates the correct load order for a directory of skill filepaths, detecting
 #+end_src
 
 ** Jailed Loading (load-skill-from-org)
-The core "hot-loading" mechanism. It extracts Lisp blocks from an Org file and evaluates them within a dedicated package ("Jail"). 
+The core "hot-loading" mechanism. It extracts Lisp blocks from an Org file and evaluates them within a dedicated package ("Jail").
 
+*** Phase A: Demand
+- *Need:* Safely load skills from `.org` files without evaluating docstrings or kernel-level tangled blocks as logic.
+- *Success:* Exclude `#+begin_src lisp :tangle` blocks and ignore `:PROPERTIES:` and `:END:` drawers embedded within src blocks.
+
+*** Phase B: Blueprint
+The loader must actively scan block arguments and filter out those containing `:tangle`. It also needs to cleanly skip Org properties that Emacs might auto-generate inside src blocks.
+
+*** Phase D: Build
 #+begin_src lisp :tangle ../src/skills.lisp
 (defun load-skill-from-org (filepath)
   "Parses and evaluates Lisp blocks from an Org file into a jailed package."
@@ -183,9 +191,17 @@ The core "hot-loading" mechanism. It extracts Lisp blocks from an Org file and e
           
           (dolist (line lines)
             (let ((clean-line (string-trim '(#\Space #\Tab #\Return) line)))
-              (cond ((uiop:string-prefix-p "#+begin_src lisp" (string-downcase clean-line)) (setf in-lisp-block t))
+              (cond ((uiop:string-prefix-p "#+begin_src lisp" (string-downcase clean-line))
-                    ((uiop:string-prefix-p "#+end_src" (string-downcase clean-line)) (setf in-lisp-block nil))
+                     ;; Only load blocks that are NOT tangled to src/ or elsewhere
-                    (in-lisp-block (setf lisp-code (concatenate 'string lisp-code line (string #\Newline)))))))
+                     (if (search ":tangle" (string-downcase clean-line))
+                         (setf in-lisp-block nil)
+                         (setf in-lisp-block t)))
+                    ((uiop:string-prefix-p "#+end_src" (string-downcase clean-line))
+                     (setf in-lisp-block nil))
+                    (in-lisp-block 
+                     (unless (or (uiop:string-prefix-p ":PROPERTIES:" (string-upcase clean-line))
+                                 (uiop:string-prefix-p ":END:" (string-upcase clean-line)))
+                       (setf lisp-code (concatenate 'string lisp-code line (string #\Newline))))))))
           
           (if (= (length lisp-code) 0)
               (progn (setf (skill-entry-status entry) :ready) t) ;; Valid empty skill

literate/system-definition.org

@@ -11,35 +11,52 @@
   :version "0.1.0"
   :license "MIT"
   :description "The Neurosymbolic Lisp Machine Kernel"
-  :depends-on (:usocket :cl-json :bordeaux-threads :dexador :uiop :cl-dotenv :cl-ppcre :hunchentoot :ironclad)
+  :depends-on (:usocket :cl-json :bordeaux-threads :dexador :uiop :cl-dotenv :cl-ppcre :hunchentoot :ironclad :str)
   :serial t
-  :components ((:module "src"
+  :components ((:file "src/package")
-                :components ((:file "package")
+               (:file "src/protocol")
-                             (:file "protocol")
+               (:file "src/object-store")
-                             (:file "object-store")
+               (:file "src/embedding")
-                             (:file "embedding")
+               (:file "src/context")
-                             (:file "context")
+               (:file "src/skills")
-                             (:file "skills")
+               (:file "src/neuro")
-                             (:file "neuro")
+               (:file "src/credentials-vault")
-                             (:file "symbolic")
+               (:file "src/llm-gateway")
-                             (:file "safety-harness")
+               (:file "src/symbolic")
-                             (:file "core"))))
+               (:file "src/safety-harness")
+               (:file "src/self-fix")
+               (:file "src/lisp-repair")
+               (:file "src/bouncer")
+               (:file "src/verification-logic")
+               (:file "src/core")
+               (:file "src/gateway-telegram")
+               (:file "src/gateway-signal")
+               (:file "src/gateway-matrix")
+               (:file "src/playwright"))
   :build-operation "program-op"
   :build-pathname "org-agent-server"
-  :entry-point "org-agent:main"
+  :entry-point "org-agent:main")
-  :in-order-to ((test-op (test-op :org-agent/tests))))
 
 (defsystem :org-agent/tests
   :depends-on (:org-agent :fiveam)
-  :components ((:module "tests"
+  :components ((:file "tests/oacp-tests")
-                :components ((:file "oacp-tests")
+               (:file "tests/pipeline-tests")
-                             (:file "pipeline-tests")
+               (:file "tests/peripheral-vision-tests")
-                             (:file "peripheral-vision-tests")
+               (:file "tests/safety-harness-tests")
-                             (:file "safety-harness-tests")
+               (:file "tests/boot-sequence-tests")
-                             (:file "boot-sequence-tests")
+               (:file "tests/object-store-tests")
-                             (:file "object-store-tests")
+               (:file "tests/immune-system-tests")
-                             (:file "immune-system-tests")
+               (:file "tests/task-orchestrator-tests")
-                             (:file "chaos-qa"))))
+               (:file "tests/self-fix-tests")
+               (:file "tests/lisp-repair-tests")
+               (:file "tests/bouncer-tests")
+               (:file "tests/formal-verification-tests")
+               (:file "tests/llm-gateway-tests")
+               (:file "tests/gateway-telegram-tests")
+               (:file "tests/gateway-signal-tests")
+               (:file "tests/gateway-matrix-tests")
+               (:file "tests/playwright-tests")
+               (:file "tests/chaos-qa"))
   :perform (test-op (o s) 
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :oacp-suite :org-agent-tests))
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :pipeline-suite :org-agent-pipeline-tests))
@@ -48,5 +65,16 @@
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :boot-suite :org-agent-boot-tests))
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :object-store-suite :org-agent-object-store-tests))
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :immune-suite :org-agent-immune-system-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :task-orchestrator-suite :org-agent-task-orchestrator-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :self-fix-suite :org-agent-self-fix-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :lisp-repair-suite :org-agent-lisp-repair-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :bouncer-suite :org-agent-bouncer-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :formal-verification-suite :org-agent-formal-verification-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :llm-gateway-suite :org-agent-llm-gateway-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :shell-actuator-suite :org-agent-shell-actuator-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :gateway-telegram-suite :org-agent-gateway-telegram-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :gateway-signal-suite :org-agent-gateway-signal-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :gateway-matrix-suite :org-agent-gateway-matrix-tests))
+             (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :playwright-suite :org-agent-playwright-tests))
              (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :chaos-suite :org-agent-chaos-qa))))
 #+end_src

skills/org-skill-agent.org

@@ -90,7 +90,7 @@ We register tools for kernel introspection and state management.
 
 #+begin_src lisp
 (org-agent:def-cognitive-tool :kernel-status "Returns the current operational status of the Org-Agent kernel, including loaded skills and telemetry."
-  :parameters nil
+  nil
   :body (lambda (args)
           (declare (ignore args))
           (format nil "KERNEL STATUS:
@@ -104,7 +104,7 @@ We register tools for kernel introspection and state management.
                   org-agent:*provider-cascade*)))
 
 (org-agent:def-cognitive-tool :list-skills "Lists all currently loaded skills and their metadata."
-  :parameters nil
+  nil
   :body (lambda (args)
           (declare (ignore args))
           (let ((output "LOADED SKILLS:

skills/org-skill-chaos.org

@@ -52,17 +52,6 @@ Verify the system's stability and error-handling capabilities under stress.
     (format nil "SUCCESS - Chaos stress test initiated.")))
 #+end_src
 
-* Registration
-#+begin_src lisp
-(defskill :skill-chaos
-  :priority 10 ; Lower priority, used for background testing
-  :trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :chaos-trigger))
-  :neuro (lambda (context) 
-           (let ((p (getf context :payload)))
-             (format nil "A chaos trigger was received (~a). Should I run a stress test?" (getf p :mode))))
-  :symbolic #'chaos-stress-test)
-#+end_src
-
 
 * Phase B: Blueprint (PROTOCOL)
 :PROPERTIES:
@@ -131,3 +120,14 @@ Verify the system's stability and error-handling capabilities under stress.
   (defun chaos-force-skip-event ())
  #+end_src
  
+
+* Registration
+#+begin_src lisp
+(defskill :skill-chaos
+  :priority 10 ; Lower priority, used for background testing
+  :trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :chaos-trigger))
+  :neuro (lambda (context) 
+           (let ((p (getf context :payload)))
+             (format nil "A chaos trigger was received (~a). Should I run a stress test?" (getf p :mode))))
+  :symbolic #'chaos-stress-test)
+#+end_src

skills/org-skill-delegation.org

@@ -44,7 +44,7 @@ The Delegation Manager will utilize a message-passing architecture. Tasks are pa
 #+BEGIN_SRC lisp
 ;; Sends a task to a target.  Returns a promise (future) that will eventually resolve
 ;; to the task result.  The :delegation-id is internally managed to track the delegation lifecycle.
-(defun delegate-task (task :priority :context)
+(defun delegate-task (task priority context)
   :returns (promise task-result))
 #+END_SRC
 
@@ -54,7 +54,7 @@ The Delegation Manager will utilize a message-passing architecture. Tasks are pa
 ;; Determines the target agent/skill for a given task.
 ;; It receives the task details and any relevant context.  Returns the ID of the
 ;; targeted agent/skill.  Can return `:error` if no suitable delegation is found.
-(defun resolve-skill (task :context)
+(defun resolve-skill (task context)
   :returns (skill-id or :error))
 #+END_SRC
 
@@ -62,7 +62,7 @@ The Delegation Manager will utilize a message-passing architecture. Tasks are pa
 
 #+BEGIN_SRC lisp
 ;; Reports a delegation failure. This allows for fallback strategies.
-(defun report-delegation-failure (delegation-id :error-message)
+(defun report-delegation-failure (delegation-id error-message)
   :returns nil)
 #+END_SRC
 
@@ -70,7 +70,7 @@ The Delegation Manager will utilize a message-passing architecture. Tasks are pa
 
 #+BEGIN_SRC lisp
 ;; Informs the Delegation Manager that a task has been completed successfully.
-(defun report-task-completion (delegation-id :result)
+(defun report-task-completion (delegation-id result)
   :returns nil)
 #+END_SRC
  

skills/org-skill-safety-harness.org

@@ -127,7 +127,7 @@ We allow other skills to register safe symbols for the harness.
 ** Cognitive Tools
 #+begin_src lisp
 (org-agent:def-cognitive-tool :security-telemetry "Returns security-related telemetry, including blocked actions and harness status."
-  :parameters nil
+  nil
   :body (lambda (args)
           (declare (ignore args))
           (format nil "SAFETY HARNESS STATUS:

src/core.lisp

@@ -53,7 +53,7 @@
         (let ((neuro-fn (skill-neuro-prompt skill)))
           (if neuro-fn
               (let ((proposals (funcall neuro-fn signal)))
-                (setf (getf signal :proposals) (if (listp (first proposals)) proposals (list proposals))))
+                (setf (getf signal :proposals) (if (and (listp proposals) (listp (first proposals))) proposals (list proposals))))
               (setf (getf signal :proposals) nil)))
         (setf (getf signal :proposals) nil))
     (setf (getf signal :status) :reasoned)
@@ -94,14 +94,12 @@
                                         :context context))))
 
 (defun decide-gate (signal)
-  "System 2: Safety and validation."
+  "Stage 3: Symbolic verification (System 2)."
   (let ((candidate (getf signal :candidate)))
     (if candidate
-        (let ((decision (decide candidate signal)))
+        (let* ((normalized-candidate (if (listp candidate) candidate (list :type :RESPONSE :payload (list :text candidate))))
-          ;; If decision is different from candidate, it's an interception (EVENT or LOG)
+               (decision (decide normalized-candidate signal)))
-          (setf (getf signal :approved-action) decision)
+          (setf (getf signal :approved-action) decision))
-          (unless (equal decision candidate)
-            (kernel-log "GATE [Decide]: Intercepted/Rejected by System 2")))
         (setf (getf signal :approved-action) nil))
     (setf (getf signal :status) :decided)
     signal))

src/package.lisp

@@ -110,8 +110,9 @@
    ;; --- Symbolic Logic ---
    #:list-objects-with-attribute
    #:org-id-new
-   
+
    ;; --- AST Helpers ---
+
    #:find-headline-missing-id
    
    ;; --- Environment Config ---

src/skills.lisp

@@ -126,9 +126,17 @@
           
           (dolist (line lines)
             (let ((clean-line (string-trim '(#\Space #\Tab #\Return) line)))
-              (cond ((uiop:string-prefix-p "#+begin_src lisp" (string-downcase clean-line)) (setf in-lisp-block t))
+              (cond ((uiop:string-prefix-p "#+begin_src lisp" (string-downcase clean-line))
-                    ((uiop:string-prefix-p "#+end_src" (string-downcase clean-line)) (setf in-lisp-block nil))
+                     ;; Only load blocks that are NOT tangled to src/ or elsewhere
-                    (in-lisp-block (setf lisp-code (concatenate 'string lisp-code line (string #\Newline)))))))
+                     (if (search ":tangle" (string-downcase clean-line))
+                         (setf in-lisp-block nil)
+                         (setf in-lisp-block t)))
+                    ((uiop:string-prefix-p "#+end_src" (string-downcase clean-line))
+                     (setf in-lisp-block nil))
+                    (in-lisp-block 
+                     (unless (or (uiop:string-prefix-p ":PROPERTIES:" (string-upcase clean-line))
+                                 (uiop:string-prefix-p ":END:" (string-upcase clean-line)))
+                       (setf lisp-code (concatenate 'string lisp-code line (string #\Newline))))))))
           
           (if (= (length lisp-code) 0)
               (progn (setf (skill-entry-status entry) :ready) t) ;; Valid empty skill

tests/immune-system-tests.lisp

@@ -13,6 +13,7 @@
   "Verify that a crashing tool triggers a :tool-error stimulus."
   (clrhash org-agent::*cognitive-tools*)
   (def-cognitive-tool :crashing-tool "Always fails."
+    nil
     :body (lambda (args) (declare (ignore args)) (error "KABOOM")))
   
   (let* ((stimulus '(:type :EVENT :payload (:sensor :user-command :command :trigger-crash)))