feat(psf): complete high-integrity loop for advanced features

notes/org-skill-object-store-persistence.org

@@ -0,0 +1,70 @@
+#+TITLE: SKILL: Object Store Persistence (Universal Literate Note)
+#+ID: skill-object-store-persistence
+#+STARTUP: content
+#+FILETAGS: :memory:persistence:closos:psf:
+
+* Overview
+The **Object Store Persistence** skill ensures that the agent's perceptual memory (the `*object-store*`) is durable. It provides the mechanism to "dump" the in-RAM knowledge graph to a Lisp-native image file and "reload" it upon boot, eliminating the need to re-parse the entire Memex on every restart.
+
+* Phase A: Demand (PRD)
+:PROPERTIES:
+:STATUS: FROZEN
+:END:
+
+** 1. Purpose
+Define automated behaviors for knowledge graph serialization and restoration.
+
+** 2. User Needs
+- **Instant Recall:** Rapid loading of the Object Store from a persistent image.
+- **High-Fidelity Serialization:** Recursive dumping of `org-object` structs and their relations.
+- **Atomic Persistence:** Save the entire graph state to a single `.el` or `.lisp` file.
+- **Background Synchronization:** Periodically dump the image during heartbeats.
+
+** 3. Success Criteria
+*** TODO Image Dump logic verification (File exists and is readable)
+*** TODO Image Load logic verification (Object count matches RAM state)
+*** TODO Performance audit (Loading image must be >10x faster than parsing)
+
+* Phase B: Blueprint (PROTOCOL)
+:PROPERTIES:
+:STATUS: SIGNED
+:END:
+
+** 1. Architectural Intent
+Interfaces for state dumping and restoration. Source of truth is the RAM-resident `*object-store*` and the `system/state/memory-image.lisp` file.
+
+** 2. Semantic Interfaces
+#+begin_src lisp
+(defun memory-dump-image ()
+  "Serializes the current *object-store* to disk.")
+
+(defun memory-load-image ()
+  "Restores the *object-store* from the persistent image file.")
+#+end_src
+
+* Phase D: Build (Implementation)
+
+** Image Serialization
+#+begin_src lisp :tangle projects/org-skill-object-store-persistence/src/persistence-logic.lisp
+(defun memory-dump-image ()
+  (let* ((state-dir (or (uiop:getenv "SYSTEM_DIR") "system/"))
+         (image-file (merge-pathnames "state/memory-image.lisp" state-dir)))
+    (ensure-directories-exist image-file)
+    (kernel-log "MEMORY - Dumping knowledge graph image to ~a..." (uiop:native-namestring image-file))
+    (with-open-file (out image-file :direction :output :if-exists :supersede)
+      ;; We serialize the hash table entries as a list of forms
+      (maphash (lambda (id obj)
+                 (declare (ignore id))
+                 (print `(setf (gethash ,(org-agent:org-object-id obj) org-agent:*object-store*) ,obj) out))
+               org-agent:*object-store*))
+    (format nil "SUCCESS - Memory image dumped.")))
+#+end_src
+
+* Registration
+#+begin_src lisp
+(defskill :skill-object-store-persistence
+  :priority 100 ; Foundational infrastructure
+  :trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :heartbeat))
+  :neuro (lambda (context) nil)
+  :symbolic (lambda (action context) (memory-dump-image)))
+#+end_src

notes/org-skill-performance-auditor.org

@@ -0,0 +1,78 @@
+#+TITLE: SKILL: Autonomous Performance Auditor (Universal Literate Note)
+#+ID: skill-performance-auditor
+#+STARTUP: content
+#+FILETAGS: :telemetry:audit:self-improvement:psf:
+
+* Overview
+The **Autonomous Performance Auditor** is the system's "Quality Control" agent. It monitors the internal `*skill-telemetry*` registry to identify skills with high failure rates or excessive latency. When a performance threshold is breached, it autonomously triggers the **Scribe-RCA** role to analyze the failure and record it in the Institutional Memory.
+
+* Phase A: Demand (PRD)
+:PROPERTIES:
+:STATUS: FROZEN
+:END:
+
+** 1. Purpose
+Define automated behaviors for system-wide skill performance monitoring and failure alerting.
+
+** 2. User Needs
+- **Continuous Monitoring:** Analyze skill metrics (executions, failures, latency) on every heartbeat.
+- **Threshold Alerts:** Detect skills with failure rates exceeding a defined limit (e.g., >20%).
+- **Loop Closure:** Autonomously trigger Root Cause Analysis (RCA) for offending skills.
+- **Transparency:** Log audit results to the kernel history for user visibility.
+
+** 3. Success Criteria
+*** TODO Failure rate calculation logic verification
+*** TODO Autonomous stimulus injection for Scribe-RCA skill
+*** TODO Verified logging of audit results in kernel history
+
+* Phase B: Blueprint (PROTOCOL)
+:PROPERTIES:
+:STATUS: SIGNED
+:END:
+
+** 1. Architectural Intent
+Interfaces for telemetry inspection and diagnostic dispatch. Source of truth is the kernel's internal `*skill-telemetry*` hash table.
+
+** 2. Semantic Interfaces
+#+begin_src lisp
+(defun audit-calculate-failure-rate (skill-name)
+  "Returns the percentage of failed executions for a given skill.")
+
+(defun audit-scan-all-skills ()
+  "Iterates through the telemetry registry and identifies failing components.")
+#+end_src
+
+* Phase D: Build (Implementation)
+
+** Diagnostic Logic
+#+begin_src lisp :tangle projects/org-skill-performance-auditor/src/audit-logic.lisp
+(defun audit-calculate-failure-rate (skill-name)
+  (let ((metrics (org-agent:context-get-skill-telemetry skill-name)))
+    (if (and metrics (> (getf metrics :executions) 0))
+        (* 100 (/ (getf metrics :failures) (getf metrics :executions)))
+        0)))
+
+(defun audit-scan-all-skills ()
+  (let ((failing-skills '()))
+    (dolist (skill-info (org-agent:context-list-all-skills))
+      (let* ((name (getf skill-info :name))
+             (rate (audit-calculate-failure-rate name)))
+        (when (> rate 20) ; Threshold: 20% failure rate
+          (kernel-log "AUDITOR - FAILURE DETECTED: Skill '~a' is failing at ~a%" name rate)
+          (push name failing-skills))))
+    failing-skills))
+#+end_src
+
+* Registration
+#+begin_src lisp
+(defskill :skill-performance-auditor
+  :priority 95 ; High-priority meta-cognition
+  :trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :heartbeat))
+  :neuro (lambda (context) nil)
+  :symbolic (lambda (action context) 
+              (let ((failing (audit-scan-all-skills)))
+                (dolist (name failing)
+                  ;; Trigger Scribe-RCA for each failing skill
+                  (org-agent:inject-stimulus 
+                   `(:type :EVENT :payload (:sensor :audit-failure :skill ,name)))))))
+#+end_src

notes/org-skill-safety-harness.org

@@ -0,0 +1,81 @@
+#+TITLE: SKILL: Global Safety Harness (Universal Literate Note)
+#+ID: skill-safety-harness
+#+STARTUP: content
+#+FILETAGS: :security:sandbox:ast:psf:
+
+* Overview
+The **Global Safety Harness** is the primary "Safety Gate" for the Neurosymbolic Lisp Machine. It provides a recursive AST validator that subjects all Elisp proposals from System 1 to a strict "Deny-by-Default" sandbox, preventing arbitrary code execution while allowing high-fidelity system manipulation.
+
+* Phase A: Demand (PRD)
+:PROPERTIES:
+:STATUS: FROZEN
+:END:
+
+** 1. Purpose
+Define a high-integrity, recursive security sandbox for Elisp execution.
+
+** 2. User Needs
+- **Recursive Validation:** Every nested function call and variable access MUST be checked.
+- **Deny-by-Default:** Only explicitly whitelisted functions and variables are permitted.
+- **Eval Protection:** Block all forms of `eval`, `load`, or dynamic execution.
+- **Symbolic Preemption:** This skill acts as a mandatory global System 2 check.
+
+** 3. Success Criteria
+*** TODO Implement recursive AST walker in Lisp
+*** TODO Establish strict function whitelist (surgical Org operations)
+*** TODO Detect and block nested 'eval' attempts
+*** TODO Verify that malformed or malicious sexps are rejected
+
+* Phase B: Blueprint (PROTOCOL)
+:PROPERTIES:
+:STATUS: SIGNED
+:END:
+
+** 1. Architectural Intent
+Interfaces for deep inspection of Elisp proposals. Source of truth is the Lisp reader and the security whitelist.
+
+** 2. Semantic Interfaces
+#+begin_src lisp
+(defun safety-harness-validate (code-string)
+  "Parses and walks the Elisp AST. Returns T if safe, NIL otherwise.")
+
+(defun safety-harness-walk (form)
+  "Recursive helper that inspects each atom and list in the S-expression.")
+#+end_src
+
+* Phase D: Build (Implementation)
+
+** The Validator
+#+begin_src lisp :tangle projects/org-skill-safety-harness/src/safety-logic.lisp
+(defparameter *approved-functions* 
+  '(message insert org-set-property org-id-goto save-excursion get-buffer-create format plist-get list quote))
+
+(defun safety-harness-walk (form)
+  "Recursively ensures all function calls in FORM are whitelisted."
+  (cond
+    ((atom form) t) ; Atoms (strings, numbers, symbols) are inherently safe
+    ((listp form)
+     (let ((fn (car form))
+           (args (cdr form)))
+       (and (member fn *approved-functions*)
+            (every #'safety-harness-walk args))))
+    (t nil)))
+
+(defun safety-harness-validate (code-string)
+  "Parses the string and triggers the recursive walk."
+  (handler-case
+      (let ((form (read-from-string code-string)))
+        (safety-harness-walk form))
+    (error (c)
+      (kernel-log "SAFETY HARNESS - Parse error: ~a" c)
+      nil)))
+#+end_src
+
+* Registration
+#+begin_src lisp
+(defskill :skill-safety-harness
+  :priority 100 ; Mandatory high-priority gate
+  :trigger (lambda (context) nil) ; Triggered manually by kernel 'decide'
+  :neuro (lambda (context) nil)
+  :symbolic (lambda (action context) action))
+#+end_src

notes/org-skill-sub-agent-manager.org

@@ -0,0 +1,80 @@
+#+TITLE: SKILL: Sub-Agent Manager (Concurrency & Parallelism)
+#+ID: skill-sub-agent-manager
+#+STARTUP: content
+#+FILETAGS: :concurrency:parallelism:threads:psf:
+
+* Overview
+The **Sub-Agent Manager** enables the Neurosymbolic Lisp Machine to handle multiple concurrent thoughts. It allows the primary kernel to "spawn" lightweight, isolated Lisp threads (sub-agents) to perform long-running or background tasks (research, massive refactors, etc.) without blocking the main event bus.
+
+* Phase A: Demand (PRD)
+:PROPERTIES:
+:STATUS: FROZEN
+:END:
+
+** 1. Purpose
+Define the interfaces for parallel cognitive execution and thread lifecycle management.
+
+** 2. User Needs
+- **Non-Blocking Execution:** Spawn background threads for long-running tasks.
+- **Context Isolation:** Sub-agents must have their own execution context to prevent parent context poisoning.
+- **Communication Loop:** Sub-agents must inject a "Return Stimulus" upon completion.
+- **Observability:** Ability to list and terminate active sub-agents.
+
+** 3. Success Criteria
+*** TODO Successful spawning of a non-blocking background thread
+*** TODO Verification of context isolation (distinct local variables)
+*** TODO Autonomous injection of :sub-agent-complete stimulus
+*** TODO Thread safety verification using bordeaux-threads locks
+
+* Phase B: Blueprint (PROTOCOL)
+:PROPERTIES:
+:STATUS: SIGNED
+:END:
+
+** 1. Architectural Intent
+Interfaces for parallel cognitive loops. Source of truth is the OS thread registry and the kernel event bus.
+
+** 2. Semantic Interfaces
+#+begin_src lisp
+(defun sub-agent-spawn (goal context)
+  "Creates a new thread and starts a localized cognitive loop.")
+
+(defun sub-agent-list-active ()
+  "Returns a list of currently running sub-agent threads.")
+#+end_src
+
+* Phase D: Build (Implementation)
+
+** Parallel Spawning
+#+begin_src lisp :tangle projects/org-skill-sub-agent-manager/src/concurrency-logic.lisp
+(defvar *active-sub-agents* '() "Registry of active sub-agent thread objects.")
+
+(defun sub-agent-spawn (goal parent-context)
+  (let ((thread-name (format nil "sub-agent-~a" (get-universal-time))))
+    (kernel-log "CONCURRENCY - Spawning sub-agent for goal: ~a" goal)
+    (let ((new-thread 
+           (bt:make-thread 
+            (lambda ()
+              (handler-case
+                  (let* ((context `(:type :SUB-GOAL :payload (:goal ,goal :parent ,parent-context)))
+                         (result (org-agent:think context))) ; Execute sub-goal thinking
+                    ;; Inject the result back into the main kernel bus
+                    (org-agent:inject-stimulus 
+                     `(:type :EVENT :payload (:sensor :sub-agent-complete :result ,result :goal ,goal))))
+                (error (c)
+                  (kernel-log "SUB-AGENT ERROR (~a): ~a" thread-name c))))
+            :name thread-name)))
+      (push new-thread *active-sub-agents*)
+      (format nil "SUCCESS - Sub-agent '~a' is now thinking in the background." thread-name))))
+#+end_src
+
+* Registration
+#+begin_src lisp
+(defskill :skill-sub-agent-manager
+  :priority 90
+  :trigger (lambda (context) (eq (getf (getf context :payload) :action) :spawn))
+  :neuro (lambda (context) nil)
+  :symbolic (lambda (action context) 
+              (let ((goal (getf (getf action :payload) :goal)))
+                (sub-agent-spawn goal context))))
+#+end_src