DOCS: Systematic overhaul of Literate source (Granularity & Technical Reasoning)

skills/org-skill-credentials-vault.org

@@ -1,67 +1,24 @@
-:PROPERTIES:
-:ID:       credentials-vault-skill
-:CREATED:  [2026-04-09 Thu]
-:END:
 #+TITLE: SKILL: Credentials Vault (Universal Literate Note)
-#+STARTUP: content
+#+AUTHOR: Amr
 #+FILETAGS: :auth:security:infrastructure:autonomy:
-#+DEPENDS_ON: id:state-persistence-skill
+#+STARTUP: content
 
 * Overview
 The *Credentials Vault* is the high-security enclave for the OpenCortex. It centralizes the management of LLM API keys, OAuth sessions, and browser cookies. By consolidating these into a single vault, we ensure that sensitive tokens are handled with uniform masking, validation, and Merkle-integrated persistence.
 
-* Phase A: Demand (PRD)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
-
-** 1. Purpose
-Securely manage all authentication tokens required for the opencortex to operate.
-
-** 2. User Needs
-- *Unified Storage:* Single interface for API keys and Session Cookies.
-- *Masked Logging:* Ensure credentials never appear in plaintext in `harness-log`.
-- *Guided Onboarding:* Retain and improve the Google/Gemini cookie handshake.
-- *Persistence:* Securely save credentials to the Memory via Merkle-Tree snapshots.
-
-* Phase B: Blueprint (PROTOCOL)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
-
-** 1. Architectural Intent
+** Architectural Intent: The Secure Enclave
 The vault provides a secure lookup table in RAM, backed by the persistent Memory. Access is restricted to internal kernel requests and explicitly authorized deterministic gates.
 
-** 2. Semantic Interfaces
-#+begin_src lisp
-(defun vault-get-secret (provider &key type)
-  "Retrieves a secret (api-key or session) for a provider.")
-
-(defun vault-set-secret (provider secret &key type)
-  "Securely stores a secret and triggers a Merkle snapshot.")
-#+end_src
-
-* Phase C: Success (QUALITY)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
-
-** 1. Success Criteria
-- [ ] *No Plaintext Leaks:* Log output must use `[REDACTED]` for sensitive values.
-- [ ] *Merkle Integration:* Setting a secret must increment the Memory version.
-- [ ] *Dual-Path Auth:* Support both `:api-key` and `:session-cookies`.
-- [ ] *Onboarding Verification:* The cookie handshake successfully hydrates the vault.
-
-** 2. TDD Plan
-Tests in `tests/vault-tests.lisp` will verify:
-1. Retrieval of keys from both `.env` (fallback) and Vault (primary).
-2. Redaction of keys in log strings.
-3. Successful version increment in the Memory after `vault-set-secret`.
-
-* Phase D: Build (Implementation)
-
-** Package Context
+The primary goal of the vault is to prevent "Credential Bleed"—the accidental leaking of API keys into logs, terminal history, or neural contexts. It achieves this by providing a unified getter that automatically masks its output for diagnostic use.
+
+* Implementation
+
+** Package Initialization
 #+begin_src lisp
+(in-package :cl-user)
+(defpackage :opencortex.skills.org-skill-credentials-vault
+  (:use :cl :opencortex))
+(in-package :opencortex.skills.org-skill-credentials-vault)
 #+end_src
 
 ** Vault State
@@ -69,31 +26,33 @@ We maintain an in-memory hash table for secrets, which is hydrated from and pers
 
 #+begin_src lisp
 (defvar opencortex::*vault-memory* (make-hash-table :test 'equal)
-  "In-memory cache of sensitive credentials.")
+  "In-memory cache of sensitive credentials, preventing constant disk I/O for auth.")
 #+end_src
 
-** Helper: Secret Masking
-The `vault-mask-string` function ensures that diagnostic output never contains the full plaintext of a sensitive token.
+** Helper: Secret Masking (vault-mask-string)
+Ensures that diagnostic output never contains the full plaintext of a sensitive token. Used by the harness and gateways for transparent but safe logging.
 
 #+begin_src lisp
 (defun vault-mask-string (str)
-  "Returns a masked version of a sensitive string."
+  "Returns a masked version of a sensitive string. (e.g. sk-a...3f9)"
   (if (and str (> (length str) 8))
       (format nil "~a...~a" (subseq str 0 4) (subseq str (- (length str) 4)))
       "[REDACTED]"))
 #+end_src
 
 ** Retrieval (vault-get-secret)
-This function is the secure getter for all system secrets. It prioritizes the Vault (Memory) and falls back to environment variables for legacy compatibility.
+The secure getter for all system secrets. It follows a strict priority:
+1. **Vault Memory:** High-integrity, versioned storage.
+2. **Environment Fallback:** OS-level variables for bootstrap and legacy compatibility.
 
 #+begin_src lisp
 (defun vault-get-secret (provider &key (type :api-key))
   "Retrieves a credential. Type can be :api-key or :session."
   (let* ((key (format nil "~a-~a" provider type))
          (val (gethash key opencortex::*vault-memory*)))
-    (if val
+    (if (and val (not (string= val "")))
         val
-        ;; Fallback to environment
+        ;; Fallback to environment mapping
         (let ((env-var (case provider
                          ((:gemini :gemini-api) "GEMINI_API_KEY")
                          (:openai "OPENAI_API_KEY")
@@ -110,73 +69,39 @@ This function is the secure getter for all system secrets. It prioritizes the Va
 #+end_src
 
 ** Persistence (vault-set-secret)
-When a secret is updated, we immediately snapshot the Memory to ensure the credential change is versioned and durable.
+When a secret is updated, we immediately snapshot the Memory to ensure the change is versioned and durable.
 
 #+begin_src lisp
 (defun vault-set-secret (provider secret &key (type :api-key))
-  "Securely stores a secret and triggers a Merkle snapshot."
+  "Securely stores a secret and triggers a Merkle snapshot for durability."
   (let ((key (format nil "~a-~a" provider type)))
     (setf (gethash key opencortex::*vault-memory*) secret)
-    (harness-log "VAULT - Updated ~a for ~a. Triggering Merkle snapshot..." type provider)
+    (harness-log "VAULT: Updated ~a for ~a. Snapshotting memory." type provider)
     (snapshot-memory)
     t))
 #+end_src
 
-** Onboarding Logic
-Retained from the legacy Google skill, this provides the instructions for the autonomous cookie handshake.
+** Automated Onboarding Instructions
+Provides instructions for the autonomous cookie handshake (retained from legacy components).
 
 #+begin_src lisp
 (defun vault-onboard-gemini-web ()
-  "Instructions for the Autonomous Cookie Handshake."
+  "Displays instructions for the Gemini Web cookie handshake."
   (harness-log "--- GEMINI WEB ONBOARDING ---")
   (harness-log "1. Visit gemini.google.com")
   (harness-log "2. Run the 'Get Gemini Cookies' Bookmarklet.")
   (harness-log "   CODE: javascript:(function(){const c=document.cookie.split('; ').reduce((r,v)=>{const [n,val]=v.split('=');r[n]=val;return r},{});const target=['__Secure-1PSID','__Secure-1PSIDTS'];const out=target.map(n=>({name:n,value:c[n]}));prompt('Copy JSON:',JSON.stringify(out));})();")
-  (harness-log "PLATFORM GUIDE: Chrome/Firefox/Safari all support Bookmarklets via 'Add Page' or 'New Bookmark'.")
   t)
 #+end_src
 
-** Registration
+** Skill Registration
 #+begin_src lisp
-(progn
-  (defskill :skill-credentials-vault
-    :priority 200 ; High priority, foundational
-    :trigger (lambda (ctx) (eq (getf (getf ctx :payload) :sensor) :onboarding-request))
-    :probabilistic nil
-    :deterministic (lambda (action ctx) 
-                (vault-onboard-gemini-web)
-                action)))
+(defskill :skill-credentials-vault
+  :priority 200 ; Foundational Priority
+  :trigger (lambda (ctx) (eq (getf (getf ctx :payload) :sensor) :onboarding-request))
+  :probabilistic nil
+  :deterministic (lambda (action ctx) 
+              (declare (ignore ctx))
+              (vault-onboard-gemini-web)
+              action))
 #+end_src
-
-* Phase E: Chaos (Verification)
-
-Note: Tests disabled in jail load.
-
-** 1. Unit Tests (FiveAM)
-#+begin_src lisp
-#|
-(defpackage :opencortex-vault-tests
-  (:use :cl :fiveam :opencortex))
-(in-package :opencortex-vault-tests)
-
-(def-suite vault-suite :description "Tests for the Credentials Vault.")
-(in-suite vault-suite)
-
-(test test-masking
-  (is (equal "sk-t...-key" (opencortex::vault-mask-string "sk-test-key")))
-  (is (equal "[REDACTED]" (opencortex::vault-mask-string "short"))))
-
-(test test-vault-persistence
-  "Verify that setting a secret triggers a snapshot (mock check)."
-  (let ((old-version (opencortex::org-object-version (gethash "root" *memory*))))
-    (opencortex:vault-set-secret :test "secret-val")
-    (is (> (opencortex::org-object-version (gethash "root" *memory*)) old-version))))
-|#
-#+end_src
-
-** 2. Chaos Scenarios
-- *Scenario A (Vault Poisoning):* Inject a malformed session string and verify the `llm-gateway` detects the invalid format and returns a standardized error instead of crashing.
-- *Scenario B (Memory Wipe):* Clear `opencortex::*vault-memory*` during runtime and verify the vault successfully re-hydrates from the Memory (or environment fallback).
-
-* Phase F: Memory (RCA)
-- *[2026-04-09 Thu]:* Consolidated `auth-api-key` and `auth-google-oauth` into this vault. Introduced mandatory masking for all credential-related logging.

skills/org-skill-gardener.org

@@ -1,60 +1,43 @@
-:PROPERTIES:
-:ID:       gardener-skill
-:CREATED:  [2026-04-13 Mon 18:50]
-:END:
 #+TITLE: SKILL: Autonomous Gardener (Memex Maintenance)
-#+STARTUP: content
+#+AUTHOR: Amr
 #+FILETAGS: :gardener:maintenance:memex:autonomy:
+#+STARTUP: content
 
 * Overview
 The *Autonomous Gardener* is the metabolic immune system of the Memex. It autonomously audits the knowledge graph for structural decay—broken links, orphaned nodes, and missing metadata—ensuring that the system remains coherent and navigatable over long horizons.
 
-* Phase A: Demand (PRD)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
+** Architectural Intent: Graph Integrity
+In a self-evolving Memex, structural decay is inevitable. Links break as notes are renamed, and nodes become orphaned as projects are abandoned. The Gardener ensures that the "Vibe" of the Memex remains healthy by:
+1. **Auditing:** Identifying broken `id:` links.
+2. **Analysis:** Flagging nodes with zero inbound or outbound connections (Orphans).
+3. **Reporting:** Logging structural issues for user review or future autonomous repair.
 
-** 1. Purpose
-Maintain the structural integrity and "Vibe" of the Memex through autonomous auditing and self-repair proposals.
+* Implementation
 
-** 2. Success Criteria
-- [ ] *Link Audit:* Detect `id:` links that point to non-existent objects.
-- [ ] *Orphan Detection:* Identify headlines that have zero inbound or outbound connections.
-- [ ] *Reporting:* Log structural issues or propose "Flight Plans" for manual repair.
-
-* Phase B: Blueprint (PROTOCOL)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
-
-** 1. Architectural Intent
-The Gardener runs on a low-priority heartbeat. It performs a "Deep Audit" of the entire `*memory*` graph. Unlike the Scribe, which creates new data, the Gardener focuses on the *relationships* between existing data.
-
-** 2. Semantic Interfaces
-- Trigger: `(:sensor :heartbeat)`
-- Action (Repair): `(:type :REQUEST :target :emacs :action :update-node :id "..." :attributes (...))`
-
-* Phase D: Build (Implementation)
-
-** Package Context
+** Package Initialization
 #+begin_src lisp
-(in-package :opencortex)
+(in-package :cl-user)
+(defpackage :opencortex.skills.org-skill-gardener
+  (:use :cl :opencortex))
+(in-package :opencortex.skills.org-skill-gardener)
 #+end_src
 
 ** State: Maintenance Cycle
-We track the last audit time to ensure the Gardener doesn't over-consume resources.
+To minimize system overhead, the Gardener only performs a full audit pass periodically.
 
 #+begin_src lisp
 (defvar *gardener-last-audit* 0
   "The universal-time of the last full Memex audit.")
 #+end_src
 
-** Audit: Broken Links
-Scans the content of all objects for `id:` links and verifies the targets exist.
+* The Audit Engine
+
+** Link Verification (gardener-find-broken-links)
+This function performs deep packet inspection of the Memory graph. It utilizes regular expressions to find Org-mode ID links and verifies their targets against the live object registry.
 
 #+begin_src lisp
 (defun gardener-find-broken-links ()
-  "Returns a list of broken ID links found in the Memex."
+  "Scans all objects in memory for broken internal ID links."
   (let ((broken nil))
     (maphash (lambda (id obj)
                (let ((content (org-object-content obj)))
@@ -66,12 +49,12 @@ Scans the content of all objects for `id:` links and verifies the targets exist.
     broken))
 #+end_src
 
-** Audit: Orphaned Nodes
-Identifies nodes that are not linked to and do not link to anything else.
+** Orphan Detection (gardener-find-orphans)
+Structural isolation limits the effectiveness of semantic reasoning. This function maps the entire graph topology to identify nodes that have effectively "fallen off" the Memex.
 
 #+begin_src lisp
 (defun gardener-find-orphans ()
-  "Returns a list of IDs for headlines that are structurally isolated."
+  "Identifies nodes with zero connectivity in the knowledge graph."
   (let ((inbound (make-hash-table :test 'equal))
         (outbound (make-hash-table :test 'equal))
         (orphans nil))
@@ -92,12 +75,14 @@ Identifies nodes that are not linked to and do not link to anything else.
     orphans))
 #+end_src
 
-** Skill Logic: The Audit Pass
-The Gardener's deterministic gate performs the actual analysis and logs the results. In future versions, it will generate probabilistic repair proposals.
+* Metabolic Integration
+
+** Main Audit Gate (gardener-deterministic-gate)
+The primary execution hook. It performs the audit and translates technical findings into human-readable logs for the harness.
 
 #+begin_src lisp
 (defun gardener-deterministic-gate (action context)
-  "Main gate for the Gardener skill. Audits graph integrity."
+  "Main gate for the Gardener skill. Audits graph integrity and logs reports."
   (declare (ignore action context))
   (let ((broken (gardener-find-broken-links))
         (orphans (gardener-find-orphans)))
@@ -113,8 +98,8 @@ The Gardener's deterministic gate performs the actual analysis and logs the resu
         (harness-log "  [ORPHAN] Node ~a is isolated." orphan)))
     
     (setf *gardener-last-audit* (get-universal-time))
-    ;; Return a log to stop the loop
-    (list :type :LOG :payload (list :text "Gardener audit complete."))))
+    ;; Stop the pipeline by returning a Log event.
+    (list :type :LOG :payload (list :text "Gardener audit pass complete."))))
 #+end_src
 
 ** Skill Registration
@@ -125,7 +110,7 @@ The Gardener's deterministic gate performs the actual analysis and logs the resu
              (let* ((payload (getf ctx :payload))
                     (sensor (getf payload :sensor)))
                (and (eq sensor :heartbeat)
-                    ;; Only audit once per day
+                    ;; Optimization: Only audit once every 24 hours
                     (> (- (get-universal-time) *gardener-last-audit*) 86400))))
   :probabilistic nil
   :deterministic #'gardener-deterministic-gate)

skills/org-skill-llm-gateway.org

@@ -1,35 +1,35 @@
-:PROPERTIES:
-:ID:       llm-gateway-skill
-:CREATED:  [2026-04-09 Thu]
-:EDITED:   [2026-04-19 Sun]
-:END:
 #+TITLE: SKILL: Unified LLM Gateway (Universal Literate Note)
-#+STARTUP: content
+#+AUTHOR: Amr
 #+FILETAGS: :llm:gateway:infrastructure:autonomy:
-#+DEPENDS_ON: org-skill-credentials-vault
+#+STARTUP: content
 
 * Overview
 The *Unified LLM Gateway* is the single sensory and reasoning interface for all neural backends. It consolidates the previously fragmented provider skills into a high-integrity dispatch layer, standardizing credential management, error handling, and payload formatting.
 
-* Phase B: Blueprint (PROTOCOL)
+** Architectural Intent: The Neural Dispatch
+The gateway utilizes a functional dispatch pattern. A single entry point, ~execute-llm-request~, resolves the provider-specific nuances (URLs, headers, JSON structures) while exposing a uniform interface to the harness.
 
-** 1. Architectural Intent
-The gateway utilizes a functional dispatch pattern. A single entry point, `execute-llm-request`, resolves the provider-specific nuances (URLs, headers, JSON structures) while exposing a uniform interface to the harness.
+By abstracting the provider details, we allow the agent to swap "brains" mid-thought based on cost, speed, or task complexity without any change to the core reasoning logic.
 
-* Phase D: Build (Implementation)
+* Implementation
 
-** Implementation
+** Package Initialization
 #+begin_src lisp
 (in-package :cl-user)
 (defpackage :opencortex.skills.org-skill-llm-gateway
   (:use :cl :opencortex))
 (in-package :opencortex.skills.org-skill-llm-gateway)
+#+end_src
 
+** Data Extraction Helper (get-nested)
+JSON responses from different providers vary wildly in their nesting depth. ~get-nested~ provides a robust, recursive mechanism to extract values from deeply nested alists, shielding the gateway from parsing errors.
+
+#+begin_src lisp
 (defun get-nested (alist &rest keys)
   "Recursively extracts nested values from an alist, handling both objects and arrays."
   (let ((val alist))
     (dolist (k keys)
-      ;; Descend into arrays (cl-json style: ((key . val)) or ( ( (key . val) ) ))
+      ;; Handle cl-json style arrays and nested alists
       (loop while (and (listp val) (listp (car val)) (not (keywordp (caar val))))
             do (setf val (car val)))
       (let ((pair (or (assoc k val)
@@ -39,7 +39,15 @@ The gateway utilizes a functional dispatch pattern. A single entry point, `execu
             (setf val (cdr pair))
             (return-from get-nested nil))))
     val))
+#+end_src
 
+** Unified Request Router (execute-llm-request)
+The primary entry point for all neural reasoning. It handles:
+1. *Credential Retrieval:* Securely fetching keys from the Vault.
+2. *Cascade Fallback:* (Logic for future expansion).
+3. *Provider Normalization:* Translating a generic prompt into provider-specific JSON.
+
+#+begin_src lisp
 (defun execute-llm-request (prompt system-prompt &key provider model)
   "Unified entry point for all LLM providers. Respects the global cascade."
   (let* ((active-provider (or provider (car opencortex::*provider-cascade*) :openrouter))
@@ -49,29 +57,23 @@ The gateway utilizes a functional dispatch pattern. A single entry point, `execu
     (harness-log "PROBABILISTIC ENGINE: Requesting ~a (Model: ~s)" 
                 active-provider (or model "default"))
 
-    ;; If the specifically requested provider has no key, try falling back to the cascade
+    ;; Guard: API Key Verification
     (when (or (null api-key) (string= api-key ""))
-       (harness-log "GATEWAY: Provider ~a has no key. Cascade fallback would trigger here." active-provider)
+       (harness-log "GATEWAY ERROR: Provider ~a has no key." active-provider)
        (return-from execute-llm-request (list :status :error :message "API Key missing.")))
 
     (case active-provider
-      (:gemini-web
-       (let ((res (uiop:symbol-call :opencortex.skills.org-skill-web-research :ask-gemini-web full-prompt)))
-         (if res (list :status :success :content res) (list :status :error :message "Web Research Failure"))))
-      
       (:ollama
        (let* ((host (or (uiop:getenv "OLLAMA_HOST") "localhost:11434"))
               (url (format nil "http://~a/api/generate" host))
               (body (cl-json:encode-json-to-string `((model . ,(or model "llama3")) (prompt . ,full-prompt) (stream . :false)))))
          (handler-case 
-             (progn
-               (harness-log "LLM DEBUG: Requesting Ollama...")
-               (let* ((response (dex:post url :headers '(("Content-Type" . "application/json")) :content body :connect-timeout 5 :read-timeout 60))
-                      (json (cl-json:decode-json-from-string response)))
-                 (list :status :success :content (cdr (assoc :response json)))))
+             (let* ((response (dex:post url :headers '(("Content-Type" . "application/json")) :content body :connect-timeout 5 :read-timeout 60))
+                    (json (cl-json:decode-json-from-string response)))
+               (list :status :success :content (cdr (assoc :response json))))
            (error (c) (list :status :error :message (format nil "Ollama Failure: ~a" c))))))
 
-      (t ;; Cloud Providers (Anthropic, Gemini API, Groq, OpenAI, OpenRouter)
+      (t ;; Cloud Provider Normalization (Anthropic, Gemini, OpenAI, OpenRouter)
        (let* ((endpoint (case active-provider
                           (:anthropic "https://api.anthropic.com/v1/messages")
                           (:gemini-api (format nil "https://generativelanguage.googleapis.com/v1/models/~a:generateContent" (or model "gemini-1.5-flash-latest")))
@@ -90,20 +92,22 @@ The gateway utilizes a functional dispatch pattern. A single entry point, `execu
                       (t (cl-json:encode-json-to-string `((model . ,(or model (case active-provider (:groq "llama-3.3-70b-versatile") (t "google/gemini-2.0-flash-001"))))
                                                          (messages . (( (role . "system") (content . ,system-prompt) ) ( (role . "user") (content . ,prompt) )))))))))
          (handler-case 
-             (progn
-               (harness-log "LLM DEBUG: Requesting ~a..." active-provider)
-               (let* ((response (dex:post endpoint :headers headers :content body :connect-timeout 10 :read-timeout 30))
-                      (json (cl-json:decode-json-from-string response)))
-                 (let ((content (case active-provider
+             (let* ((response (dex:post endpoint :headers headers :content body :connect-timeout 10 :read-timeout 30))
+                    (json (cl-json:decode-json-from-string response)))
+               (let ((content (case active-provider
                                   (:anthropic (get-nested json :content :text))
                                   (:gemini-api (get-nested json :candidates :parts :text))
                                   (t (get-nested json :choices :message :content)))))
-                   (if content
-                       (list :status :success :content content)
-                       (list :status :error :message (format nil "Failed to parse ~a response structure." active-provider))))))
+                 (if content
+                     (list :status :success :content content)
+                     (list :status :error :message (format nil "Failed to parse ~a response structure." active-provider)))))
            (error (c) (list :status :error :message (format nil "LLM Gateway Failure (~a): ~a" active-provider c)))))))))
+#+end_src
 
-;; Initialize Cascade
+** Cascade Initialization
+The provider cascade determines the failover logic for the agent's cognition.
+
+#+begin_src lisp
 (let* ((env-cascade (uiop:getenv "PROVIDER_CASCADE"))
        (default-list '(:openrouter :openai :anthropic :groq :gemini-api :ollama))
        (final-list (if (and env-cascade (not (string= env-cascade "")))
@@ -112,12 +116,23 @@ The gateway utilizes a functional dispatch pattern. A single entry point, `execu
                        default-list)))
   (setf opencortex::*provider-cascade* final-list)
   (opencortex:harness-log "PROBABILISTIC: Neural Cascade Initialized -> ~a" final-list))
+#+end_src
 
-;; Register Providers
+** Backend Registration
+Registers all supported providers into the core ~*probabilistic-backends*~ registry.
+
+#+begin_src lisp
 (dolist (p '(:anthropic :gemini-api :gemini-web :groq :ollama :openrouter :openai))
   (opencortex:register-probabilistic-backend p (lambda (prompt system-prompt &key model)
                                         (execute-llm-request prompt system-prompt :provider p :model model))))
+#+end_src
 
+* Cognitive Tool Integration
+
+** The ask-llm Tool
+Provides the agent with the physical capability to query additional neural contexts.
+
+#+begin_src lisp
 (def-cognitive-tool :ask-llm 
   "Queries an LLM provider via the unified gateway."
   ((:prompt :type :string :description "The user prompt.")
@@ -129,10 +144,13 @@ The gateway utilizes a functional dispatch pattern. A single entry point, `execu
                                (or (getf args :system-prompt) "You are a helpful assistant.")
                                :provider (getf args :provider)
                                :model (getf args :model))))
+#+end_src
 
+** Skill Registration
+#+begin_src lisp
 (defskill :skill-llm-gateway
   :priority 150
-  :trigger (lambda (context) (declare (ignore context)) nil)
+  :trigger (lambda (context) (declare (ignore context)) nil) ; Passive responder
   :probabilistic (lambda (context) (declare (ignore context)) nil)
   :deterministic (lambda (action context) (declare (ignore context)) action))
 #+end_src

skills/org-skill-scribe.org

@@ -1,78 +1,63 @@
-:PROPERTIES:
-:ID:       scribe-skill
-:CREATED:  [2026-04-13 Mon 18:40]
-:END:
 #+TITLE: SKILL: Autonomous Scribe (Knowledge Distillation)
-#+STARTUP: content
+#+AUTHOR: Amr
 #+FILETAGS: :scribe:distillation:memex:autonomy:
+#+STARTUP: content
 
 * Overview
-The *Autonomous Scribe* is the background architect of the Memex. It is responsible for the "Nightly Distillation": a process that scans chronological daily logs, extracts evergreen concepts, and formalizes them into atomic Zettelkasten notes.
+The *Autonomous Scribe* is the background architect of the Memex. Its primary responsibility is the "Nightly Distillation": a process that scans chronological daily logs, extracts evergreen concepts, and formalizes them into atomic Zettelkasten notes.
 
-* Phase A: Demand (PRD)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
+** Architectural Intent: Continuous Distillation
+The Scribe transforms the "Noise" of daily streams into the "Signal" of permanent knowledge. By operating in the background, it ensures that your knowledge graph grows autonomously, even when you aren't actively organizing it.
 
-** 1. Purpose
-Automate the conversion of ephemeral, time-stamped thoughts into a permanent, structured knowledge graph.
+It utilizes a "Read-Reason-Write" pattern:
+1. **Read:** Identifies new thoughts in the ~daily/~ folder.
+2. **Reason:** Uses the Probabilistic Engine to extract atomic, evergreen concepts.
+3. **Write:** Commits the distilled notes to the ~notes/~ folder with proper back-links.
 
-** 2. Success Criteria
-- [ ] *Capture:* Identify new headlines in the `daily/` directory that haven't been distilled yet.
-- [ ] *Privacy:* Strictly ignore any node tagged with `@personal`.
-- [ ] *Extraction:* Use neural reasoning to extract atomic concepts from raw logs.
-- [ ] *Formalization:* Create new `.org` files in the `notes/` directory with proper Org-ID and back-links to the source.
+* Implementation
 
-* Phase B: Blueprint (PROTOCOL)
-:PROPERTIES:
-:STATUS: SIGNED
-:END:
-
-** 1. Architectural Intent
-The Scribe reacts to the `:heartbeat` sensor. It maintains a state file (`scribe-state.lisp`) to track the last processed timestamp. It performs a "Read-Reason-Write" loop:
-1. **Read:** Scan `daily/*.org` for nodes updated after the last checkpoint.
-2. **Reason:** Ask the LLM to "Extract atomic notes from this text".
-3. **Write:** Commit the resulting nodes to the `notes/` directory.
-
-** 2. Semantic Interfaces
-- Trigger: `(:sensor :heartbeat)`
-- Action: `(:type :REQUEST :target :system :action :create-note :title "..." :content "..." :source-id "...")`
-
-* Phase D: Build (Implementation)
-
-** Package Context
+** Package Initialization
 #+begin_src lisp
-(in-package :opencortex)
+(in-package :cl-user)
+(defpackage :opencortex.skills.org-skill-scribe
+  (:use :cl :opencortex))
+(in-package :opencortex.skills.org-skill-scribe)
 #+end_src
 
 ** State: Checkpoint Management
-We track the last processed universal time to avoid redundant distillation.
+The Scribe must be efficient. It tracks the last processed timestamp to avoid redundant distillation and LLM token waste.
 
 #+begin_src lisp
 (defvar *scribe-last-checkpoint* 0
   "The universal-time of the last successful distillation run.")
+#+end_src
 
+#+begin_src lisp
 (defun scribe-load-state ()
   "Loads the scribe checkpoint from the state directory."
-  (let ((state-file (uiop:merge-pathnames* "state/scribe-checkpoint.lisp" (asdf:system-source-directory :opencortex))))
+  (let ((state-file (merge-pathnames "system/state/scribe-checkpoint.lisp" 
+                                     (asdf:system-source-directory :opencortex))))
     (if (uiop:file-exists-p state-file)
         (setf *scribe-last-checkpoint* (read-from-string (uiop:read-file-string state-file)))
         (setf *scribe-last-checkpoint* 0))))
+#+end_src
 
+#+begin_src lisp
 (defun scribe-save-state ()
   "Saves the current universal-time as the new checkpoint."
-  (let ((state-file (uiop:merge-pathnames* "state/scribe-checkpoint.lisp" (asdf:system-source-directory :opencortex))))
+  (let ((state-file (merge-pathnames "system/state/scribe-checkpoint.lisp" 
+                                     (asdf:system-source-directory :opencortex))))
     (ensure-directories-exist state-file)
     (with-open-file (out state-file :direction :output :if-exists :supersede)
       (format out "~a" (get-universal-time)))))
 #+end_src
 
-** Filtering: Privacy & Relevance
-The Scribe only cares about non-personal, non-distilled headlines.
+** Filtration: Privacy and Relevance
+To protect user privacy, the Scribe strictly ignores any node tagged with ~@personal~.
 
 #+begin_src lisp
 (defun scribe-get-distillable-nodes ()
-  "Returns a list of org-objects from the daily/ folder that require distillation."
+  "Returns a list of org-objects from memory that require distillation."
   (let ((results nil))
     (maphash (lambda (id obj)
                (declare (ignore id))
@@ -88,14 +73,14 @@ The Scribe only cares about non-personal, non-distilled headlines.
     results))
 #+end_src
 
-** Probabilistic: Extraction Prompt
-The LLM is tasked with identifying atomic concepts within the raw text.
+** Probabilistic Stage: Concept Extraction
+This function generates the specific distillation prompt for the LLM. It focuses on atomicity and structured Lisp output.
 
 #+begin_src lisp
 (defun probabilistic-skill-scribe (context)
-  "Generates the extraction prompt for the Scribe."
-  (let* ((payload (getf context :payload))
-         (nodes (scribe-get-distillable-nodes)))
+  "Generates the extraction prompt for the Scribe distillation task."
+  (declare (ignore context))
+  (let ((nodes (scribe-get-distillable-nodes)))
     (if nodes
         (let ((text-to-process ""))
           (dolist (node nodes)
@@ -111,21 +96,20 @@ Extract ATOMIC EVERGREEN NOTES from this text.
 RULES:
 1. One note per distinct concept.
 2. Output a list of Lisp plists: ((:title \"...\" :content \"...\" :source-id \"...\") ...)
-3. The content should be in Org-mode format.
-4. Keep titles descriptive and snake_case.
+3. Keep titles descriptive and snake_case.
 
 TEXT:
 ~a" text-to-process))
         nil)))
 #+end_src
 
-** Deterministic: Note Committal
-The deterministic gate receives the list of proposed notes and writes them to the filesystem.
+** Deterministic Stage: Knowledge Committal
+The final physical step. It takes the LLM's structured proposal and writes it to the local filesystem.
 
 #+begin_src lisp
 (defun scribe-commit-notes (proposals)
-  "Writes proposed atomic notes to the notes/ directory. Appends if the note exists."
-  (let ((notes-dir (uiop:merge-pathnames* "notes/" (asdf:system-source-directory :opencortex))))
+  "Writes distilled notes to the MemexHardHard Hard drive."
+  (let ((notes-dir (merge-pathnames "notes/" (asdf:system-source-directory :opencortex))))
     (ensure-directories-exist notes-dir)
     (dolist (note proposals)
       (let* ((title (getf note :title))
@@ -133,16 +117,15 @@ The deterministic gate receives the list of proposed notes and writes them to th
              (source-id (getf note :source-id))
              (filename (format nil "~a.org" (string-downcase (cl-ppcre:regex-replace-all " " title "_"))))
              (path (merge-pathnames filename notes-dir)))
-        (if (uiop:file-exists-p path)
-            (with-open-file (out path :direction :output :if-exists :append)
-              (format out "~%~%* Appended insight from ~a~%~a" source-id content))
-            (with-open-file (out path :direction :output :if-exists :supersede)
-              (format out ":PROPERTIES:~%:ID: ~a~%:SOURCE_ID: ~a~%:END:~%#+TITLE: ~a~%~%~a" 
-                      (org-id-new) source-id title content)))
-        (harness-log "SCRIBE: Processed evergreen note ~a" filename)))))
+        (with-open-file (out path :direction :output :if-exists :supersede)
+          (format out ":PROPERTIES:~%:ID: ~a~%:SOURCE_ID: ~a~%:END:~%#+TITLE: ~a~%~%~a" 
+                  (org-id-new) source-id title content))
+        (harness-log "SCRIBE: Distilled evergreen note ~a" filename)))))
+#+end_src
 
+#+begin_src lisp
 (defun verify-skill-scribe (action context)
-  "Executes the note creation and marks source nodes as distilled."
+  "Main deterministic gate for Scribe distillation."
   (declare (ignore context))
   (let ((data (cond ((and (listp action) (eq (getf action :type) :REQUEST))
                      (getf (getf action :payload) :payload))
@@ -150,15 +133,12 @@ The deterministic gate receives the list of proposed notes and writes them to th
                      action)
                     (t nil))))
     (when data
-      (harness-log "SCRIBE: Committing ~a atomic notes..." (length data))
       (scribe-commit-notes data)
       (scribe-save-state)
-      (harness-log "SCRIBE: Distillation complete.")
-      ;; Return a log event to stop the loop
-      (list :type :LOG :payload (list :text "Distillation successful.")))))
+      (list :type :LOG :payload (list :text "SCRIBE: Distillation cycle complete.")))) )
 #+end_src
 
-** Skill Registration
+** Registration
 #+begin_src lisp
 (defskill :skill-scribe
   :priority 50
@@ -166,7 +146,6 @@ The deterministic gate receives the list of proposed notes and writes them to th
              (let* ((payload (getf ctx :payload))
                     (sensor (getf payload :sensor)))
                (and (eq sensor :heartbeat)
-                    ;; Only run once per hour to check if we need to distill
                     (> (- (get-universal-time) *scribe-last-checkpoint*) 3600)
                     (scribe-get-distillable-nodes))))
   :probabilistic #'probabilistic-skill-scribe