14 KiB
The Neurosymbolic Bridge (neuro.lisp & symbolic.lisp)
- The Neurosymbolic Bridge (neuro.lisp & symbolic.lisp)
- Probabilistic Engine (neuro.lisp)
- Deterministic Engine (symbolic.lisp)
The Neurosymbolic Bridge (neuro.lisp & symbolic.lisp)
The Neurosymbolic Loop
In our loop, the Probabilistic Engine never speaks to the world directly. It only proposes "thoughts" to the Deterministic Engine. the Deterministic Engine, the Lisp harness, evaluates these thoughts against a chain of symbolic safety gates (Skills) before any action is actually dispatched to an actuator (Emacs, Shell, etc.).
flowchart TD
Stimulus[External Stimulus/Signal] --> Perceive[Perceive: Skill Trigger]
Perceive --> Probabilistic[Probabilistic Engine: LLM]
Probabilistic --> Proposal[Lisp Action Proposal]
Proposal --> Deterministic[Deterministic Engine: Lisp Gates]
Deterministic --> Gate1[Safety Gate: Skill A]
Gate1 --> Gate2[Safety Gate: Skill B]
Gate2 --> Verified[Verified Action]
Verified --> Dispatch[Dispatch: Actuator]
style Probabilistic fill:#f9f,stroke:#333,stroke-width:2px
style Deterministic fill:#bbf,stroke:#333,stroke-width:2pxSovereign Decoupling (The Thin Harness)
The harness files neuro.lisp and symbolic.lisp are intentionally "Thin Harnesses." They do not know how to talk to Google Gemini or Anthropic Claude; they do not know what a "Bouncer" or a "Token Accountant" is. Instead, they provide the protocol for these behaviors.
By moving the "Fat" logic (vendor APIs, security rules) into Skills, we achieve total sovereign decoupling. You can swap your LLM provider or your security policy without ever touching the harness.
Probabilistic Engine (neuro.lisp)
The Probabilistic engine handles the interface with LLM providers, providing a unified probabilistic space regardless of the underlying model.
Package Context
(in-package :org-agent)Probabilistic Backends Registry
The harness maintains a neutral registry of backends. Skills (like the LLM Gateway) register themselves here to provide actual neural reasoning capabilities.
(defvar *neuro-backends* (make-hash-table :test 'equal))Provider Cascade
The ordered list of backends to attempt for neural reasoning. This list is nil by default and must be populated by skills (e.g., the LLM Gateway or Token Accountant) during the harness boot sequence.
(defvar *provider-cascade* nil)Register Probabilistic Backend
A simple mapping from a keyword identifier to a backend implementation function.
(defun register-neuro-backend (name fn) (setf (gethash name *neuro-backends*) fn))Model Selector Function
A hook for dynamic model selection. A skill might look at the current context and decide to use a "cheap" model for summaries and an "expensive" model for coding tasks.
(defvar *model-selector-fn* nil "A function called with (provider context) to return a model ID.")Probabilistic Dispatch (ask-neuro)
This is the primary entrance to the Probabilistic engine. It implements two modes of operation:
- Sequential Cascade: Attempt backends one by one until success.
- Parallel Consensus: Query multiple backends simultaneously to resolve hallucinations or select the best "thought."
sequenceDiagram
participant Harness
participant ProviderA as LLM 1
participant ProviderB as LLM 2
Harness->>ProviderA: Parallel Query
Harness->>ProviderB: Parallel Query
ProviderA-->>Harness: Suggestion A
ProviderB-->>Harness: Suggestion B
Harness->>Harness: Resolve Consensus(defvar *consensus-enabled-p* nil "If T, ask-neuro queries all backends in parallel.")
(defun ask-neuro (prompt &key (system-prompt "You are the Probabilistic engine of a Neurosymbolic Lisp Machine.") (cascade nil) (context nil))
"Dispatches a neural request through the provider cascade or parallel consensus."
(let ((backends (cond
((and cascade (listp cascade)) cascade)
((functionp cascade) (funcall cascade context))
(t *provider-cascade*))))
(if *consensus-enabled-p*
;; PARALLEL CONSENSUS MODE
(let ((results nil)
(threads nil)
(lock (bt:make-lock)))
(dolist (backend backends)
(let ((backend-fn (gethash backend *neuro-backends*)))
(when backend-fn
(push (bt:make-thread
(lambda ()
(harness-log "PROBABILISTIC [Consensus]: Querying backend ~a..." backend)
(let* ((model (when *model-selector-fn* (funcall *model-selector-fn* backend context)))
(result (ignore-errors
(if model
(funcall backend-fn prompt system-prompt :model model)
(funcall backend-fn prompt system-prompt)))))
(bt:with-lock-held (lock)
(push result results)))))
threads))))
;; Wait for all threads with a timeout (e.g., 30s)
(let ((start-time (get-universal-time)))
(loop while (and (< (length results) (length threads))
(< (- (get-universal-time) start-time) 30))
do (sleep 0.1)))
;; Return the list of raw results (filtering out nils or errors)
(let ((valid-results (remove-if-not #'stringp results)))
(if valid-results
(format nil "~{~a~^|CONSENSUS-SEP|~}" valid-results)
"(:type :LOG :payload (:text \"Neural Consensus Failure\"))")))
;; SEQUENTIAL CASCADE MODE
(or (dolist (backend backends)
(let ((backend-fn (gethash backend *neuro-backends*)))
(when backend-fn
(harness-log "PROBABILISTIC: Attempting backend ~a..." backend)
(let* ((model (when *model-selector-fn* (funcall *model-selector-fn* backend context)))
(result (if model
(funcall backend-fn prompt system-prompt :model model)
(funcall backend-fn prompt system-prompt))))
(unless (or (null result)
(and (stringp result) (search ":LOG" result) (or (search "Failure" result) (search "missing" result))))
(return result))))))
"(:type :LOG :payload (:text \"Neural Cascade Failure\"))"))))Probabilistic Reasoning (think)
The think function is where the "Neuro" meets the "Symbolic." It gathers the global awareness context (Peripheral Vision), the tool definitions (The Tool Belt), and any skill-specific triggers to form the final prompt.
Crucially, it mandates that the output be a Common Lisp property list, forcing the LLM to "think in Lisp."
(defun think (context)
"Invokes the neural Probabilistic engine to propose a Lisp action based on context."
(let ((active-skill (find-triggered-skill context))
(tool-belt (generate-tool-belt-prompt))
(global-context (context-assemble-global-awareness)))
(if active-skill
(progn
(harness-log "PROBABILISTIC: Engaging skill '~a'~%" (skill-name active-skill))
(let* ((prompt-generator (skill-neuro-prompt active-skill))
(raw-prompt (when prompt-generator (funcall prompt-generator context)))
(full-system-prompt (concatenate 'string
"ACTUATOR IDENTITY: You are the pure Lisp actuator for the org-agent kernel.
MANDATE: Output EXACTLY ONE Common Lisp property list starting with (:type :REQUEST).
ZERO CONVERSATION: Do not explain. Do not say 'Okay'. Do not use markdown blocks.
STRICT RULE: Do not output multiple lists. Do not chain multiple requests.
DO NOT embed tool calls inside text strings.
"
global-context
"
"
tool-belt
"
IMPORTANT: To reply to the user, you MUST use:
(:type :REQUEST :target :emacs :action :insert-at-end :buffer \"*org-agent-chat*\" :text \"* <Response Text>\")
To call a tool, you MUST use:
(:type :REQUEST :target :tool :action :call :tool \"<name>\" :args (:arg1 \"val\"))
")))
(if (and raw-prompt (> (length raw-prompt) 1))
(let* ((thought (ask-neuro raw-prompt :system-prompt full-system-prompt :context context))
(raw-thoughts (cl-ppcre:split (cl-ppcre:quote-meta-chars "|CONSENSUS-SEP|") thought))
(suggestions nil))
(dolist (raw-thought raw-thoughts)
(harness-log "PROBABILISTIC RAW: ~a~%" raw-thought)
(let* ((cleaned-thought
(let ((match (cl-ppcre:scan-to-strings "(?s)```(?:lisp)?\\n?(.*?)\\n?```" raw-thought)))
(if match
(let ((regs (nth-value 1 (cl-ppcre:scan-to-strings "(?s)```(?:lisp)?\\n?(.*?)\\n?```" raw-thought))))
(if (and regs (> (length regs) 0)) (elt regs 0) raw-thought))
(string-trim '(#\Space #\Newline #\Tab) raw-thought))))
(suggestion (handler-case (read-from-string cleaned-thought)
(error (c)
;; EMIT ASYNCHRONOUS REPAIR STIMULUS
(list :type :EVENT :payload
(list :sensor :syntax-error
:code cleaned-thought
:error (format nil "~a" c)))))))
(harness-log "PROBABILISTIC Suggestion: ~a~%" cleaned-thought)
(when (and suggestion (listp suggestion))
(push suggestion suggestions))))
(if (and *consensus-enabled-p* suggestions)
(nreverse suggestions)
(first (nreverse suggestions))))
'(:type :LOG :payload (:text "Skill triggered (Deterministic only)")))))
nil)))Prompt Meta-Cognition (distill-prompt)
Even the Probabilistic engine can benefit from introspection. This function allows the agent to observe its own prompts and successful results to distill them into reusable templates.
(defun distill-prompt (full-prompt successful-output)
(let ((system-instr "You are a Meta-Cognitive Prompt Architect. DISTILL into template."))
(ask-neuro (format nil "PROMPT: ~a~%RESULT: ~a" full-prompt successful-output) :system-prompt system-instr)))Deterministic Engine (symbolic.lisp)
The Deterministic engine is the deterministic gatekeeper that ensures all proposed actions—whether from the user or from the neural engine—are safe and logically valid.
As a "Thin Harness," the Deterministic engine does not contain specific security rules or task integrity checks. Instead, it provides a priority-based dispatcher that iterates through all loaded skills to validate or transform proposed actions.
Package Context
(in-package :org-agent)Validation Gate (decide)
This is the Supervisor. It intercepts every action proposal and runs it through the symbolic gates of all registered skills, sorted by priority.
This sequential chain allows for multi-layered defense:
- Low Priority Gates: Suggest refinements or logging.
- Medium Priority Gates: Transform actions (e.g., adding project IDs).
- High Priority Gates: Absolute security blocks (e.g., the Bouncer blocking shell access).
flowchart LR
Proposal[Proposal] --> SkillA[Skill A: Priority 10]
SkillA --> SkillB[Skill B: Priority 50]
SkillB --> SkillC[Skill C: Priority 100]
SkillC --> Verified[Verified Action]
style SkillC fill:#fbb,stroke:#333(defun decide (proposed-action context)
"The Deterministic Safety Gate: iterates through all skill symbolic-gates sorted by priority."
(let ((current-action proposed-action)
(skills nil))
;; 1. Collect all skills with symbolic gates
(maphash (lambda (name skill)
(declare (ignore name))
(when (skill-symbolic-fn skill)
(push skill skills)))
*skills-registry*)
;; 2. Sort skills by priority (highest first)
(setf skills (sort skills #'> :key #'skill-priority))
;; 3. Execute symbolic gates sequentially
(dolist (skill skills)
(let ((gate (skill-symbolic-fn skill)))
(setf current-action (funcall gate current-action context))
;; If any gate returns a LOG or EVENT (blocking/intercepting), stop and return it.
(when (and (listp current-action)
(member (getf current-action :type) '(:LOG :EVENT :log :event)))
(harness-log "DETERMINISTIC: Intercepted by skill '~a'~%" (skill-name skill))
(return-from decide current-action))))
current-action))Store Filtering (list-objects-with-attribute)
A symbolic helper function to find nodes with specific attributes. This is used by skills to perform complex semantic lookups in the Object Store.
(defun list-objects-with-attribute (attr-key attr-val)
"Filters the Object Store for nodes having a specific attribute value."
(let ((results nil))
(maphash (lambda (id obj)
(declare (ignore id))
(when (equal (getf (org-object-attributes obj) attr-key) attr-val)
(push obj results)))
*object-store*)
results))