Amr Gharbeia
dd3873cd5e
9.1 KiB
System Interface (package.lisp)
- System Interface (package.lisp)
- System State Management
- Support Functions
- Cognitive Tooling System
- Logging Implementation
System Interface (package.lisp)
The package.lisp file defines the public API of the opencortex harness. It serves as the primary membrane between the deterministic core modules and the dynamic world of skills and actuators.
Architectural Intent: The Package Membrane
By strictly defining the public interface, we ensure that skills remain decoupled from the harness implementation details. This allows for autonomous replacement of any component (e.g., swapping the Memory or the Probabilistic Engine) without breaking existing skills.
flowchart TD
External[Actuators / Clients] -- communication protocol --> Package[Package Membrane: API]
Skills[Dynamic Skills] -- API Calls --> Package
Package --> Internal[Harness Internal Modules]
style Package fill:#f9f,stroke:#333,stroke-width:4pxPublic API Export
(defpackage :opencortex
(:use :cl)
(:export
;; --- Communication Protocol ---
#:frame-message
#:read-framed-message
#:PROTO-GET
#:LIST-OBJECTS-WITH-ATTRIBUTE
#:COSINE-SIMILARITY
#:VAULT-MASK-STRING
#:*VAULT-MEMORY*
#:parse-message
#:make-hello-message
#:validate-communication-protocol-schema
;; --- Daemon Lifecycle ---
#:start-daemon
#:stop-daemon
#:harness-log
#:main
;; --- Memory (CLOSOS) ---
#:ingest-ast
#:lookup-object
#:list-objects-by-type
#:org-id-new
#:*memory*
#:*history-store*
#:org-object
#:make-org-object
#:org-object-id
#:org-object-type
#:org-object-attributes
#:org-object-parent-id
#:org-object-children
#:org-object-version
#:org-object-last-sync
#:org-object-vector
#:org-object-content
#:org-object-hash
#:snapshot-memory
#:rollback-memory
;; --- Context API (Peripheral Vision) ---
#:context-query-store
#:context-get-active-projects
#:context-get-recent-completed-tasks
#:context-list-all-skills
#:context-get-skill-source
#:context-get-system-logs
#:context-resolve-path
#:context-get-skill-telemetry
#:harness-track-telemetry
#:context-assemble-global-awareness
;; --- Reactive Signal Pipeline ---
#:process-signal
#:perceive-gate
#:probabilistic-gate
#:consensus-gate
#:act-gate
#:reason-gate
#:perceive-gate
#:dispatch-gate
#:inject-stimulus
#:initialize-actuators
#:dispatch-action
#:register-actuator
;; --- Skill Engine ---
#:load-skill-from-org
#:initialize-all-skills
#:load-skill-with-timeout
#:topological-sort-skills
#:validate-lisp-syntax
#:defskill
#:*skills-registry*
#:skill
#:skill-name
#:skill-priority
#:skill-dependencies
#:skill-trigger-fn
#:skill-probabilistic-prompt
#:skill-deterministic-fn
;; --- Tool Registry ---
#:def-cognitive-tool
#:*cognitive-tools*
#:cognitive-tool
#:cognitive-tool-name
#:cognitive-tool-description
#:cognitive-tool-parameters
#:cognitive-tool-guard
#:cognitive-tool-body
;; --- Emacs Client Registry ---
#:*emacs-clients*
#:*clients-lock*
#:register-emacs-client
#:unregister-emacs-client
;; --- Probabilistic Engine ---
#:ask-probabilistic
#:register-probabilistic-backend
#:distill-prompt
#:*provider-cascade*
;; --- Security Vault ---
#:vault-get-secret
#:vault-set-secret
;; --- Deterministic Logic ---
#:list-objects-with-attribute
#:deterministic-verify
;; --- AST Helpers ---
#:find-headline-missing-id))Package Implementation Initialization
Ensuring the compiler enters the correct namespace for all subsequent definitions.
(in-package :opencortex)System State Management
The package layer manages the core data structures that represent the live state of the harness.
Harness Logging State
OpenCortex maintains a thread-safe circular log buffer. This is critical for two reasons:
- Neural Introspection: The probabilistic engine can read the recent system logs to understand why an action failed.
- Real-time Debugging: Clients can subscribe to a live log stream without needing to read the physical log file.
(defvar *system-logs* nil
"Thread-safe list of the most recent system messages.")(defvar *logs-lock* (bt:make-lock "harness-logs-lock")
"Protects the circular log buffer from race conditions during concurrent skill execution.")(defvar *max-log-history* 100
"The maximum number of entries to preserve in the in-memory log buffer.")Skills Registry
All Literate Skills, once compiled, are registered here. This allows for topological sorting and priority-based execution.
(defvar *skills-registry* (make-hash-table :test 'equal)
"Global registry of all loaded skills, keyed by their unique identifier.")Skill Telemetry State
To ensure the system remains performant and reliable, the harness tracks execution metrics for every skill.
(defvar *skill-telemetry* (make-hash-table :test 'equal)
"Stores execution duration and failure counts for every registered skill.")(defvar *telemetry-lock* (bt:make-lock "harness-telemetry-lock")
"Protects the telemetry store from concurrent updates.")Support Functions
Protocol Property Access (proto-get)
Lisp keywords can be inconsistent between capitalized and lowercase versions depending on the client (e.g., Emacs vs. Python socket). proto-get provides a robust abstraction to ensure the system correctly extracts values regardless of keyword casing.
(defun proto-get (plist key)
"Robustly retrieves a value from a plist, checking both uppercase and lowercase keyword versions."
(let* ((s (string key))
(up (intern (string-upcase s) :keyword))
(dn (intern (string-downcase s) :keyword)))
(or (getf plist up) (getf plist dn))))Telemetry Tracking
The harness-track-telemetry function provides the hook for the metabolic loop to report performance data.
(defun harness-track-telemetry (skill-name duration status)
"Updates performance metrics for a specific skill. Status should be :success or :rejected."
(when skill-name
(bt:with-lock-held (*telemetry-lock*)
(let ((entry (or (gethash skill-name *skill-telemetry*) (list :executions 0 :total-time 0 :failures 0))))
(incf (getf entry :executions))
(incf (getf entry :total-time) duration)
(when (eq status :rejected) (incf (getf entry :failures)))
(setf (gethash skill-name *skill-telemetry*) entry)))))Cognitive Tooling System
The Tool Registry is the agent's physical interface. It separates the proposal of an action from its execution.
Tool Structure
(defvar *cognitive-tools* (make-hash-table :test 'equal)
"The active set of physical capabilities available to the agent.")(defstruct cognitive-tool
"Represents a physical or virtual capability with explicit documentation and security guards."
name
description
parameters
guard
body)Tool Registration Macro (def-cognitive-tool)
We use a macro to ensure that tools are consistently registered and accessible to the LLM's "tool-belt" prompt generator.
(defmacro def-cognitive-tool (name description parameters &key guard body)
"Registers a new cognitive tool.
NAME: Keyword identifier.
DESCRIPTION: Human-readable intent (used in LLM prompts).
PARAMETERS: List of property lists defining arguments.
GUARD: (context -> boolean) function to prevent unsafe calls.
BODY: The actual Lisp execution logic."
`(setf (gethash (string-downcase (string ',name)) *cognitive-tools*)
(make-cognitive-tool :name (string-downcase (string ',name))
:description ,description
:parameters ',parameters
:guard ,guard
:body ,body)))Logging Implementation
Centralized Logging (harness-log)
The primary mechanism for system transparency. It ensures all activity is both visible to the user and recorded for neural reasoning.
(defun harness-log (msg &rest args)
"Centralized logging for the harness. Writes to STDOUT and the thread-safe circular buffer."
(let ((formatted-msg (apply #'format nil msg args)))
(bt:with-lock-held (*logs-lock*)
(push formatted-msg *system-logs*)
(when (> (length *system-logs*) *max-log-history*)
(setq *system-logs* (subseq *system-logs* 0 *max-log-history*))))
(format t "~a~%" formatted-msg)
(finish-output)))