Amr Gharbeia
f940861921
Some checks failed
Deploy-Agent-V15-Stdin / JOB-V15-STDIN (push) Failing after 2s
- Updated all 150+ :tangle headers across harness/ and skills/ to use elisp (expand-file-name) to target INSTALL_DIR dynamically. - Cleaned up environment/ directory depth by moving memory-image.lisp to state/. - Moved test scripts to tests/ and deleted redundant chat scripts.
242 lines
9.3 KiB
Org Mode
242 lines
9.3 KiB
Org Mode
#+TITLE: Manifest (opencortex.asd)
|
|
#+AUTHOR: Amr
|
|
#+FILETAGS: :harness:system:
|
|
#+STARTUP: content
|
|
|
|
* Manifest (opencortex.asd)
|
|
|
|
** Architectural Intent: The Thin Harness Philosophy
|
|
|
|
The ~opencortex.asd~ file is the physical blueprint of the Lisp Machine. It uses **ASDF** (Another System Definition Facility) to orchestrate compilation and loading of all harness modules.
|
|
|
|
The core design principle is *Thin Harness, Fat Skills*:
|
|
|
|
- **Harness** = The minimal, unbreakable core (protocol, signal processing, memory)
|
|
- **Skills** = The intelligence layer (policy, validation, actuation, LLM integration)
|
|
|
|
This separation means:
|
|
- The harness rarely changes (immune system)
|
|
- Skills can be hot-loaded, modified, and swapped without touching the core
|
|
- Bugs in skills don't crash the system
|
|
|
|
** Why ASDF?**
|
|
|
|
ASDF is the de facto standard for Common Lisp project management. It:
|
|
1. Handles dependency resolution and loading order
|
|
2. Compiles files in the right order (preventing "undefined function" errors)
|
|
3. Supports system building for deployment
|
|
4. Integrates with Quicklisp for dependency management
|
|
|
|
* The Build Pipeline
|
|
|
|
#+begin_src mermaid
|
|
flowchart TD
|
|
Org[Literate Org Files] -- Org-Babel Tangle --> Lisp[Source .lisp Files]
|
|
Lisp --> ASDF[ASDF Manifest: opencortex.asd]
|
|
ASDF --> Loader[SBCL Compiler / Loader]
|
|
Loader --> Image[Live Harness Image]
|
|
Image -- Build --> Binary[Standalone Binary]
|
|
|
|
subgraph Skills["Skills Layer (Dynamic)"]
|
|
S1[Policy Skill]
|
|
S2[Bouncer Skill]
|
|
S3[LLM Gateway]
|
|
S4[...other skills]
|
|
end
|
|
|
|
Image --> Skills
|
|
#+end_src
|
|
|
|
* Design Decisions
|
|
|
|
** Strict Serial Loading
|
|
|
|
The harness uses ~:serial t~ in the ASDF definition. This means:
|
|
|
|
1. Files are loaded in order: package → skills → communication → memory → context → perceive → reason → act → loop
|
|
2. ~package.lisp~ is always loaded before any code that uses its symbols
|
|
3. ~skills.lisp~ (defining macros like ~defskill~, ~def-cognitive-tool~) loads before skills
|
|
|
|
This eliminates "macro not found" errors that plague non-linear loading systems.
|
|
|
|
** Why Not Module Dependencies?**
|
|
|
|
Traditional ASDF uses ~:depends-on~ to declare dependencies. We use ~:serial t~ because:
|
|
|
|
1. *Explicit is better than implicit* - the loading order is visible in one place
|
|
2. *Prevents circular dependencies* - skills are loaded after the harness, never before
|
|
3. *Simpler debugging* - when something fails, the loading order is always clear
|
|
|
|
** Isolation of Tests
|
|
|
|
The testing system (~:opencortex/tests~) is separate from the production system (~:opencortex~). This means:
|
|
|
|
- Production deployments don't load FiveAM (saves memory, reduces attack surface)
|
|
- Tests can be run independently: ~(ql:quickload :opencortex/tests)~
|
|
- Test data doesn't pollute the production image
|
|
|
|
* System Definitions
|
|
|
|
** Main Harness System
|
|
|
|
#+begin_src lisp :tangle (expand-file-name "../opencortex.asd" (concat (or (getenv "INSTALL_DIR") ".") "/harness"))
|
|
(defsystem :opencortex
|
|
:name "opencortex"
|
|
:author "Amr"
|
|
:version "0.1.0"
|
|
:license "AGPLv3"
|
|
:description "The Probabilistic-Deterministic Lisp Machine Harness"
|
|
|
|
:depends-on (:usocket ; TCP socket networking
|
|
:bordeaux-threads ; Threading (heartbeat, async sensors)
|
|
:dexador ; HTTP client (LLM APIs)
|
|
:uiop ; Portable I/O, file operations
|
|
:cl-dotenv ; Environment variable loading
|
|
:cl-ppcre ; Regular expressions (parsing)
|
|
:hunchentoot ; HTTP server (optional web interface)
|
|
:ironclad ; Cryptography (Merkle hashing)
|
|
:str ; String utilities
|
|
:cl-json ; JSON parsing/serialization
|
|
:uuid) ; UUID generation for org-mode IDs
|
|
|
|
:serial t ; Load files in order listed below
|
|
|
|
:components ((:file "library/package") ; Package definitions, core vars
|
|
(:file "library/skills") ; Skill engine, cognitive tools
|
|
(:file "library/communication") ; Protocol, framing
|
|
(:file "library/communication-validator") ; Schema validation
|
|
(:file "library/memory") ; Org-object store, snapshots
|
|
(:file "library/context") ; Context assembly, query
|
|
(:file "library/perceive") ; Stage 1: Sensory normalization
|
|
(:file "library/reason") ; Stage 2: Neural + deterministic
|
|
(:file "library/act") ; Stage 3: Actuation
|
|
(:file "library/loop")) ; Main entry, heartbeat
|
|
|
|
:build-operation "program-op"
|
|
:build-pathname "opencortex-server"
|
|
:entry-point "opencortex:main")
|
|
#+end_src
|
|
|
|
** Test System
|
|
|
|
#+begin_src lisp :tangle (expand-file-name "../opencortex.asd" (concat (or (getenv "INSTALL_DIR") ".") "/harness"))
|
|
(defsystem :opencortex/tests
|
|
:depends-on (:opencortex ; The harness we're testing
|
|
:fiveam) ; Testing framework
|
|
|
|
:components ((:file "library/gen/org-skill-emacs-edit")
|
|
(:file "library/gen/org-skill-lisp-utils")
|
|
(:file "tests/communication-tests")
|
|
(:file "tests/pipeline-tests")
|
|
(:file "tests/act-tests")
|
|
(:file "tests/boot-sequence-tests")
|
|
(:file "tests/memory-tests")
|
|
(:file "tests/immune-system-tests")
|
|
(:file "tests/emacs-edit-tests")
|
|
(:file "tests/lisp-utils-tests"))
|
|
|
|
:perform (test-op (o s)
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :communication-protocol-suite :opencortex-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :pipeline-suite :opencortex-pipeline-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :boot-suite :opencortex-boot-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :memory-suite :opencortex-memory-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :immune-suite :opencortex-immune-system-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :emacs-edit-suite :opencortex-emacs-edit-tests))
|
|
(uiop:symbol-call :fiveam :run!
|
|
(uiop:find-symbol* :lisp-utils-suite :opencortex-lisp-utils-tests))))
|
|
#+end_src
|
|
|
|
** TUI Client System
|
|
|
|
#+begin_src lisp :tangle (expand-file-name "../opencortex.asd" (concat (or (getenv "INSTALL_DIR") ".") "/harness"))
|
|
(defsystem :opencortex/tui
|
|
:depends-on (:opencortex ; The daemon we're connecting to
|
|
:croatoan ; Terminal UI library
|
|
:usocket ; Socket communication
|
|
:bordeaux-threads) ; Background listening thread
|
|
|
|
:components ((:file "library/tui-client")))
|
|
#+end_src
|
|
|
|
* The Harness Boundary Contract
|
|
|
|
** Why a Boundary Contract?
|
|
|
|
The harness is the immune system of OpenCortex. If it grows fat (accumulating features, dependencies, complexity), it becomes harder to:
|
|
- Verify for security
|
|
- Debug when things go wrong
|
|
- Maintain across versions
|
|
|
|
The Boundary Contract defines what IS the harness vs. what belongs in skills.
|
|
|
|
** Primary Boundary Files
|
|
|
|
| File | Purpose | Modification |
|
|
|------|---------|--------------|
|
|
| ~harness/*.org~ | Literate source of truth | Only via Org edits + tangle |
|
|
| ~opencortex.asd~ | System manifest | Only via Org edits + tangle |
|
|
| ~library/*.lisp~ | Tangled from .org | NEVER edit directly |
|
|
|
|
** Generated Artifacts (NOT Primary)
|
|
|
|
The ~library/*.lisp~ files are tangles from the ~harness/*.org~ files. They are derivative artifacts. Direct modification violates the Literate Granularity standard.
|
|
|
|
** Protected Paths
|
|
|
|
The Policy skill guards these paths by default:
|
|
|
|
#+begin_src lisp
|
|
(defvar *modularity-protected-paths*
|
|
'("harness/"
|
|
"opencortex.asd"
|
|
"library/package.lisp"
|
|
"library/communication.lisp"
|
|
"library/memory.lisp"
|
|
"library/context.lisp"
|
|
"library/perceive.lisp"
|
|
"library/reason.lisp"
|
|
"library/act.lisp"
|
|
"library/loop.lisp"))
|
|
#+end_src
|
|
|
|
Any agent action proposing to modify these files must include a ~:modularity-justification~ field explaining why the change cannot be implemented as a skill.
|
|
|
|
** Enforcement Chain
|
|
|
|
1. *Policy Skill* (priority 500) - Checks for missing justifications
|
|
2. *Bouncer Skill* (priority 100) - Intercepts unauthorized modifications
|
|
3. *Git Hooks* (optional) - Prevents direct .lisp commits
|
|
|
|
* Quick Reference
|
|
|
|
** Building the System
|
|
|
|
#+begin_src bash
|
|
# Development: Load source
|
|
(ql:quickload :opencortex)
|
|
|
|
# Build standalone binary
|
|
(asdf:make :opencortex)
|
|
|
|
# Run tests
|
|
(ql:quickload :opencortex/tests)
|
|
(asdf:test-system :opencortex/tests)
|
|
#+end_src
|
|
|
|
** Loading Order
|
|
|
|
1. ~library/package.lisp~ - Creates ~:opencortex~ package
|
|
2. ~library/skills.lisp~ - Defines ~defskill~, ~def-cognitive-tool~ macros
|
|
3. ~library/communication.lisp~ - Protocol, framing, validation
|
|
4. ~library/memory.lisp~ - Org-object, Merkle tree, snapshots
|
|
5. ~library/context.lisp~ - Context assembly functions
|
|
6. ~library/perceive.lisp~ - Stage 1: Perceive gate
|
|
7. ~library/reason.lisp~ - Stage 2: Reason (think + verify)
|
|
8. ~library/act.lisp~ - Stage 3: Act (dispatch + execute)
|
|
9. ~library/loop.lisp~ - Main entry point, heartbeat |