passepartout/harness/manifest.org

Manifest (opencortex.asd)

• Manifest (opencortex.asd)
  • Architectural Intent: The Thin Harness Philosophy
  • Why ASDF?**
• The Build Pipeline
• Design Decisions
  • Strict Serial Loading
  • Why Not Module Dependencies?**
  • Isolation of Tests
• System Definitions
  • Main Harness System
  • Test System
  • TUI Client System
• The Harness Boundary Contract
  • Why a Boundary Contract?
  • Primary Boundary Files
  • Generated Artifacts (NOT Primary)
  • Protected Paths
  • Enforcement Chain
• Quick Reference
  • Building the System
  • Loading Order

Manifest (opencortex.asd)

Architectural Intent: The Thin Harness Philosophy

The opencortex.asd file is the physical blueprint of the Lisp Machine. It uses ASDF (Another System Definition Facility) to orchestrate compilation and loading of all harness modules.

The core design principle is Thin Harness, Fat Skills:

• Harness = The minimal, unbreakable core (protocol, signal processing, memory)
• Skills = The intelligence layer (policy, validation, actuation, LLM integration)

This separation means:

• The harness rarely changes (immune system)
• Skills can be hot-loaded, modified, and swapped without touching the core
• Bugs in skills don't crash the system

Why ASDF?**

ASDF is the de facto standard for Common Lisp project management. It:

1. Handles dependency resolution and loading order
2. Compiles files in the right order (preventing "undefined function" errors)
3. Supports system building for deployment
4. Integrates with Quicklisp for dependency management

The Build Pipeline

flowchart TD
    Org[Literate Org Files] -- Org-Babel Tangle --> Lisp[Source .lisp Files]
    Lisp --> ASDF[ASDF Manifest: opencortex.asd]
    ASDF --> Loader[SBCL Compiler / Loader]
    Loader --> Image[Live Harness Image]
    Image -- Build --> Binary[Standalone Binary]

    subgraph Skills["Skills Layer (Dynamic)"]
        S1[Policy Skill]
        S2[Bouncer Skill]
        S3[LLM Gateway]
        S4[...other skills]
    end

    Image --> Skills

Design Decisions

Strict Serial Loading

The harness uses :serial t in the ASDF definition. This means:

1. Files are loaded in order: package → skills → communication → memory → context → perceive → reason → act → loop
2. package.lisp is always loaded before any code that uses its symbols
3. skills.lisp (defining macros like defskill, def-cognitive-tool) loads before skills

This eliminates "macro not found" errors that plague non-linear loading systems.

Why Not Module Dependencies?**

Traditional ASDF uses :depends-on to declare dependencies. We use :serial t because:

1. Explicit is better than implicit - the loading order is visible in one place
2. Prevents circular dependencies - skills are loaded after the harness, never before
3. Simpler debugging - when something fails, the loading order is always clear

Isolation of Tests

The testing system (:opencortex/tests) is separate from the production system (:opencortex). This means:

• Production deployments don't load FiveAM (saves memory, reduces attack surface)
• Tests can be run independently: (ql:quickload :opencortex/tests)
• Test data doesn't pollute the production image

System Definitions

Main Harness System

(defsystem :opencortex
  :name "opencortex"
  :author "Amr"
  :version "0.1.0"
  :license "AGPLv3"
  :description "The Probabilistic-Deterministic Lisp Machine Harness"

  :depends-on (:usocket              ; TCP socket networking
               :bordeaux-threads     ; Threading (heartbeat, async sensors)
               :dexador              ; HTTP client (LLM APIs)
               :uiop                 ; Portable I/O, file operations
               :cl-dotenv            ; Environment variable loading
               :cl-ppcre             ; Regular expressions (parsing)
               :hunchentoot          ; HTTP server (optional web interface)
               :ironclad             ; Cryptography (Merkle hashing)
               :str                  ; String utilities
               :cl-json              ; JSON parsing/serialization
               :uuid)                ; UUID generation for org-mode IDs

  :serial t                          ; Load files in order listed below

   :components ((:file "library/package")           ; Package definitions, core vars
                (:file "library/skills")            ; Skill engine, cognitive tools
                (:file "library/communication")     ; Protocol, framing
                (:file "library/communication-validator") ; Schema validation
                (:file "library/memory")            ; Org-object store, snapshots
                (:file "library/context")           ; Context assembly, query
                (:file "library/perceive")         ; Stage 1: Sensory normalization
                (:file "library/reason")           ; Stage 2: Neural + deterministic
                (:file "library/act")               ; Stage 3: Actuation
                (:file "library/loop"))             ; Main entry, heartbeat

  :build-operation "program-op"
  :build-pathname "opencortex-server"
  :entry-point "opencortex:main")

Test System

(defsystem :opencortex/tests
  :depends-on (:opencortex           ; The harness we're testing
               :fiveam)              ; Testing framework

   :components ((:file "library/gen/org-skill-emacs-edit")
                (:file "library/gen/org-skill-lisp-utils")
                (:file "tests/communication-tests")
                (:file "tests/pipeline-tests")
                (:file "tests/act-tests")
                (:file "tests/boot-sequence-tests")
                (:file "tests/memory-tests")
                (:file "tests/immune-system-tests")
                (:file "tests/emacs-edit-tests")
                (:file "tests/lisp-utils-tests"))

   :perform (test-op (o s)
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :communication-protocol-suite :opencortex-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :pipeline-suite :opencortex-pipeline-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :boot-suite :opencortex-boot-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :memory-suite :opencortex-memory-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :immune-suite :opencortex-immune-system-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :emacs-edit-suite :opencortex-emacs-edit-tests))
              (uiop:symbol-call :fiveam :run!
                (uiop:find-symbol* :lisp-utils-suite :opencortex-lisp-utils-tests))))

TUI Client System

(defsystem :opencortex/tui
  :depends-on (:opencortex           ; The daemon we're connecting to
               :croatoan            ; Terminal UI library
               :usocket              ; Socket communication
               :bordeaux-threads)    ; Background listening thread

  :components ((:file "library/tui-client")))

The Harness Boundary Contract

Why a Boundary Contract?

The harness is the immune system of OpenCortex. If it grows fat (accumulating features, dependencies, complexity), it becomes harder to:

• Verify for security
• Debug when things go wrong
• Maintain across versions

The Boundary Contract defines what IS the harness vs. what belongs in skills.

Primary Boundary Files

File	Purpose	Modification
harness/*.org	Literate source of truth	Only via Org edits + tangle
opencortex.asd	System manifest	Only via Org edits + tangle
library/*.lisp	Tangled from .org	NEVER edit directly

Generated Artifacts (NOT Primary)

The library/*.lisp files are tangles from the harness/*.org files. They are derivative artifacts. Direct modification violates the Literate Granularity standard.

Protected Paths

The Policy skill guards these paths by default:

(defvar *modularity-protected-paths*
  '("harness/"
    "opencortex.asd"
    "library/package.lisp"
    "library/communication.lisp"
    "library/memory.lisp"
    "library/context.lisp"
    "library/perceive.lisp"
    "library/reason.lisp"
    "library/act.lisp"
    "library/loop.lisp"))

Any agent action proposing to modify these files must include a :modularity-justification field explaining why the change cannot be implemented as a skill.

Enforcement Chain

1. Policy Skill (priority 500) - Checks for missing justifications
2. Bouncer Skill (priority 100) - Intercepts unauthorized modifications
3. Git Hooks (optional) - Prevents direct .lisp commits

Quick Reference

Building the System

# Development: Load source
(ql:quickload :opencortex)

# Build standalone binary
(asdf:make :opencortex)

# Run tests
(ql:quickload :opencortex/tests)
(asdf:test-system :opencortex/tests)

Loading Order

1. library/package.lisp - Creates :opencortex package
2. library/skills.lisp - Defines defskill, def-cognitive-tool macros
3. library/communication.lisp - Protocol, framing, validation
4. library/memory.lisp - Org-object, Merkle tree, snapshots
5. library/context.lisp - Context assembly functions
6. library/perceive.lisp - Stage 1: Perceive gate
7. library/reason.lisp - Stage 2: Reason (think + verify)
8. library/act.lisp - Stage 3: Act (dispatch + execute)
9. library/loop.lisp - Main entry point, heartbeat