7.5 KiB
The Harness Communication (communication.lisp)
- The Harness Communication (communication.lisp)
The Harness Communication (communication.lisp)
Architectural Intent: Secure Inter-Process Communication & Deterministic Framing
Yes, the Harness Communication is fundamentally about Communication. It defines the exact physical and semantic boundaries for how the isolated Lisp environment talks to the outside world.
The org-agent harness operates as a perfectly deterministic, highly secure computational engine. When communicating with external processes or actuators—such as an Emacs client, a web dashboard, or a remote Shell script—the harness cannot rely on unpredictable, "loose" data streams.
Streaming raw Lisp or JSON over a TCP socket is inherently fragile. If a multi-megabyte Org Abstract Syntax Tree (AST) is fragmented by the operating system's network stack during transmission, a standard stream parser might attempt to evaluate an incomplete string, leading to immediate crashes or desynchronization.
To solve this, we implement the Harness Communication, which enforces absolute deterministic boundaries around every message.
1. Physical Boundary: Hex-Length Prefixing
Every message crossing the wire is prefixed with a strict 6-character hexadecimal length string (zero-padded). This creates an unbreakable physical boundary. The harness reads exactly the number of bytes specified by the hex length. It will never under-read (crashing on a partial form) and never over-read (consuming bytes meant for the next message).
2. Actuator-Agnosticism ("Dumb Terminal" Architecture)
The protocol keeps the Lisp harness completely agnostic of its clients. The harness does not care if the client is written in Emacs Lisp, Python, or Rust. Any environment capable of calculating a byte length and opening a TCP socket can interface with the Lisp Machine.
3. Preventing Reader Macro Injection
Common Lisp's read-from-string is extremely powerful but dangerous; it allows "reader macros" (like #.) which execute code during the parsing phase. The Harness Communication mandates that *read-eval* is explicitly bound to nil before any network data is parsed, physically preventing arbitrary code execution.
Message Framing Logic
flowchart LR
subgraph Client
A[Lisp Property List] --> B[Calculate Length]
B --> C[Hex Prefix: 6 Chars]
C --> D[Concatenate: Length + List]
end
D -- TCP Socket --> Harness
subgraph Harness
Harness --> E[Read 6 Hex Chars]
E --> F[Read Exact Byte Count]
F --> G[Parse S-Expression]
endPackage Context
We ensure all protocol logic resides within the isolated harness namespace.
(in-package :org-agent)Actuator Registry
The harness maintains a decoupled registry of target actuators. This allows the system to route messages to Emacs, the Shell, or Web Gateways without hardcoding the routing logic into the protocol itself.
(defvar *actuator-registry* (make-hash-table :test 'equal)
"Global registry mapping target keywords to their physical actuator functions.")
(defun register-actuator (name fn)
"Registers an actuator function. Actuators receive: (ACTION CONTEXT)."
(setf (gethash name *actuator-registry*) fn))Message Framing (frame-message)
The frame-message function prepares an outgoing Lisp string for transmission. It calculates the byte length, converts it into a 6-character padded hex string, and prefixes it. If HARNESS_PROTOCOL_ENFORCE_HMAC is enabled in the environment, it also prepends a cryptographic signature to ensure the message hasn't been tampered with.
(defun frame-message (msg-string)
"Prefixes MSG-STRING with a 6-character hex length.
If security is enabled, prefixes a 64-char HMAC-SHA256 signature."
(let ((len (length msg-string))
(enforce-hmac (uiop:getenv "HARNESS_PROTOCOL_ENFORCE_HMAC")))
(if (and enforce-hmac (string-equal enforce-hmac "true"))
(let ((secret (uiop:getenv "HARNESS_PROTOCOL_HMAC_SECRET")))
(unless secret (error "HARNESS_PROTOCOL_HMAC_SECRET is required when security is enabled."))
(let* ((key (ironclad:ascii-string-to-byte-array secret))
(hmac (ironclad:make-mac :hmac key :sha256))
(payload-bytes (ironclad:ascii-string-to-byte-array msg-string)))
(ironclad:update-mac hmac payload-bytes)
(let ((signature (ironclad:byte-array-to-hex-string (ironclad:produce-mac hmac))))
(format nil "~(~6,'0x~)~a~a" len signature msg-string))))
(format nil "~(~6,'0x~)~a" len msg-string))))Message Parsing (parse-message)
Parsing is the high-security inverse of framing. This function acts as the final perimeter defense. It validates the length, verifies the HMAC integrity, and—most importantly—jails the Lisp reader by disabling *read-eval*.
(defun parse-message (framed-string)
"Extracts and parses the S-expression from a framed string securely."
(when (< (length framed-string) 6)
(error "Framed string too short"))
(let* ((enforce-hmac (uiop:getenv "HARNESS_PROTOCOL_ENFORCE_HMAC"))
(use-hmac (and enforce-hmac (string-equal enforce-hmac "true")))
(prefix-len (if use-hmac 70 6)))
(when (< (length framed-string) prefix-len)
(error "Framed string too short for Harness Communication prefix"))
(let* ((len-str (subseq framed-string 0 6))
(signature (when use-hmac (subseq framed-string 6 70)))
(actual-msg (subseq framed-string prefix-len))
(expected-len (ignore-errors (parse-integer len-str :radix 16))))
(unless expected-len
(error "Invalid hex length prefix: ~a" len-str))
(unless (= expected-len (length actual-msg))
(error "Message length mismatch. Expected ~a, got ~a" expected-len (length actual-msg)))
(when use-hmac
(let ((secret (uiop:getenv "HARNESS_PROTOCOL_HMAC_SECRET")))
(unless secret (error "HARNESS_PROTOCOL_HMAC_SECRET is required when security is enabled."))
(let* ((key (ironclad:ascii-string-to-byte-array secret))
(hmac (ironclad:make-mac :hmac key :sha256))
(payload-bytes (ironclad:ascii-string-to-byte-array actual-msg)))
(ironclad:update-mac hmac payload-bytes)
(let ((expected-signature (ironclad:byte-array-to-hex-string (ironclad:produce-mac hmac))))
(unless (string-equal signature expected-signature)
(error "Harness Communication Integrity Failure: HMAC mismatch"))))))
;; SECURITY: Disable the reader's ability to execute code during parsing
(let ((*read-eval* nil))
(let ((msg (read-from-string actual-msg)))
(validate-harness-protocol-schema msg)
msg)))))Handshaking (make-hello-message)
Every session begins with a standard HELLO handshake, allowing the harness to announce its capabilities and protocol version to the connecting client.
(defun make-hello-message (version)
"Constructs the standard HELLO handshake message."
(list :type :EVENT
:payload (list :action :handshake
:version version
:capabilities '(:auth :swank :org-ast))))