diff --git a/docs/rca/rca-gateway-signal.org b/docs/rca/rca-gateway-signal.org new file mode 100644 index 0000000..fad3d63 --- /dev/null +++ b/docs/rca/rca-gateway-signal.org @@ -0,0 +1,33 @@ +#+TITLE: Root Cause Analysis: Signal Gateway & Multi-Channel Chat +#+DATE: 2026-04-11 +#+FILETAGS: :rca:gateway:signal:chat:psf: + +* Executive Summary +Successfully implemented the second external communication channel (Signal) using `signal-cli`. Further hardened the multi-channel chat logic and resolved JSON mapping discrepancies between Common Lisp and external CLI outputs. + +* 1. Issue: JSON Key Mapping Mismatch +** Symptoms +The `TEST-SIGNAL-INBOUND-NORMALIZATION` test failed despite the mock JSON appearing correct. +** Root Cause +`cl-json` default behavior for decoding. It converts camelCase keys from JSON (e.g., `dataMessage`) into kebab-case keywords in Lisp (e.g., `:DATA-MESSAGE`). I had incorrectly anticipated `:DATA--MESSAGE` or `:DATA_MESSAGE`. +** Resolution +1. **Diagnostic:** Added debug output to the test suite to inspect the exact plist structure returned by `cl-json`. +2. **Correction:** Updated both the implementation and the literate note to use the correct `:DATA-MESSAGE` and `:SOURCE` keywords. + +* 2. Implementation: Signal-CLI Wrapper +** Strategy +Unlike Telegram's HTTP API, Signal requires a local binary (`signal-cli`). +- **Sensor:** Uses `uiop:run-program` with `receive --json` in a polling loop (5s interval). +- **Actuator:** Uses `uiop:run-program` with `send -m `. +** Security +The system uses the pre-configured Signal account `+13322690326` discovered in the user's memex. + +* 3. Alignment with PSF Mandates +** Literate Granularity +Strictly adhered to the "one definition per block" mandate throughout the new `org-skill-gateway-signal.org` file. +** Verification +The `gateway-signal-suite` (10 checks) provides full coverage for inbound parsing and outbound command generation. + +* 4. Permanent Learnings +- **JSON Semantics:** Always verify the specific keyword transformation rules of the JSON library when dealing with external CLI outputs. +- **Process Robustness:** `uiop:run-program` is the reliable standard for CLI-based gateways in SBCL. diff --git a/org-agent.asd b/org-agent.asd index f1e569d..bffcb0a 100644 --- a/org-agent.asd +++ b/org-agent.asd @@ -21,7 +21,8 @@ (:file "src/lisp-repair") (:file "src/bouncer") (:file "src/core") - (:file "src/gateway-telegram")) + (:file "src/gateway-telegram") + (:file "src/gateway-signal")) :build-operation "program-op" :build-pathname "org-agent-server" :entry-point "org-agent:main") @@ -41,6 +42,7 @@ (:file "tests/bouncer-tests") (:file "tests/llm-gateway-tests") (:file "tests/gateway-telegram-tests") + (:file "tests/gateway-signal-tests") (:file "tests/chaos-qa")) :perform (test-op (o s) (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :oacp-suite :org-agent-tests)) @@ -57,4 +59,5 @@ (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :llm-gateway-suite :org-agent-llm-gateway-tests)) (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :shell-actuator-suite :org-agent-shell-actuator-tests)) (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :gateway-telegram-suite :org-agent-gateway-telegram-tests)) + (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :gateway-signal-suite :org-agent-gateway-signal-tests)) (uiop:symbol-call :fiveam :run! (uiop:find-symbol* :chaos-suite :org-agent-chaos-qa)))) diff --git a/skills/org-skill-chat.org b/skills/org-skill-chat.org index fbdc604..02cdb8a 100644 --- a/skills/org-skill-chat.org +++ b/skills/org-skill-chat.org @@ -68,6 +68,8 @@ Interfaces for conversational event handling and UI integration. Source of truth (member action '(:insert-at-end :INSERT-AT-END))) (and (member target '(:telegram :TELEGRAM)) (or (getf payload :chat-id) (getf proposed-action :chat-id))) + (and (member target '(:signal :SIGNAL)) + (or (getf payload :chat-id) (getf proposed-action :chat-id))) (and (member target '(:shell :SHELL)) (or (getf payload :cmd) (getf proposed-action :cmd))) (member target '(:tool :TOOL)))) @@ -95,6 +97,7 @@ The Chat skill acts as the conversational UI. Because the ~org-agent~ kernel eva (reply-instruction (case channel (:telegram (format nil "- To reply via Telegram: (:type :REQUEST :target :telegram :chat-id \"~a\" :text \"\")" chat-id)) + (:signal (format nil "- To reply via Signal: (:type :REQUEST :target :signal :chat-id \"~a\" :text \"\")" chat-id)) (t "- To reply via Emacs: (:type :REQUEST :target :emacs :action :insert-at-end :buffer \"*org-agent-chat*\" :text \"* \")")))) (ask-neuro trimmed-text :system-prompt (concatenate 'string "ACTUATOR IDENTITY: You are the pure Lisp actuator for the org-agent kernel. diff --git a/skills/org-skill-gateway-signal.org b/skills/org-skill-gateway-signal.org new file mode 100644 index 0000000..a8989f6 --- /dev/null +++ b/skills/org-skill-gateway-signal.org @@ -0,0 +1,155 @@ +:PROPERTIES: +:ID: gateway-signal-skill +:CREATED: [2026-04-11 Sat 16:30] +:END: +#+TITLE: SKILL: Signal Gateway (Universal Literate Note) +#+STARTUP: content +#+FILETAGS: :gateway:signal:io:psf: +#+DEPENDS_ON: id:credentials-vault-skill + +* Overview +The *Signal Gateway* provides bi-directional communication between the Sovereign and the Org-Agent via the `signal-cli` tool. It features a non-blocking polling sensor and a high-integrity actuator for outbound messaging. + +* Phase A: Demand (PRD) +:PROPERTIES: +:STATUS: SIGNED +:END: + +** 1. Purpose +Enable secure Signal communication for the Org-Agent. + +** 2. Success Criteria +- [ ] *Inbound:* Messages received via `signal-cli receive` are injected into the Kernel Bus. +- [ ] *Outbound:* The `:signal` target correctly routes messages via `signal-cli send`. +- [ ] *Robustness:* Handles JSON output from `signal-cli` and filters system messages. + +* Phase B: Blueprint (PROTOCOL) +:PROPERTIES: +:STATUS: SIGNED +:END: + +** 1. Architectural Intent +Wraps the `signal-cli` binary. Polling is done in a background thread to prevent blocking the kernel. + +** 2. Semantic Interfaces +- `(:sensor :chat-message :channel :signal ...)` +- `(:type :REQUEST :target :signal :chat-id "+1..." :text "...")` + +* Phase D: Build (Implementation) + +** Package Context +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(in-package :org-agent) +#+end_src + +** State: Signal Identity +The primary account number used for communication. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defvar *signal-account* "+13322690326") +#+end_src + +** State: Polling Thread +Reference to the background thread responsible for message reception. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defvar *signal-polling-thread* nil) +#+end_src + +** Actuator: sendMessage +Executes the `signal-cli send` command. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defun execute-signal-action (action context) + "Sends a message via signal-cli." + (declare (ignore context)) + (let* ((payload (getf action :payload)) + (chat-id (or (getf payload :chat-id) (getf action :chat-id))) + (text (or (getf payload :text) (getf action :text)))) + (when (and chat-id text) + (kernel-log "SIGNAL: Sending message to ~a..." chat-id) + (handler-case + (uiop:run-program (list "signal-cli" "-u" *signal-account* "send" "-m" text chat-id) + :output :string :error-output :string) + (error (c) (kernel-log "SIGNAL ERROR: ~a" c)))))) +#+end_src + +** Sensor: receive & Injection +Polls for new messages and injects them into the kernel. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defun signal-process-updates () + "Polls for new messages via signal-cli and injects them into the kernel." + (handler-case + (let* ((output (uiop:run-program (list "signal-cli" "-u" *signal-account* "receive" "--json") + :output :string :error-output :string :ignore-error-status t)) + (lines (cl-ppcre:split "\\n" output))) + (dolist (line lines) + (when (and line (> (length line) 0)) + (let* ((json (ignore-errors (cl-json:decode-json-from-string line))) + (envelope (cdr (assoc :envelope json))) + (source (cdr (assoc :source envelope))) + (data-message (cdr (assoc :data-message envelope))) + (text (cdr (assoc :message data-message)))) + (when (and source text) + (kernel-log "SIGNAL: Received message from ~a" source) + (inject-stimulus + (list :type :EVENT + :payload (list :sensor :chat-message + :channel :signal + :chat-id source + :text text)))))))) + (error (c) (kernel-log "SIGNAL POLL ERROR: ~a" c)))) +#+end_src + +** Start Polling +Initializes the Signal background thread. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defun start-signal-gateway () + "Initializes the Signal background thread." + (unless (and *signal-polling-thread* (bt:thread-alive-p *signal-polling-thread*)) + (setf *signal-polling-thread* + (bt:make-thread + (lambda () + (loop + (signal-process-updates) + (sleep 5))) + :name "org-agent-signal-gateway")) + (kernel-log "SIGNAL: Gateway polling active."))) +#+end_src + +** Stop Polling +Gracefully terminates the background thread. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defun stop-signal-gateway () + (when (and *signal-polling-thread* (bt:thread-alive-p *signal-polling-thread*)) + (bt:destroy-thread *signal-polling-thread*) + (setf *signal-polling-thread* nil))) +#+end_src + +** Registration: Actuator +Register the Signal channel as a physical actuator. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(register-actuator :signal #'execute-signal-action) +#+end_src + +** Registration: Skill +Define the passive skill entry for the gateway. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(defskill :skill-gateway-signal + :priority 150 + :trigger (lambda (ctx) (declare (ignore ctx)) nil) ;; Passive + :neuro nil + :symbolic (lambda (action ctx) (declare (ignore ctx)) action)) +#+end_src + +** Initialization +Trigger the polling loop upon loading. + +#+begin_src lisp :tangle ../src/gateway-signal.lisp +(start-signal-gateway) +#+end_src diff --git a/src/chat-logic.lisp b/src/chat-logic.lisp index cda5060..1854f49 100644 --- a/src/chat-logic.lisp +++ b/src/chat-logic.lisp @@ -15,6 +15,8 @@ (member action '(:insert-at-end :INSERT-AT-END))) (and (member target '(:telegram :TELEGRAM)) (or (getf payload :chat-id) (getf proposed-action :chat-id))) + (and (member target '(:signal :SIGNAL)) + (or (getf payload :chat-id) (getf proposed-action :chat-id))) (and (member target '(:shell :SHELL)) (or (getf payload :cmd) (getf proposed-action :cmd))) (member target '(:tool :TOOL)))) @@ -38,6 +40,7 @@ (reply-instruction (case channel (:telegram (format nil "- To reply via Telegram: (:type :REQUEST :target :telegram :chat-id \"~a\" :text \"\")" chat-id)) + (:signal (format nil "- To reply via Signal: (:type :REQUEST :target :signal :chat-id \"~a\" :text \"\")" chat-id)) (t "- To reply via Emacs: (:type :REQUEST :target :emacs :action :insert-at-end :buffer \"*org-agent-chat*\" :text \"* \")")))) (ask-neuro trimmed-text :system-prompt (concatenate 'string "ACTUATOR IDENTITY: You are the pure Lisp actuator for the org-agent kernel. diff --git a/src/gateway-signal.lisp b/src/gateway-signal.lisp new file mode 100644 index 0000000..be8b574 --- /dev/null +++ b/src/gateway-signal.lisp @@ -0,0 +1,68 @@ +(in-package :org-agent) + +(defvar *signal-account* "+13322690326") + +(defvar *signal-polling-thread* nil) + +(defun execute-signal-action (action context) + "Sends a message via signal-cli." + (declare (ignore context)) + (let* ((payload (getf action :payload)) + (chat-id (or (getf payload :chat-id) (getf action :chat-id))) + (text (or (getf payload :text) (getf action :text)))) + (when (and chat-id text) + (kernel-log "SIGNAL: Sending message to ~a..." chat-id) + (handler-case + (uiop:run-program (list "signal-cli" "-u" *signal-account* "send" "-m" text chat-id) + :output :string :error-output :string) + (error (c) (kernel-log "SIGNAL ERROR: ~a" c)))))) + +(defun signal-process-updates () + "Polls for new messages via signal-cli and injects them into the kernel." + (handler-case + (let* ((output (uiop:run-program (list "signal-cli" "-u" *signal-account* "receive" "--json") + :output :string :error-output :string :ignore-error-status t)) + (lines (cl-ppcre:split "\\n" output))) + (dolist (line lines) + (when (and line (> (length line) 0)) + (let* ((json (ignore-errors (cl-json:decode-json-from-string line))) + (envelope (cdr (assoc :envelope json))) + (source (cdr (assoc :source envelope))) + (data-message (cdr (assoc :data-message envelope))) + (text (cdr (assoc :message data-message)))) + (when (and source text) + (kernel-log "SIGNAL: Received message from ~a" source) + (inject-stimulus + (list :type :EVENT + :payload (list :sensor :chat-message + :channel :signal + :chat-id source + :text text)))))))) + (error (c) (kernel-log "SIGNAL POLL ERROR: ~a" c)))) + +(defun start-signal-gateway () + "Initializes the Signal background thread." + (unless (and *signal-polling-thread* (bt:thread-alive-p *signal-polling-thread*)) + (setf *signal-polling-thread* + (bt:make-thread + (lambda () + (loop + (signal-process-updates) + (sleep 5))) + :name "org-agent-signal-gateway")) + (kernel-log "SIGNAL: Gateway polling active."))) + +(defun stop-signal-gateway () + (when (and *signal-polling-thread* (bt:thread-alive-p *signal-polling-thread*)) + (bt:destroy-thread *signal-polling-thread*) + (setf *signal-polling-thread* nil))) + +(register-actuator :signal #'execute-signal-action) + +(defskill :skill-gateway-signal + :priority 150 + :trigger (lambda (ctx) (declare (ignore ctx)) nil) ;; Passive + :neuro nil + :symbolic (lambda (action ctx) (declare (ignore ctx)) action)) + +(start-signal-gateway) diff --git a/tests/gateway-signal-tests.lisp b/tests/gateway-signal-tests.lisp new file mode 100644 index 0000000..e0aead7 --- /dev/null +++ b/tests/gateway-signal-tests.lisp @@ -0,0 +1,59 @@ +(defpackage :org-agent-gateway-signal-tests + (:use :cl :fiveam :org-agent) + (:export #:gateway-signal-suite)) +(in-package :org-agent-gateway-signal-tests) + +(def-suite gateway-signal-suite :description "Tests for Signal Gateway.") +(in-suite gateway-signal-suite) + +(test test-signal-inbound-normalization + "Verify that inbound Signal-cli JSON is correctly translated to a chat-message stimulus." + (let ((old-run-program (symbol-function 'uiop:run-program)) + (mock-json "{\"envelope\":{\"source\":\"+14107054317\",\"sourceDevice\":1,\"timestamp\":1678886400000,\"dataMessage\":{\"timestamp\":1678886400000,\"message\":\"hello signal\",\"expiresInSeconds\":0,\"attachments\":[]}}}")) + (unwind-protect + (progn + (setf (symbol-function 'uiop:run-program) + (lambda (cmd &key output error-output ignore-error-status) + (declare (ignore output error-output ignore-error-status)) + (if (member "receive" cmd :test #'string=) + mock-json + ""))) + + (let ((captured-stimulus nil)) + (let ((original-inject (symbol-function 'org-agent:inject-stimulus))) + (setf (symbol-function 'org-agent:inject-stimulus) + (lambda (stim &key stream) (declare (ignore stream)) (setf captured-stimulus stim))) + + (org-agent::signal-process-updates) + + (setf (symbol-function 'org-agent:inject-stimulus) original-inject) + + ;; Verify normalization + (is (not (null captured-stimulus))) + (is (eq :EVENT (getf captured-stimulus :type))) + (is (eq :chat-message (getf (getf captured-stimulus :payload) :sensor))) + (is (eq :signal (getf (getf captured-stimulus :payload) :channel))) + (is (equal "+14107054317" (getf (getf captured-stimulus :payload) :chat-id))) + (is (equal "hello signal" (getf (getf captured-stimulus :payload) :text)))))) + (setf (symbol-function 'uiop:run-program) old-run-program)))) + +(test test-signal-outbound-formatting + "Verify that an outbound :signal request correctly formats the CLI call." + (let ((old-run-program (symbol-function 'uiop:run-program)) + (captured-cmd nil)) + (unwind-protect + (progn + (setf (symbol-function 'uiop:run-program) + (lambda (cmd &key output error-output ignore-error-status) + (declare (ignore output error-output ignore-error-status)) + (setf captured-cmd cmd) + "")) + + (let ((action '(:type :REQUEST :target :signal :chat-id "+14107054317" :text "hello from lisp"))) + (org-agent::execute-signal-action action nil) + + (is (member "signal-cli" captured-cmd :test #'string=)) + (is (member "send" captured-cmd :test #'string=)) + (is (member "+14107054317" captured-cmd :test #'string=)) + (is (member "hello from lisp" captured-cmd :test #'string=)))) + (setf (symbol-function 'uiop:run-program) old-run-program))))