Initial commit: extracted from memex

This commit is contained in:
2026-03-27 15:41:57 -04:00
commit 9acc7713e0
37 changed files with 2308 additions and 0 deletions

31
.env Normal file
View File

@@ -0,0 +1,31 @@
# org-agent: Neural Engine Configuration
LLM_API_KEY="" # Add your Gemini API key here
LLM_ENDPOINT="https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent"
# System 2: Symbolic Constraints
SAFETY_BLOCK_SHELL=true
GTD_ENFORCE_INTEGRITY=true
# OACP Daemon Configuration
ORG_AGENT_DAEMON_PORT=9105
ORG_AGENT_WEB_PORT=8080
DAEMON_HOST="0.0.0.0" # Changed to 0.0.0.0 for Docker accessibility
HEARTBEAT_INTERVAL=60
# Memex Integration
MEMEX_DIR="/home/amr/memex"
ZETTELKASTEN_DIR="/app/2_notes"
SKILLS_DIR="/app/2_notes"
# PARA Structure (Consolidated)
INBOX_DIR="/app/0_inbox"
DAILY_DIR="/app/1_daily"
PROJECTS_DIR="/app/5_projects"
AREAS_DIR="/app/6_areas"
RESOURCES_DIR="/app/7_resources"
ARCHIVES_DIR="/app/8_archives"
SYSTEM_DIR="/app/9_system"
MEMEX_USER="Amr"
MEMEX_ASSISTANT="Agent"
RECIPIENT_ID="+14107054317"

43
.env.example Normal file
View File

@@ -0,0 +1,43 @@
# org-agent: Neural Engine Configuration
# Get your API key from the respective provider (e.g., Google AI Studio for Gemini)
LLM_API_KEY="your_api_key_here"
LLM_ENDPOINT="https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent"
# OpenRouter Configuration (Optional)
OPENROUTER_API_KEY="your_openrouter_key_here"
# Dynamic Model Configuration (Provider Suffixes)
# These can also be set via Org-mode properties in your Memex
# :LLM_MODEL_OPENAI: gpt-4o
# :LLM_MODEL_ANTHROPIC: claude-3-5-sonnet-20240620
# :LLM_MODEL_OPENROUTER: google/gemini-pro-1.5
# System 2: Symbolic Constraints
SAFETY_BLOCK_SHELL=true
GTD_ENFORCE_INTEGRITY=true
# OACP Daemon Configuration
ORG_AGENT_DAEMON_PORT=9105
ORG_AGENT_WEB_PORT=8080
DAEMON_HOST="0.0.0.0"
HEARTBEAT_INTERVAL=60
# Memex Integration
# Inside Docker, /app/ is the root for consolidated notes
MEMEX_DIR="/memex"
ZETTELKASTEN_DIR="/memex/notes"
SKILLS_DIR="/memex/notes"
# PARA Structure (Consolidated)
INBOX_DIR="/memex/inbox"
DAILY_DIR="/memex/daily"
PROJECTS_DIR="/memex/projects"
AREAS_DIR="/memex/areas"
RESOURCES_DIR="/memex/resources"
ARCHIVES_DIR="/memex/archives"
SYSTEM_DIR="/memex/system"
# Identity Configuration
MEMEX_USER="YourName"
MEMEX_ASSISTANT="AgentName"
RECIPIENT_ID="+1..." # For Signal/Telegram delivery

0
README.md Normal file
View File

183
README.org Normal file
View File

@@ -0,0 +1,183 @@
#+TITLE: org-agent: The Neurosymbolic Kernel
#+AUTHOR: User
#+CREATED: [2026-03-17 Tue]
#+UPDATED: [2026-03-24 Tue]
A hyper-minimalist, self-editing, proactive AI agent framework. `org-agent` acts as the "executive soul" of a personal OS, using Org-mode as its native memory and Common Lisp as its deterministic reasoning engine.
* The Philosophy
** Mandate 1: Strictly Org-mode and Common Lisp
The system is built on a "No Legacy" policy. Markdown (.md) and JSON are strictly prohibited for internal system logic, planning, and memory. Org-mode is the native Abstract Syntax Tree (AST) for both human and machine, and Common Lisp (SBCL) is the deterministic reasoning engine.
** Mandate 2: Minimalist Core, Skill-Based Extension
The `org-agent` kernel (the Daemon) MUST remain a minimalist microkernel. It handles only the cognitive loop, the persistent Object-Store, and the communication protocol. All business logic, LLM provider connectors, and task-management rules MUST be implemented as hot-reloadable **Skills** living in the user's Memex.
** Why Org-mode? (Homoiconic Memory)
Most agent frameworks rely on a messy combination of Python scripts, JSON states, and Markdown prompts. This breaks the human-agent interface. JSON is for machines; Markdown is for humans.
*Org-mode is for both.* It provides a rigorous, hierarchical Abstract Syntax Tree (AST) that a machine can navigate deterministically, while remaining a perfectly ergonomic, human-readable text document. In this system, your notes, your tasks, your prompts, and your agent's code all live in the exact same format.
** Why Common Lisp? (The Kernel vs. The Actuators)
The `org-agent` kernel is built in Common Lisp to provide a persistent, high-performance background process (SBCL) that maintains a live, threaded Object Store in RAM. It performs heavy neurosymbolic reasoning asynchronously, decoupled from any single user interface.
This architecture treats all interfaces as external **Actuators** and **Sensors**:
- **Editor Actuator (Emacs):** A sensor array that detects file changes and executes structural refactoring.
- **Messaging Actuator (Signal/Telegram/Discord):** A delivery channel for proactive alerts and human-in-the-loop decisions.
- **Web Actuator (Dashboard):** A visual telemetry interface for monitoring the live kernel state.
* The Architecture: The Cognitive Loop
The core engine is agnostic to both business logic and communication channels. It routes data through a strict four-stage cognitive pipeline:
1. **Perceive:** Sensors (Emacs, Webhooks, CRON) send updates over the Org-Agent Communication Protocol (OACP). The kernel updates its live Object Store.
2. **Think (System 1):** The `neuro.lisp` module queries an LLM (e.g., Gemini, OpenAI, or local models) based on the context, asking for an intuitive, pattern-matched suggestion. It returns an *unverified* proposed action.
3. **Decide (System 2):** The `symbolic.lisp` module is the absolute gatekeeper. It takes the LLM's proposal and runs it through strict Lisp constraints (e.g., "A parent task cannot be marked DONE if it has active TODO children"). If the logic fails, the LLM is overruled.
4. **Act:** Verified commands are dispatched to the appropriate Actuators (refactoring a buffer, sending a Signal message, or updating a database).
* Extensibility: The Org-Native Skill Standard
To keep the core microkernel minimal, all capabilities (API connectors, GTD logic, Atomic Notes (Zettelkasten) memory management) are abstracted into **Skills**.
Adhering to the Lisp Machine Mandate (Code is Data), a skill is not a Python folder. A skill is a single `.org` file located in the Atomic Notes (Zettelkasten) directory.
The kernel parses these `.org` files at startup, extracts the `#+begin_src lisp` blocks, and hot-loads them into the live system. You can define a System 1 Prompt and a System 2 Verification Rule entirely within your personal notes.
* Security & Isolation
Using `eval` on text generated by LLMs or extracted from text files is fundamentally dangerous. `org-agent` implements strict defense-in-depth:
** Layer 1: Lisp-Level Sandboxing
- **Reader Safety:** `*read-eval*` is strictly disabled during AST parsing, completely neutralizing reader macro injection attacks (`#.(uiop:run-program ...)`).
- **Package Jailing:** Every Org-Native skill is dynamically compiled into its own isolated Lisp package (`:org-agent.skills.<name>`). Skills cannot accidentally (or maliciously) overwrite the core System 2 gatekeeper or collide with other skills.
** Layer 2: OS-Level Containerization
The entire Common Lisp kernel can be isolated within a "Hardware Compartment" to protect the host OS.
* Documentation
Detailed specifications and planning documents are located in the [[file:docs/][docs/]] directory:
- [[file:docs/PRD.org][Product Requirements Document (PRD)]]
- [[file:docs/PROTOCOL.org][Communication Protocol (OACP)]]
- [[file:docs/PHASE_2_ROADMAP.org][Phase 2 Roadmap]]
- Specialized PRDs for [[file:docs/PRD_PROJECT_FOUNDRY.org][Project Foundry]], [[file:docs/PRD_ORG_DELIVERY.org][Org Delivery]], [[file:docs/PRD_LLM_CASCADE.org][LLM Cascade]], and more.
* Hardware Compartments (Deployment)
`org-agent` supports multiple levels of isolation. Choose the compartment that fits your security and performance needs. See the `deploy/` directory for templates.
** 1. Bare Metal
Run directly on your host CPU for maximum performance. Best for development.
** 2. Docker (Standard)
The default containerized experience.
** 3. LXC / Systemd-nspawn
Lightweight Linux containers with lower overhead than Docker.
** 4. Virtual Machines (Debian / Fedora)
Strong isolation using Vagrant/VirtualBox. Ensures zero-leakage from the Lisp machine.
** 5. Functional Deployment (Guix)
Reproducible, declarative environment management.
* Installation & Setup Guide
This guide covers the standard distributed deployment: running the `org-agent` daemon on a remote Docker server, connecting to it from your local Emacs instance, and configuring dynamic LLMs (like OpenRouter).
** Step 1: Server Setup (Global Docker Compose)
`org-agent` is designed to fit into a professional multi-app Docker environment.
1. **Clone the repository on your server:**
#+begin_src bash
git clone http://10.10.10.43:3000/amr/memex-amero.git /home/amr/memex
#+end_src
2. **Configure your Environment (.env):**
Place your `.env` file in `/docker/compose/` alongside your master `docker-compose.yml`.
#+begin_src bash
# Create /docker/compose/.env with your keys:
# OPENROUTER_API_KEY=your_key_here
# ORG_AGENT_DAEMON_PORT=9105
# ORG_AGENT_WEB_PORT=8080
# MEMEX_DIR=/memex
#+end_src
3. **Integrate into Global Compose:**
Add the following service fragment to your master file at `/docker/compose/docker-compose.yml`:
#+begin_src yaml
services:
org-agent:
build:
context: /home/amr/memex/projects/org-agent
dockerfile: deploy/docker/Dockerfile
container_name: org-agent
restart: unless-stopped
ports:
- "9105:9105"
- "8080:8080"
volumes:
- /docker/memex:/memex
env_file:
- .env
#+end_src
4. **Start the Service:**
#+begin_src bash
cd /docker/compose
docker-compose up -d org-agent
#+end_src
** Step 2: Local Emacs Setup (The Actuator)
Your laptop acts as the sensor/actuator array.
1. **Load the Emacs Package:**
Evaluate the `org-agent.el` file in your local Emacs.
#+begin_src elisp
(add-to-list 'load-path "/path/to/local/org-agent/src")
(require 'org-agent)
#+end_src
2. **Configure the Connection:**
Tell Emacs where your Docker server is located.
#+begin_src elisp
(setq org-agent-host "10.0.0.5") ;; Replace with your server's IP
(setq org-agent-port 9105)
#+end_src
3. **Connect to the Brain:**
Run the interactive command to establish the OACP socket.
`M-x org-agent-connect`
** Step 3: Dynamic Model Configuration (Homoiconic Setup)
`org-agent` does not use external JSON config files for its behavior. You configure the agent directly within your Org-mode Memex.
1. Open any `.org` file in your Memex (e.g., `settings.org`).
2. Add the following property to define your preferred model:
#+begin_src org
* Agent Settings
:PROPERTIES:
:LLM_MODEL_OPENROUTER: google/gemini-pro-1.5
:END:
#+end_src
3. **Save the buffer.** The agent instantly detects the change via Emacs, updates its internal Object Store, and routes all future neural thoughts through the selected model.
To see all available models, simply type `@agent list models` in any Org buffer and save.
* Current State: Phase 3 (The Self-Editing Kernel) Achieved
- DONE Core Lisp microkernel (Cognitive Loop: Perceive -> Think -> Decide -> Act)
- DONE OACP (Swank/Socket communication protocol) implemented
- DONE Org AST-to-Lisp conversion logic & Object Store integration
- DONE System 2 Safety Gating (The Harness) established
- DONE Org-Native Skill parsing and loading
- DONE Secure Docker containerization
- DONE Skill Graph & Recursive Dependencies (Ars Contexta)
- DONE Multi-Provider LLM Failover Cascade
- DONE Context API (Peripheral Vision)
- DONE Heartbeat Loop (Proactive Awareness)
- DONE Immune System (Autonomous Self-Repair)
- DONE Web Dashboard (Visual Telemetry)
- DONE Org-Native Multi-modal Delivery (Signal/Telegram/Discord)
- DONE Project Foundry (Autonomous Scaffolding & Git Stewardship)
- DONE Strictly Org-mode Mandate (.md purge)

View File

@@ -0,0 +1,46 @@
#!/bin/bash
# org-agent: Bare Metal Installation Script
# This script sets up the org-agent daemon on a Linux host (Debian/Fedora).
set -e
echo "--- org-agent: Bare Metal Installation ---"
# 1. Check Dependencies
echo "[1/4] Checking dependencies..."
for cmd in sbcl curl git ripgrep; do
if ! command -v $cmd &> /dev/null; then
echo "Error: $cmd is not installed. Please install it first."
exit 1
fi
done
# 2. Setup Quicklisp
if [ ! -d "$HOME/quicklisp" ]; then
echo "[2/4] Quicklisp not found. Installing..."
curl -O https://beta.quicklisp.org/quicklisp.lisp
sbcl --non-interactive --load quicklisp.lisp --eval '(quicklisp-quickstart:install)'
rm quicklisp.lisp
echo "Quicklisp installed."
else
echo "[2/4] Quicklisp already installed."
fi
# 3. Build standalone binary
echo "[3/4] Building standalone binary..."
PROJECT_ROOT=$(pwd)/../..
sbcl --non-interactive \
--eval "(push \"$PROJECT_ROOT/\" asdf:*central-registry*)" \
--eval "(ql:quickload :org-agent)" \
--eval "(asdf:make :org-agent)"
echo "Binary built: $PROJECT_ROOT/org-agent-server"
# 4. Instructions for Systemd
echo "[4/4] Installation complete."
echo ""
echo "To run as a systemd service:"
echo "1. Edit org-agent.service to set correct paths."
echo "2. sudo cp org-agent.service /etc/systemd/system/"
echo "3. sudo systemctl daemon-reload"
echo "4. sudo systemctl enable --now org-agent"

View File

@@ -0,0 +1,18 @@
[Unit]
Description=org-agent: Neurosymbolic Lisp Machine Kernel
After=network.target
[Service]
Type=simple
# Update User and WorkingDirectory to match your local setup
User=amr
WorkingDirectory=/home/amr/.openclaw/workspace/memex/5_projects/org-agent
ExecStart=/home/amr/.openclaw/workspace/memex/5_projects/org-agent/org-agent-server
Restart=always
RestartSec=10
# Environment variables can be loaded from the .env file
EnvironmentFile=/home/amr/.openclaw/workspace/memex/5_projects/org-agent/.env
[Install]
WantedBy=multi-user.target

41
deploy/docker/Dockerfile Normal file
View File

@@ -0,0 +1,41 @@
FROM debian:bookworm-slim
# Install SBCL, ripgrep, and build dependencies
RUN apt-get update && \
apt-get install -y sbcl build-essential curl git ripgrep libsqlite3-dev lynx python3 python3-pip && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Install Quicklisp globally
RUN curl -O https://beta.quicklisp.org/quicklisp.lisp && \
sbcl --non-interactive \
--load quicklisp.lisp \
--eval '(quicklisp-quickstart:install :path "/opt/quicklisp")' \
--eval '(ql-util:without-prompting (ql:add-to-init-file))' && \
rm quicklisp.lisp
# Set up the working directory
WORKDIR /app
# Copy source code and system definition
COPY org-agent.asd /app/
COPY src/ /app/src/
# Build the standalone binary natively inside the container
# This ensures GLIBC compatibility with the runtime environment.
RUN sbcl --non-interactive \
--eval '(push "/app/" asdf:*central-registry*)' \
--eval '(ql:quickload :org-agent)' \
--eval '(asdf:make :org-agent)'
# Ensure the binary is executable
RUN chmod +x /app/org-agent-server
# Expose the OACP and Web Dashboard ports
EXPOSE 9105 8080
# The app expects the memex to be mounted here
VOLUME /memex
# Run the natively compiled standalone daemon
CMD ["./org-agent-server"]

View File

@@ -0,0 +1,20 @@
version: '3.8'
services:
org-agent:
build:
context: ../..
dockerfile: deploy/docker/Dockerfile
container_name: org-agent
restart: unless-stopped
ports:
- "${ORG_AGENT_DAEMON_PORT:-9105}:${ORG_AGENT_DAEMON_PORT:-9105}"
- "${ORG_AGENT_WEB_PORT:-8080}:${ORG_AGENT_WEB_PORT:-8080}"
volumes:
- /docker/memex:/home/amr/.openclaw/workspace/memex
env_file:
- .env
networks:
sandbox-net:
driver: bridge

14
deploy/guix/manifest.scm Normal file
View File

@@ -0,0 +1,14 @@
;; org-agent: Guix Environment Manifest
;; Usage: guix shell -m manifest.scm -- sbcl --eval ...
(specifications->manifest
'("sbcl"
"sbcl-cl-json"
"sbcl-bordeaux-threads"
"sbcl-usocket"
"sbcl-dexador"
"sbcl-cl-ppcre"
"ripgrep"
"git"
"curl"
"sqlite"))

33
deploy/lxc/setup.org Normal file
View File

@@ -0,0 +1,33 @@
#+TITLE: LXC / Systemd-nspawn Deployment Guide
#+AUTHOR: org-agent
* Overview
For users who prefer containerization without the overhead or dependency on the Docker daemon, `org-agent` can be run within a standard Linux Container (LXC) or a systemd-nspawn container.
* Systemd-nspawn Setup (Fastest for Linux users)
1. **Create the container root:**
#+begin_src bash
sudo debootstrap --arch=amd64 bookworm /var/lib/machines/org-agent
#+end_src
2. **Start and enter the container:**
#+begin_src bash
sudo systemd-nspawn -D /var/lib/machines/org-agent
#+end_src
3. **Install dependencies (inside container):**
#+begin_src bash
apt-get update && apt-get install -y sbcl curl git ripgrep libsqlite3-dev build-essential
#+end_src
4. **Bind mount the Memex directory:**
Add this to your container startup or use the `--bind` flag:
#+begin_src bash
sudo systemd-nspawn -D /var/lib/machines/org-agent --bind /home/amr/.openclaw/workspace/memex
#+end_src
* Proxmox LXC Setup
1. Create a new LXC container using the Debian 12 template.
2. Ensure the network is bridged so Emacs can reach it.
3. Run the `deploy/bare-metal/install.sh` script inside the container.

22
deploy/vms/debian/Vagrantfile vendored Normal file
View File

@@ -0,0 +1,22 @@
Vagrant.configure("2") do |config|
config.vm.box = "debian/bookworm64"
config.vm.network "forwarded_port", guest: 9105, host: 9105
config.vm.provider "virtualbox" do |vb|
vb.memory = "1024"
vb.cpus = 2
end
config.vm.provision "shell", inline: <<-SHELL
apt-get update
apt-get install -y sbcl curl git ripgrep libsqlite3-dev build-essential
# Setup for org-agent
mkdir -p /home/vagrant/org-agent
cp -r /vagrant/* /home/vagrant/org-agent/
chown -R vagrant:vagrant /home/vagrant/org-agent
# Build binary natively
sudo -u vagrant bash -c "cd /home/vagrant/org-agent && ./deploy/bare-metal/install.sh"
SHELL
end

21
deploy/vms/fedora/Vagrantfile vendored Normal file
View File

@@ -0,0 +1,21 @@
Vagrant.configure("2") do |config|
config.vm.box = "fedora/39-cloud-base"
config.vm.network "forwarded_port", guest: 9105, host: 9105
config.vm.provider "virtualbox" do |vb|
vb.memory = "1024"
vb.cpus = 2
end
config.vm.provision "shell", inline: <<-SHELL
dnf install -y sbcl curl git ripgrep sqlite-devel make gcc
# Setup for org-agent
mkdir -p /home/vagrant/org-agent
cp -r /vagrant/* /home/vagrant/org-agent/
chown -R vagrant:vagrant /home/vagrant/org-agent
# Build binary natively
sudo -u vagrant bash -c "cd /home/vagrant/org-agent && ./deploy/bare-metal/install.sh"
SHELL
end

39
docs/PHASE_2_ROADMAP.org Normal file
View File

@@ -0,0 +1,39 @@
#+TITLE: Phase 2: The Delegator (Roadmap)
#+AUTHOR: org-agent
#+DATE: [2026-03-23 Mon]
* Overview
Phase 2 shifts focus from kernel architecture to capability expansion. Because the kernel (Phase 1) is a generalized Event Bus, **all Phase 2 features are implemented entirely as Org-Native Skills**. No modifications to the Lisp Core (`core.lisp`, `neuro.lisp`, `symbolic.lisp`) are required.
* The Phase 2 Skill Roster
** 1. The LLM Router
- **Skill File:** `skills/skill-router.org`
- **Role:** Meta-cognition. Replaces complex Lisp trigger logic with LLM intent classification.
- **Trigger:** Catches general `:user-command` or ambiguous `:buffer-update` events.
- **Action:** Instead of editing a file, it emits an internal Lisp event `(:type :EVENT :payload (:sensor :delegation :target-skill <skill-name>))`, forcing the engine to re-loop and trigger the specific skill.
** 2. Deep Memory (Vector/Semantic Search)
- **Skill File:** `skills/skill-atomic-notes.org`
- **Role:** Context augmentation beyond the live RAM `*object-store*`.
- **System 2 (Lisp):** Wraps a local search tool (like `ripgrep` or a lightweight Lisp vector db).
- **System 1 (LLM):** Formats the retrieved text into the Context API so downstream skills can read it before making decisions.
** 3. The Shell Actuator
- **Skill File:** `skills/skill-shell-actuator.org`
- **Role:** Gives the agent hands outside of Emacs.
- **Registration:** Uses `(org-agent:register-actuator :shell #'execute-shell-safely)`.
- **System 2 (Gatekeeper):** A massive, paranoid Lisp function that whitelist-checks commands (e.g., allows `git status`, blocks `rm`).
** 4. The Cron Scheduler
- **Skill File:** `skills/skill-cron.org`
- **Role:** Autonomous temporal action.
- **Trigger:** `(eq sensor :heartbeat)`
- **System 2 (Lisp):** Queries the `*object-store*` for deadlines. If a deadline has passed, it creates an action.
- **System 1 (LLM):** Drafts a polite, contextual warning message to send to the user.
** 5. Web Research
- **Skill File:** `skills/skill-web-research.org`
- **Role:** Internet connectivity via headless text browsing.
- **System 2 (Lisp):** Wraps `lynx` and `curl` to fetch webpage content.
- **System 1 (LLM):** Parses the raw HTML/text and synthesizes a summary for the user.

29
docs/PRD.org Normal file
View File

@@ -0,0 +1,29 @@
#+TITLE: PRD: org-agent Cognitive Core & Configuration
#+AUTHOR: PSF Requirements Definer
#+CREATED: [2026-03-23 Mon]
#+STATUS: FROZEN
* 0. Core Mandates
The `org-agent` project MUST adhere to these foundational mandates:
- **Mandate 1: Strict Homoiconic Memory.** All documentation, planning, and system logic MUST be authored in Org-mode (.org) and Common Lisp. Markdown (.md) and JSON are strictly prohibited for internal use.
- **Mandate 2: Minimalist Microkernel.** The core daemon MUST remain minimalist, handling only the cognitive loop, the persistent Object-Store, and the communication protocol. All domain-specific features and LLM provider logic MUST be implemented as hot-reloadable **Skills** living in the user's Memex.
* 1. Purpose
The `org-agent` must transition from a monolithic prototype to a generalized neurosymbolic kernel.
* 2. Functional Requirements
** 2.1. Cognitive Loop (PTA Refactor)
- The daemon MUST implement a 4-stage pipeline: Perceive -> Think -> Decide -> Act.
- System 1 (Neural) MUST be restricted to the 'Think' stage.
- System 2 (Symbolic) MUST have absolute authority in the 'Decide' stage to block or modify neural proposals.
- The I/O protocol (OACP) MUST be encapsulated in 'Perceive' and 'Act'.
** 2.2. Externalized Configuration (.env)
- All secrets (API keys) and environment-specific settings (ports, paths) MUST live in a `.env` file.
- The system MUST automatically load `.env` upon system initialization.
- Secrets MUST NOT be hardcoded or checked into source control.
* 3. Success Criteria
- TODO Daemon starts and loads LLM_API_KEY from .env.
- TODO `cognitive-loop` successfully routes a `:buffer-update` event through all 4 stages.
- TODO System 2 (`decide`) successfully blocks an `:eval` request containing "shell-command".

16
docs/PRD_LLM_CASCADE.org Normal file
View File

@@ -0,0 +1,16 @@
#+TITLE: PRD: LLM Failover Cascade & Multi-Provider Support
#+AUTHOR: PSF Requirements Definer
#+STATUS: FROZEN
* 1. Purpose
Ensure 100% availability of System 1 (Neural) reasoning via an ordered list of providers and automatic fallbacks.
* 2. Functional Requirements
- **Backend Registry:** The core MUST allow Skills to register new AI providers (Gemini, OpenAI, Claude, etc.).
- **Automatic Failover:** The `ask-neuro` function MUST iterate through a `*provider-cascade*` list, automatically trying the next provider if the previous one fails.
- **Interactive Re-Ordering:** The system MUST allow the user to update the cascade order in real-time from Emacs.
* 3. Success Criteria
- DONE Daemon correctly falls back from a failing API key to a working one.
- DONE `M-x org-agent-set-model-cascade` successfully updates the live Lisp hierarchy.
- DONE Support for 5+ providers (Gemini, OpenAI, Anthropic, OpenRouter, Ollama).

17
docs/PRD_ORG_DELIVERY.org Normal file
View File

@@ -0,0 +1,17 @@
#+TITLE: PRD: Org-Native Multi-Modal Delivery
#+AUTHOR: PSF Requirements Definer
#+STATUS: FROZEN
* 1. Purpose
Enable the agent to communicate outside of Emacs while maintaining all outbound records in a human-readable, homoiconic format.
* 2. Functional Requirements
- **Org-Native Outbox:** The system MUST use a central Org file (`9_system/delivery.org`) as its outbound message queue.
- **Actuator API:** A `:delivery` actuator MUST be registered to handle external messaging intents.
- **Channel Support:** The system MUST support Signal, Telegram, and Discord metadata.
- **Proactive Alerts:** The Cron skill MUST be able to route alerts to the delivery actuator based on Org properties.
* 3. Success Criteria
- DONE New messages appear as structured headlines in `delivery.org`.
- DONE Metadata (channel, target) is stored in native Org property drawers.
- DONE No hidden JSON or external database required for queueing.

View File

@@ -0,0 +1,17 @@
#+TITLE: PRD: Project Foundry (Scaffolding)
#+AUTHOR: PSF Requirements Definer
#+STATUS: FROZEN
* 1. Purpose
Act as a senior engineer by autonomously scaffolding new project directories, git repos, and documentation templates.
* 2. Functional Requirements
- **Directory Creation:** The system MUST be able to create new folders within the `$PROJECTS_DIR`.
- **Git Integration:** The foundry MUST initialize a new git repository for every scaffolded project.
- **Boilerplate Generation:** The system MUST write a customized `README.org` for new projects.
- **GTD Connection:** The foundry MUST automatically add a `PROJ` headline to `gtd.org` with a dynamic `:PROJECT_PATH:`.
* 3. Success Criteria
- DONE New projects appear in the physical filesystem and the GTD plan simultaneously.
- DONE Path resolution handles environment variables (`$PROJECTS_DIR`) correctly.
- DONE System 2 prevents overwriting existing project directories.

16
docs/PRD_SKILL_GRAPH.org Normal file
View File

@@ -0,0 +1,16 @@
#+TITLE: PRD: The Skill Graph & Self-Awareness
#+AUTHOR: PSF Requirements Definer
#+STATUS: FROZEN
* 1. Purpose
Enable recursive, networked intelligence by unifying logic (skills) and knowledge (Atomic Notes (Zettelkasten)).
* 2. Functional Requirements
- **Dependency Tracking:** Skills MUST be able to declare dependencies on other skills via `#+DEPENDS_ON:`.
- **Introspection:** The kernel MUST provide an API to list all active skills and read their source code.
- **Topological Dispatch:** The system MUST resolve dependencies recursively before engaging a skill.
* 3. Success Criteria
- DONE `(context-list-all-skills)` returns accurate priority and dependency metadata.
- DONE Skills jailed in isolated packages can still resolve symbols from their declared dependencies.
- DONE The Brain Mapper skill can successfully visualize the network.

View File

@@ -0,0 +1,17 @@
#+TITLE: PRD: Kernel Web Dashboard
#+AUTHOR: PSF Requirements Definer
#+STATUS: FROZEN
* 1. Purpose
Provide a read-only visual interface for monitoring the Lisp Machine's internal state and execution logs.
* 2. Functional Requirements
- **Web Server:** A lightweight HTTP server (Hunchentoot) MUST run inside the kernel.
- **Skill Visualization:** The dashboard MUST display the current Skill Graph, priorities, and jailing status.
- **Log Monitoring:** The dashboard MUST display the most recent 20-50 system log entries.
- **Remote Access:** The dashboard MUST be accessible from other computers on the local network (binding to 0.0.0.0).
* 3. Success Criteria
- DONE Dashboard loads at `http://<ip>:8081`.
- DONE Skill list updates dynamically as new skills are hot-loaded.
- DONE Kernel errors are visible in the web UI.

View File

@@ -0,0 +1,23 @@
#+TITLE: PRD: Skill-Based Dynamic Model Switching
#+AUTHOR: PSF Requirements Definer
#+DATE: 2026-03-24
#+STARTUP: content
* Overview
The `org-agent` currently relies on hardcoded LLM model strings within its pluggable provider skills. To match the flexibility of the legacy Openclaw system while maintaining a **Minimalist Core**, we will implement a "Homoiconic Configuration" model using the **Skill Graph**. Configuration settings will live as standard Org-mode properties in the user's memex, and a dedicated configuration skill will provide lookup services to other skills.
* Mandates
- **The Org Mandate:** Configuration MUST be human-editable and machine-readable within `.org` files.
- **Minimalist Core:** No new global variables or state-management registries shall be added to the kernel (`src/*.lisp`).
- **Skill Graph Sovereignty:** Capabilities MUST build upon one another via dependencies.
* Requirements
1. **Property-Based Configuration:** The system MUST allow defining LLM models via Org-mode properties (e.g., `:LLM_MODEL_OPENAI: gpt-4o`) anywhere in the loaded memex files.
2. **Configuration Skill:** A new skill (`skill-environment-config.org`) MUST be created to handle property lookups within the persistent `*object-store*`.
3. **Skill Integration:** Provider skills (OpenAI, Anthropic, OpenRouter) MUST declare a dependency on `skill-environment-config` and query it for their model string at runtime.
4. **Late-Binding Updates:** Because the kernel updates the `*object-store*` on every buffer save, changing a model string in an Org file results in an immediate, hot-swapped change to the agent's behavior.
* Acceptance Criteria
- A user can add `:LLM_MODEL_OPENAI: gpt-4o` to a "Settings" headline in their memex.
- The OpenAI provider skill successfully fetches this value via the configuration skill.
- No modifications are made to the core Lisp daemon files.

View File

@@ -0,0 +1,22 @@
#+TITLE: PRD: Model Discovery
#+AUTHOR: PSF Requirements Definer
#+DATE: 2026-03-24
#+STARTUP: content
* Overview
Users currently lack visibility into the LLM models available via the registered provider skills (OpenAI, Anthropic, OpenRouter). To match Openclaw's UX while preserving a Minimalist Core, the system needs an intra-skill discovery protocol.
* Mandates
- **The Org Mandate:** Outputs must be rendered natively in Org-mode buffers.
- **Minimalist Core:** No state or new capabilities shall be added to the Lisp Daemon. All discovery logic must be localized to the Skill Graph.
* Requirements
1. **Dynamic Provider Introspection:** The system MUST be able to query loaded skills dynamically to find which ones act as LLM providers.
2. **Model Listing API:** Every provider skill MUST export a function that returns a list of its available models (e.g., ID, Context Window).
3. **The Explorer Skill:** A new skill (`skill-model-explorer.org`) MUST intercept the command `@agent list models` and aggregate the results from all providers.
4. **Org-Table Output:** The Explorer Skill MUST output the aggregated list back to the Emacs buffer formatted as an `org-table` for immediate human readability.
* Acceptance Criteria
- Writing `@agent list models` in an Org buffer and saving triggers the Explorer Skill.
- An Org-mode table is inserted below the command containing columns for `Provider`, `Model`, and `Context`.
- Adding a new provider skill automatically includes its models in future queries without modifying the Explorer Skill.

137
docs/PROTOCOL.org Normal file
View File

@@ -0,0 +1,137 @@
#+TITLE: org-agent Communication Protocol (OACP)
#+AUTHOR: Agent
#+DATE: 2026-03-22
#+ID: org-agent-protocol
#+STARTUP: content
* Core Mandates
The OACP and the `org-agent` project MUST adhere to these foundational mandates:
- **Mandate 1: Strict Homoiconic Memory.** All documentation, planning, and system logic MUST be authored in Org-mode (.org) and Common Lisp. Markdown (.md) and JSON are strictly prohibited for internal use.
- **Mandate 2: Minimalist Microkernel.** The core daemon MUST remain minimalist, handling only the cognitive loop, the persistent Object-Store, and the communication protocol. All domain-specific features and LLM provider logic MUST be implemented as hot-reloadable **Skills** living in the user's Memex.
* Overview
OACP (org-agent Communication Protocol) defines the "nervous system" of the Neurosymbolic Lisp Machine. It facilitates bidirectional, asynchronous communication between the **Common Lisp Core** (The Soul/Daemon) and the **Emacs Interface** (The Actuator/Terminal).
The protocol is designed to be:
- **Homoiconic:** Messages are native Lisp S-expressions (plists).
- **Asynchronous:** Neither side should block waiting for the other.
- **Extensible:** New sensors and actuators can be added by defining new plist keys.
* Framing & Transport
- **Transport:** TCP Socket (default port: 9105).
- **Framing:** Each message is prefixed by a 6-character hexadecimal length string (e.g., `00002a` for a 42-byte message), followed by the S-expression string encoded in UTF-8.
* Message Structure
A standard message is a flat property list (plist):
#+begin_src lisp
(:type TYPE :id ID :payload PAYLOAD)
#+end_src
- **:type:** One of `:EVENT`, `:REQUEST`, `:RESPONSE`, or `:LOG`.
- **:id:** A unique integer or string for correlation (mandatory for `:REQUEST` and `:RESPONSE`).
- **:payload:** A nested plist containing the specific data or command.
* Perception (Emacs -> Core)
Emacs acts as the sensor array, pushing events to the Core daemon.
** :BUFFER-MODIFIED
Sent when an Org buffer is changed or saved.
#+begin_src lisp
(:type :EVENT :payload (:sensor :buffer-update :file "/home/amr/org/todo.org" :state :saved))
#+end_src
** :USER-INPUT
Sent when the user explicitly triggers an agent action (e.g., `M-x org-agent-ask`).
#+begin_src lisp
(:type :EVENT :payload (:sensor :user-prompt :text "Refactor this subtree" :context :current-subtree))
#+end_src
** :CURSOR-MOVED
Sent when the agent needs to track the user's focus.
#+begin_src lisp
(:type :EVENT :payload (:sensor :focus :buffer "init.el" :line 42 :column 0))
#+end_src
* Action (Core -> Emacs)
The Core daemon sends requests to Emacs to perform physical actions in the environment.
** :EXECUTE-ELISP
The primary "actuator."
#+begin_src lisp
(:type :REQUEST :id 101 :payload (:action :eval :code "(org-todo \"NEXT\")"))
#+end_src
** :UI-NOTIFY
Display info to the user without interrupting focus.
#+begin_src lisp
(:type :REQUEST :id 102 :payload (:action :message :text "I have identified a contradiction in your GTD.org" :level :warning))
#+end_src
** :PROVIDE-COMPLETION
Provide LLM-driven completions for the current point.
#+begin_src lisp
(:type :REQUEST :id 103 :payload (:action :complete :prefix "defun" :suggestions ("defun-agent" "defun-percieve")))
#+end_src
** :ORG-DELIVERY
Enqueue external messages.
#+begin_src lisp
(:type :REQUEST :target :delivery :payload (:channel :signal :to "+1..." :text "Hello"))
#+end_src
** :PROJECT-SCAFFOLD
Create new project folders and headings.
#+begin_src lisp
(:type :REQUEST :target :foundry :payload (:action :scaffold :name "Project-X" :type "Lisp"))
#+end_src
** :SYSTEM-SELF-EDIT
Kernel-level self-modification.
#+begin_src lisp
(:type :REQUEST :target :system :payload (:action :create-skill :filename "skill-x.org" :content "..."))
#+end_src
* Metadata & Handshake
** :HELLO
Sent by both sides upon connection.
#+begin_src lisp
(:type :EVENT :payload (:action :handshake :version "0.1.0" :capabilities (:auth :swank :org-ast)))
#+end_src
* Cognitive Loop Internal API (Foundry Standard)
To ensure neurosymbolic sovereignty, the Core Daemon must strictly separate stages.
** 1. Perceive
- **Signature:** `(perceive raw-stimulus) -> context-plist`
- **Responsibility:** Protocol decoding, Object-Store synchronization, context augmentation.
** 2. Think (System 1)
- **Signature:** `(think context-plist) -> proposed-action-plist`
- **Responsibility:** Neural pattern matching, associative retrieval, unverified proposal generation.
** 3. Decide (System 2)
- **Signature:** `(decide proposed-action context) -> approved-action-plist`
- **Responsibility:** Symbolic verification, safety gating, deterministic rule application (e.g., GTD logic).
** 4. Act (Dispatch)
- **Signature:** `(act stream approved-action)` / `(dispatch-action approved-action)`
- **Responsibility:** Actuator lookup and transmission. Targets: `:emacs`, `:delivery`, `:system`, `:shell`.
** 5. Context API (System 1 Peripheral Vision)
- **(context-list-all-skills):** Introspect the brain hierarchy.
- **(context-get-skill-source name):** Read the logic graph.
- **(context-resolve-path path):** Environment-relative path expansion.
- **(context-get-system-logs limit):** Perception of kernel failures.
* Success Metrics
- Latency between `:EVENT` and `:RESPONSE` < 100ms for UI actions.
- 100% fidelity in Org AST preservation across the bridge.
- Zero UI blocking in Emacs during heavy CL reasoning loops.

View File

@@ -0,0 +1,47 @@
#+TITLE: PROTOCOL: Skill-Based Configuration
#+AUTHOR: PSF Architect
#+DATE: 2026-03-24
#+STARTUP: content
* Overview
This protocol defines the skill-to-skill interface for retrieving environment configuration from the Org-mode Object-Store. It leverages the **Skill Graph** to provide a centralized configuration API for all other skills.
* The Configuration Skill (`skill-environment-config.org`)
** 1. Internal Logic
The skill iterates over the kernel's `*object-store*` to find headlines containing specific properties.
#+begin_src lisp
(defun get-config-attribute (property-key &optional default)
"Searches the global *object-store* for any headline containing PROPERTY-KEY."
(let ((store org-agent:*object-store*))
(maphash (lambda (id obj)
(declare (ignore id))
(when (eq (org-agent:org-object-type obj) :HEADLINE)
(let ((val (getf (org-agent:org-object-attributes obj) property-key)))
(when val
(return-from get-config-attribute val)))))
store)
default))
#+end_src
* Skill Graph Integration
** 2. Dependency Declaration
Other skills requiring configuration MUST declare a dependency on this skill.
#+begin_src org
#+DEPENDS_ON: skill-environment-config
#+end_src
** 3. Provider Integration Example
Provider skills will invoke the config skill's API during the System 1 prompt generation.
#+begin_src lisp
(let ((model (org-agent.skills.skill-environment-config:get-config-attribute :LLM_MODEL_OPENAI "gpt-4-turbo-preview")))
;; ... use model in API call ...
)
#+end_src
* No Core Modifications Required
This protocol adheres to the **Minimalist Core** mandate by implementing the entirety of the "Dynamic Model Switching" logic within the Skill Layer.

View File

@@ -0,0 +1,51 @@
#+TITLE: PROTOCOL: Model Discovery
#+AUTHOR: PSF Architect
#+DATE: 2026-03-24
#+STARTUP: content
* Overview
This protocol defines the interfaces allowing the `org-agent` to dynamically introspect available models from any loaded provider skill, outputting the result to Emacs as an Org table.
* 1. The Provider Export Interface
Every skill acting as a Neural Provider MUST export a function named `GET-AVAILABLE-MODELS`.
#+begin_src lisp
;; Signature
(get-available-models) -> list-of-plists
;; Return Format Example
'((:id "gpt-4-turbo" :context "128k")
(:id "gpt-4o" :context "128k"))
#+end_src
* 2. The Model Explorer Skill (`skill-model-explorer.org`)
** Dynamic Introspection
The explorer uses the kernel's `(org-agent:context-list-all-skills)` API to find all skills whose name starts with `skill-provider-`.
For each matching skill, it looks up the package:
#+begin_src lisp
(let* ((pkg (find-package (intern (string-upcase (format nil "ORG-AGENT.SKILLS.~a" skill-name)) :keyword)))
(fn (when pkg (find-symbol "GET-AVAILABLE-MODELS" pkg))))
(when (and fn (fboundp fn))
(funcall fn)))
#+end_src
** Org-Table Formatting
The explorer aggregates the plists and formats them into an Org table string:
#+begin_example
| Provider | Model ID | Context |
|----------+----------+---------|
| OpenAI | gpt-4o | 128k |
#+end_example
* 3. Emacs Actuator Command
The explorer generates a System 2 `approved-action` that instructs the `:emacs` actuator to insert this text.
#+begin_src lisp
'(:type :REQUEST
:target :emacs
:payload (:action :insert-text
:text "| Provider | Model ID | Context |\n..."
:position :after-trigger))
#+end_src

BIN
org-agent-server Executable file

Binary file not shown.

29
org-agent.asd Normal file
View File

@@ -0,0 +1,29 @@
(defsystem :org-agent
:name "org-agent"
:author "Amr"
:version "0.1.0"
:license "MIT"
:description "The Neurosymbolic Lisp Machine Kernel"
:depends-on (:usocket :cl-json :bordeaux-threads :dexador :uiop :cl-dotenv :cl-ppcre :hunchentoot)
:serial t
:components ((:module "src"
:components ((:file "package")
(:file "protocol")
(:file "object-store")
(:file "skills")
(:file "neuro")
(:file "symbolic")
(:file "core"))))
:build-operation "program-op"
:build-pathname "org-agent-server"
:entry-point "org-agent:main"
:in-order-to ((test-op (test-op :org-agent/tests))))
(defsystem :org-agent/tests
:depends-on (:org-agent :fiveam)
:components ((:module "tests"
:components ((:file "oacp-tests")
(:file "cognitive-loop-tests"))))
:perform (test-op (o s)
(uiop:symbol-call :fiveam :run! (uiop:find-symbol* :oacp-suite :org-agent-tests))
(uiop:symbol-call :fiveam :run! (uiop:find-symbol* :cognitive-suite :org-agent-cognitive-tests))))

245
src/core.lisp Normal file
View File

@@ -0,0 +1,245 @@
(in-package :org-agent)
;;; ============================================================================
;;; Internal Logging (The Kernel's Senses)
;;; ============================================================================
(defvar *system-logs* nil
"A thread-safe circular buffer of recent kernel activity.")
(defvar *logs-lock* (bt:make-lock "kernel-logs-lock"))
(defvar *max-log-history* 100
"Maximum number of log entries to retain in memory.")
(defvar *skill-telemetry* (make-hash-table :test 'equal)
"Thread-safe storage for skill performance metrics.")
(defvar *telemetry-lock* (bt:make-lock "kernel-telemetry-lock"))
(defun kernel-track-telemetry (skill-name duration status)
"Records the execution time and result status of a skill."
(when skill-name
(bt:with-lock-held (*telemetry-lock*)
(let ((entry (or (gethash skill-name *skill-telemetry*)
(list :executions 0 :total-time 0 :failures 0))))
(incf (getf entry :executions))
(incf (getf entry :total-time) duration)
(when (eq status :rejected) (incf (getf entry :failures)))
(setf (gethash skill-name *skill-telemetry*) entry)))))
(defun kernel-log (fmt &rest args)
"Logs a message to both standard output and the internal circular buffer."
(let ((msg (apply #'format nil fmt args)))
(bt:with-lock-held (*logs-lock*)
(push msg *system-logs*)
;; Enforce maximum history length
(when (> (length *system-logs*) *max-log-history*)
(setf *system-logs* (subseq *system-logs* 0 *max-log-history*))))
;; Mirror to stdout for Docker/Console monitoring
(format t "~a~%" msg)
(finish-output)))
;;; ============================================================================
;;; The Autonomic Heartbeat
;;; ============================================================================
(defvar *heartbeat-thread* nil
"The background thread that provides temporal awareness.")
;;; ============================================================================
;;; The Actuator API (Event Bus)
;;; ============================================================================
;;; The Core Daemon acts as a decoupled Event Bus. Sensors (like Emacs or
;;; Cron) inject stimuli, and Actuators (like the Emacs Bridge) execute
;;; the resulting decisions.
(defvar *actuator-registry* (make-hash-table :test 'equal)
"Registry of loaded actuators. Key is a keyword (e.g., :emacs),
value is a function that executes an action plist.")
(defun register-actuator (name fn)
"Adds a new actuator function to the system.
Called by I/O skills (like sk-emacs-bridge) during startup."
(setf (gethash name *actuator-registry*) fn))
(defun inject-stimulus (raw-message)
"The entry point for all external data. This triggers the Cognitive Loop.
It implements 'Fault-Tolerant Reasoning' using Lisp restarts. If a
skill crashes, the daemon survives and moves to the next event."
(restart-case
(handler-bind ((error (lambda (c)
(kernel-log "SYSTEM ERROR (inject-stimulus): ~a~%" c)
;; Log the error and invoke the skip-event restart
(invoke-restart 'skip-event))))
(cognitive-loop raw-message))
(skip-event ()
(kernel-log "SYSTEM RECOVERY: Stimulus dropped to prevent kernel panic.~%"))))
(defun dispatch-action (action)
"Routes an approved action intent to the correct physical actuator."
(when action
(let* ((payload (getf action :payload))
;; We default to :emacs for backward compatibility.
(target (or (getf action :target) :emacs))
(actuator-fn (gethash target *actuator-registry*)))
(if actuator-fn
(funcall actuator-fn action)
(kernel-log "DISPATCH ERROR: No actuator registered for target ~a~%" target)))))
;;; ============================================================================
;;; System Actuator (Self-Editing)
;;; ============================================================================
(defun execute-system-action (action)
"Handles internal kernel operations like skill creation and hot-reloading."
(let* ((payload (getf action :payload))
(cmd (getf payload :action)))
(case cmd
(:create-skill
(let* ((filename (getf payload :filename))
(content (getf payload :content))
(skills-dir (merge-pathnames "skills/" (asdf:system-source-directory :org-agent)))
(full-path (merge-pathnames filename skills-dir)))
(kernel-log "ACTUATOR [System] - Creating skill ~a..." filename)
(with-open-file (out full-path :direction :output :if-exists :supersede)
(write-string content out))
;; Hot-Reload immediately
(load-skill-from-org full-path)
(kernel-log "ACTUATOR [System] - Skill ~a hot-reloaded." filename)))
(:set-cascade
(let ((new-cascade (getf payload :cascade)))
(setf *provider-cascade* new-cascade)
(kernel-log "ACTUATOR [System] - LLM Cascade updated to: ~a" new-cascade)))
(:set-priority
(let* ((name (string-downcase (format nil "~a" (getf payload :skill))))
(val (getf payload :priority))
(skill (gethash name *skills-registry*)))
(if skill
(progn
(setf (skill-priority skill) val)
(kernel-log "ACTUATOR [System] - Set priority of ~a to ~a" name val))
(kernel-log "ACTUATOR [System] ERROR - Skill ~a not found" name))))
(t (kernel-log "ACTUATOR [System] - Unknown command ~a" cmd)))))
;;; ============================================================================
;;; The Cognitive Loop (OODA)
;;; ============================================================================
;;; This is the pure, deterministic pipeline of the Lisp Machine.
;;; It coordinates the transition from Perception to Action.
(defun cognitive-loop (raw-message)
"Orchestrates the four stages of cognition with performance tracking."
(let* ((start-time (get-internal-real-time))
(context (perceive raw-message))
(skill (find-triggered-skill context))
(skill-name (when skill (skill-name skill))))
(let* ((proposed-action (think context))
(approved-action (decide proposed-action context))
(status (if (and proposed-action (null approved-action)) :rejected :success))
(end-time (get-internal-real-time))
(duration (- end-time start-time)))
;; Record telemetry for the engaged skill
(when skill-name
(kernel-track-telemetry skill-name duration status))
(dispatch-action approved-action))))
(defun perceive (raw-message)
"Updates the Object Store based on incoming stimulus and returns the context."
(let ((type (getf raw-message :type))
(payload (getf raw-message :payload)))
(kernel-log "PERCEIVE: ~a (~a)" type (or (getf payload :sensor) "no-sensor"))
(cond
((eq type :EVENT)
(let ((sensor (getf payload :sensor)))
(case sensor
(:buffer-update
(let ((ast (getf payload :ast)))
(when ast (ingest-ast ast))))
;; Ensure we don't return NIL for these
(:user-command t)
(:heartbeat t)
(:chat-message t))))
((eq type :RESPONSE)
(kernel-log "ACT RESULT: ~a" (getf payload :status))))
;; ALWAYS return the raw message as the context base
raw-message))
(defun dispatch-action (action)
"Sends an approved action to the appropriate actuator."
(when (and action (not (eq action :rejected)))
(let ((target (getf action :target)))
(kernel-log "DISPATCH: Target ~a" target)
(let ((actuator (gethash target *actuators*)))
(if actuator
(funcall actuator action)
(kernel-log "ERROR: No actuator registered for ~a" target))))))
;;; ============================================================================
;;; Daemon Lifecycle Management
;;; ============================================================================
(defun start-heartbeat ()
"Spawns the background pulse thread.
Interval is controlled via HEARTBEAT_INTERVAL in .env."
(let* ((env-interval (uiop:getenv "HEARTBEAT_INTERVAL"))
(interval (if env-interval (parse-integer env-interval :junk-allowed t) 60)))
(setf *heartbeat-thread*
(bt:make-thread
(lambda ()
(loop
(sleep interval)
(kernel-log "KERNEL: Heartbeat pulse...~%")
(let* ((unix-time (get-universal-time))
;; Inject a synthetic temporal event into the Event Bus.
(heartbeat-msg `(:type :EVENT :payload (:sensor :heartbeat :unix-time ,unix-time))))
(inject-stimulus heartbeat-msg))))
:name "org-agent-heartbeat"))))
(defun stop-heartbeat ()
"Gracefully terminates the pulse thread."
(when (and *heartbeat-thread* (bt:thread-alive-p *heartbeat-thread*))
(bt:destroy-thread *heartbeat-thread*)
(setf *heartbeat-thread* nil)))
(defun load-all-skills ()
"Scans the directory defined by SKILLS_DIR (defaults to notes) and hot-loads all skills.
This is where the daemon acquires its intelligence, now unified with the Atomic Notes (Zettelkasten)."
(let* ((env-path (uiop:getenv "SKILLS_DIR"))
(skills-dir (if env-path
(uiop:ensure-directory-pathname env-path)
(merge-pathnames "notes/" (uiop:ensure-directory-pathname (uiop:getenv "MEMEX_DIR"))))))
(if (uiop:directory-exists-p skills-dir)
(progn
(kernel-log "KERNEL: Loading skills from consolidated Atomic Notes (Zettelkasten): ~a" skills-dir)
(dolist (file (uiop:directory-files skills-dir "skill-*.org"))
(load-skill-from-org file)))
(kernel-log "KERNEL ERROR: Skills directory not found at ~a" skills-dir))))
(defun start-daemon (&key (port 9105))
"Boots the Neurosymbolic Kernel.
1. Loads skills.
2. Starts the heartbeat.
3. Becomes ready to receive stimuli."
(declare (ignore port))
(register-actuator :system #'execute-system-action)
(load-all-skills)
(start-heartbeat)
(kernel-log "==================================================~%")
(kernel-log " org-agent Kernel Booted Successfully. ~%")
(kernel-log " Event Bus: ACTIVE ~%")
(kernel-log "==================================================~%"))
(defun stop-daemon ()
"Shutdown the kernel and all background threads."
(stop-heartbeat)
(kernel-log "org-agent Kernel stopped.~%"))
(defun main ()
"The entry point for the compiled standalone binary."
(start-daemon)
;; Keep the process alive.
(loop (sleep 3600)))

97
src/neuro.lisp Normal file
View File

@@ -0,0 +1,97 @@
(in-package :org-agent)
;;; ============================================================================
;;; System 1: The Neural Engine
;;; ============================================================================
;;; This module manages the connection to the LLM (Large Language Model).
;;; System 1 is responsible for 'Associative Thinking'—pattern matching over
;;; the user's notes and proposing intuitive actions. It is fast but unreliable,
;;; and its output must ALWAYS be verified by System 2.
;; Initialize environment from .env file at project root
(eval-when (:compile-toplevel :load-toplevel :execute)
(let ((env-file (merge-pathnames ".env" (asdf:system-source-directory :org-agent))))
(when (uiop:file-exists-p env-file)
(cl-dotenv:load-env env-file))))
(defun get-env (var &optional default)
"Helper: Fetches an environment variable with a fallback default."
(or (uiop:getenv var) default))
(defvar *llm-api-key* (get-env "LLM_API_KEY")
"The API key for the neural engine (LLM Provider).")
(defvar *llm-endpoint* (get-env "LLM_ENDPOINT" "https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent")
"The default neural endpoint (currently defaulting to Gemini).")
;;; --- Pluggable Neuro Backends ---
(defvar *neuro-backends* (make-hash-table :test 'equal)
"Registry of neural provider backends.")
(defvar *provider-cascade* '(:gemini)
"Ordered list of backends to try for each request.")
(defun register-neuro-backend (name fn)
"Register a function to handle LLM requests for a specific backend."
(setf (gethash name *neuro-backends*) fn))
(defun ask-neuro (prompt &key (system-prompt "You are the System 1 (Neural) engine of a Neurosymbolic Lisp Machine. Provide concise, high-fidelity suggestions in Lisp plist format.") (cascade nil))
"Dispatches a prompt to the registered neural backends in order of preference."
(let ((backends (or cascade *provider-cascade*)))
(dolist (backend backends)
(let ((backend-fn (gethash backend *neuro-backends*)))
(when backend-fn
(kernel-log "SYSTEM 1: Attempting backend ~a..." backend)
(let ((result (funcall backend-fn prompt system-prompt)))
;; Check if the result indicates failure
(if (and (stringp result) (search ":LOG" result) (search "Failure" result))
(kernel-log "SYSTEM 1: Backend ~a failed. Falling back..." backend)
(return-from ask-neuro result)))))))
;; If we fall through, the entire cascade failed
"(:type :LOG :payload (:text \"Neural Cascade Failure - All providers exhausted.\"))")
(defun execute-gemini-request (prompt system-prompt)
"The default System 1 backend (Gemini)."
(unless *llm-api-key*
(return-from execute-gemini-request "(:type :LOG :payload (:text \"Neural key missing, using mock System 1\"))"))
(let* ((url (format nil "~a?key=~a" *llm-endpoint* *llm-api-key*))
(body (cl-json:encode-json-to-string
`((contents . ((parts . ((text . ,(format nil "~a~%~%Prompt: ~a" system-prompt prompt))))))))))
(handler-case
(let* ((response (dex:post url
:headers '(("Content-Type" . "application/json"))
:content body))
(json (cl-json:decode-json-from-string response)))
(cdr (assoc :text (cdr (assoc :parts (car (cdr (assoc :parts (car (cdr (assoc :candidates json)))))))))))
(error (c)
(format nil "(:type :LOG :payload (:text \"Neural Engine Failure: ~a\"))" c)))))
;; Initialize the default backend
(register-neuro-backend :gemini #'execute-gemini-request)
(defun think (context)
"The System 1 Thinking Stage.
It dispatches to the Skill Registry to find an active skill. If found,
it executes that skill's neuro-prompt generator and queries the LLM.
Returns a proposed action plist (unverified)."
(let ((active-skill (find-triggered-skill context)))
(if active-skill
(progn
(kernel-log "SYSTEM 1: Engaging skill '~a'~%" (skill-name active-skill))
(let* ((prompt-generator (skill-neuro-prompt active-skill))
;; Execute the skill's Lisp code to build the LLM prompt.
(prompt (when prompt-generator (funcall prompt-generator context))))
(if prompt
(let* ((thought (ask-neuro prompt))
;; Read the LLM string back into a native Lisp data structure.
(suggestion (ignore-errors (read-from-string thought))))
(kernel-log "SYSTEM 1 Suggestion: ~a~%" thought)
suggestion)
;; If the skill has no neuro-prompt, it's a 'Deterministic Skill' (Symbolic-only).
'(:type :LOG :payload (:text "Skill triggered (Deterministic only)")))))
;; If no skills trigger, the agent remains silent.
nil)))

229
src/object-store.lisp Normal file
View File

@@ -0,0 +1,229 @@
(in-package :org-agent)
;;; ============================================================================
;;; CLOSOS-inspired Object Store
;;; ============================================================================
;;; This module implements the system's "Perceptual Memory."
;;; Instead of treating Org files as flat text, we parse them into a relational
;;; graph of attributed Lisp objects. This allows for fast, deterministic
;;; symbolic queries (System 2) that can inform neural suggestions (System 1).
(defvar *object-store* (make-hash-table :test 'equal)
"The global, in-memory database of all ingested Org-mode elements.
Keys are unique IDs (from Org properties or generated), values are org-object structs.")
(defstruct org-object
"The atomic unit of information in the Neurosymbolic Lisp Machine.
This mirrors the hierarchical structure of an Org-mode file but in a
format optimized for Lisp manipulation."
id ; A unique identifier (e.g., a UUID from an :ID: property)
type ; The Org element type (e.g., :HEADLINE, :PARAGRAPH, :PLAIN-LIST)
attributes ; A property list of metadata (e.g., :TITLE, :TAGS, :TODO-STATE)
content ; The raw text or non-element data within the node
parent-id ; A pointer to the parent object's ID for tree traversal
children ; A list of IDs for all immediate child nodes
version ; A timestamp or counter used for cache invalidation
last-sync ; The universal-time when this object was last updated from Emacs
)
(defun ingest-ast (ast &optional parent-id)
"Recursively transforms a nested Org AST (Abstract Syntax Tree) into a
relational graph within the *object-store*.
AST: A property list representing an Org element (from org-agent.el).
PARENT-ID: The ID of the parent element, used during recursion.
Returns the ID of the ingested node."
(let* ((type (getf ast :type))
(props (getf ast :properties))
;; We prioritize existing Org IDs. If none exists, we generate a
;; temporary ID to maintain the object's identity in the store.
(id (or (getf props :ID)
(format nil "temp-~a" (get-universal-time))))
(contents (getf ast :contents))
(child-ids nil))
;; Depth-first ingestion: Recurse into children first to gather their IDs.
(dolist (child contents)
(when (listp child)
(push (ingest-ast child id) child-ids)))
;; Create or overwrite the object in the hash table.
;; This is a 'late-binding' update—if the ID exists, we update its state.
(let ((obj (make-org-object
:id id
:type type
:attributes props
:parent-id parent-id
:children (nreverse child-ids) ; Maintain document order
:version (get-universal-time)
:last-sync (get-universal-time))))
(setf (gethash id *object-store*) obj)
id)))
(defun lookup-object (id)
"Retrieves an org-object from the store by its unique ID. Returns NIL if not found."
(gethash id *object-store*))
(defun list-objects-by-type (type)
"Returns a list of all objects matching a specific type (e.g., :HEADLINE).
Useful for bulk operations across all loaded files."
(let ((results nil))
(maphash (lambda (id obj)
(declare (ignore id))
(when (eq (org-object-type obj) type)
(push obj results)))
*object-store*)
results))
;;; ============================================================================
;;; Context API (System 1 Peripheral Vision)
;;; ============================================================================
;;; These functions provide the 'peripheral vision' for the LLM.
;;; When building a prompt, a skill can call these functions to gather
;;; relevant facts from the Object Store, preventing 'tunnel vision'.
(defun context-query-store (&key tag todo-state type)
"A high-level search engine for the Object Store.
TAG: String to search for in the :TAGS property.
TODO-STATE: The string state (e.g., 'TODO', 'DONE', 'WAITING').
TYPE: The keyword type (e.g., :HEADLINE).
Returns a list of org-object structs that satisfy ALL provided criteria."
(let ((results nil))
(maphash (lambda (id obj)
(declare (ignore id))
(let* ((attrs (org-object-attributes obj))
(obj-type (org-object-type obj))
(tags (getf attrs :TAGS))
(state (getf attrs :TODO-STATE))
(match t))
;; Filter by Type
(when (and type (not (eq obj-type type))) (setf match nil))
;; Filter by Tag (Org tags are often stored as a colon-delimited string like ':work:urgent:')
(when tag
(let ((tags-str (format nil "~a" tags)))
(unless (search tag tags-str :test #'string-equal)
(setf match nil))))
;; Filter by TODO State
(when (and todo-state (not (equal state todo-state))) (setf match nil))
(when match (push obj results))))
*object-store*)
results))
(defun context-get-active-projects ()
"Retrieves all headlines tagged with 'project' that are not yet complete.
This allows the agent to understand what the user is currently working on."
(let ((projects (context-query-store :tag "project" :type :HEADLINE)))
(remove-if (lambda (obj) (equal (getf (org-object-attributes obj) :TODO-STATE) "DONE"))
projects)))
(defun context-get-recent-completed-tasks ()
"Retrieves tasks that have been successfully finished.
Used to give the LLM context about the user's 'momentum' and recent wins."
(context-query-store :todo-state "DONE" :type :HEADLINE))
;;; ============================================================================
;;; Introspection API (Self-Awareness)
;;; ============================================================================
;;; These functions allow the agent to see its own internal configuration,
;;; such as its skill priorities and source code. This is critical for
;;; Phase 3 (Self-Editing) and autonomous priority negotiation.
(defun context-list-all-skills ()
"Returns a list of plists for all currently registered skills.
Each plist contains :name, :priority, and :dependencies.
This allows System 1 to understand the current 'Skill Graph'."
(let ((results nil))
(maphash (lambda (name skill)
(declare (ignore name))
(push (list :name (skill-name skill)
:priority (skill-priority skill)
:dependencies (skill-dependencies skill))
results))
*skills-registry*)
(sort results #'> :key (lambda (x) (getf x :priority)))))
(defun context-get-skill-source (skill-name)
"Reads the raw Org-mode source code of a specific skill.
Returns the file content as a string, or NIL if the file is missing."
(let* ((filename (format nil "~a.org" skill-name))
(skills-dir (merge-pathnames "skills/" (asdf:system-source-directory :org-agent)))
(full-path (merge-pathnames filename skills-dir)))
(if (uiop:file-exists-p full-path)
(uiop:read-file-string full-path)
nil)))
(defun context-get-system-logs (&optional (limit 20))
"Returns the most recent N lines from the kernel's execution history.
Allows the agent to 'perceive pain' (errors/rejections) and trigger self-repair."
(bt:with-lock-held (*logs-lock*)
(let ((count (min limit (length *system-logs*))))
(subseq *system-logs* 0 count))))
(defun context-get-skill-telemetry (skill-name)
"Returns performance metrics for a specific skill.
Returns a plist with :executions, :total-time, and :failures."
(bt:with-lock-held (*telemetry-lock*)
(gethash (string-downcase skill-name) *skill-telemetry*)))
(defun context-filter-sparse-tree (ast predicate)
"Recursively prunes an Org AST, keeping only nodes that match PREDICATE
and their parent hierarchies. Reduces token waste by removing noise."
(if (listp ast)
(let* ((type (getf ast :type))
(contents (getf ast :contents))
;; Recursively filter children
(filtered-contents
(remove-if #'null
(mapcar (lambda (c) (context-filter-sparse-tree c predicate))
contents))))
(if (or (funcall predicate ast)
(not (null filtered-contents)))
;; If this node matches OR has matching children, keep it
(let ((new-ast (copy-list ast)))
(setf (getf new-ast :contents) filtered-contents)
new-ast)
;; Otherwise, prune this entire branch
nil))
;; If it's a string (leaf content), keep it if the predicate says so,
;; but usually we keep it if the parent headline matches.
nil))
(defun context-resolve-path (path-string)
"Resolves environment variables in a path string (e.g., '$PROJECTS_DIR/my-proj').
This ensures project links remain valid even if base directories are moved."
(if (and (stringp path-string) (uiop:string-prefix-p "$" path-string))
(let* ((parts (uiop:split-string path-string :separator '(#\/)))
(var-name (subseq (car parts) 1)) ; Strip the '$'
(var-val (org-agent::get-env var-name))
(remaining (cl:reduce (lambda (a b) (format nil "~a/~a" a b)) (cdr parts))))
(if var-val
;; Strip any extra quotes that cl-dotenv might have preserved
(let ((clean-val (string-trim '(#\" #\Space) var-val)))
(format nil "~a/~a" (string-right-trim "/" clean-val) remaining))
path-string))
path-string))
;;; ============================================================================
;;; AST Helper Functions
;;; ============================================================================
(defun find-headline-missing-id (ast)
"A recursive utility to find any headline element that lacks a unique :ID: property.
This is used by normalization skills to ensure data integrity."
(when (listp ast)
(if (and (eq (getf ast :type) :HEADLINE)
(not (getf (getf ast :properties) :ID)))
ast
(cl:some #'find-headline-missing-id (getf ast :contents)))))
(defun file-name-nondirectory (path)
"Extracts the filename from a full path (portable across OSs)."
(let ((pos (position #\/ path :from-end t)))
(if pos (subseq path (1+ pos)) path)))

324
src/org-agent.el Normal file
View File

@@ -0,0 +1,324 @@
;;; org-agent.el --- Neurosymbolic Lisp Machine Kernel for Org-mode -*- lexical-binding: t; -*-
;; Copyright (C) 2026 Amr
;;
;; Author: Amr
;; Version: 0.1.0
;; Package-Requires: ((emacs "27.1"))
;; Keywords: convenience, org
;; URL: https://github.com/amr/org-agent
;;; Commentary:
;; org-agent provides a Neurosymbolic Lisp Machine interface for Emacs.
;; It acts as the sensor/actuator array, communicating with a persistent
;; Common Lisp daemon over a high-speed OACP socket.
;;; Code:
(require 'json)
(require 'cl-lib)
(require 'org-id)
(require 'org-element)
(defgroup org-agent nil
"Emacs interface for the org-agent Common Lisp daemon."
:group 'org)
(defcustom org-agent-port 9105
"The port the org-agent daemon is listening on."
:type 'integer
:group 'org-agent)
(defcustom org-agent-host "127.0.0.1"
"The host the org-agent daemon is running on."
:type 'string
:group 'org-agent)
(defcustom org-agent-executable-path "org-agent-server"
"Path to the compiled org-agent-server binary.
If nil, Emacs will not attempt to start the daemon automatically and
will assume you have started it manually (e.g., via SBCL)."
:type '(choice (string :tag "Path to executable")
(const :tag "Manual daemon management" nil))
:group 'org-agent)
(defvar org-agent--network-process nil
"The network process connected to the daemon.")
(defvar org-agent--daemon-process nil
"The spawned daemon child process.")
(defun org-agent--start-daemon ()
"Start the daemon binary if not already running."
(when (and org-agent-executable-path
(not (process-live-p org-agent--daemon-process)))
(message "org-agent: Starting daemon (%s)..." org-agent-executable-path)
(setq org-agent--daemon-process
(make-process
:name "org-agent-daemon"
:buffer "*org-agent-daemon*"
:command (list org-agent-executable-path (number-to-string org-agent-port))
:connection-type 'pipe))
;; Give it a moment to bind to the port
(sleep-for 1.0)))
(defun org-agent-connect ()
"Connect to the org-agent daemon, starting it if necessary."
(interactive)
(when org-agent--network-process
(delete-process org-agent--network-process))
(org-agent--start-daemon)
(condition-case err
(progn
(setq org-agent--network-process
(make-network-process
:name "org-agent"
:buffer "*org-agent*"
:family 'ipv4
:host org-agent-host
:service org-agent-port
:filter #'org-agent--filter
:sentinel #'org-agent--sentinel))
(message "org-agent: Connected to daemon."))
(error
(message "org-agent: Failed to connect to daemon at %s:%s. Ensure it is running. Error: %s"
org-agent-host org-agent-port (error-message-string err)))))
(defun org-agent-disconnect ()
"Disconnect from the org-agent daemon."
(interactive)
(when org-agent--network-process
(delete-process org-agent--network-process)
(setq org-agent--network-process nil)
(message "org-agent: Disconnected from network."))
(when org-agent--daemon-process
(delete-process org-agent--daemon-process)
(setq org-agent--daemon-process nil)
(message "org-agent: Killed daemon process.")))
(defun org-agent--filter (proc string)
"Handle incoming OACP messages from the daemon via PROC with STRING."
(let ((buf (process-buffer proc)))
(when (buffer-live-p buf)
(with-current-buffer buf
(goto-char (point-max))
(insert string)
(org-agent--process-buffer buf proc)))))
(defun org-agent--process-buffer (buffer &optional proc)
"Process the OACP message BUFFER, optionally using PROC."
(with-current-buffer buffer
(goto-char (point-min))
(while (>= (buffer-size) 6)
(let* ((len-str (buffer-substring (point-min) (+ (point-min) 6)))
(msg-len (string-to-number len-str 16)))
(if (>= (buffer-size) (+ 6 msg-len))
(let* ((msg-start (+ (point-min) 6))
(msg-end (+ msg-start msg-len))
(msg-str (buffer-substring msg-start msg-end))
(plist (car (read-from-string msg-str))))
(delete-region (point-min) msg-end)
(org-agent--handle-message proc plist))
;; Message incomplete, stop loop
(goto-char (point-max))
(setq msg-len 1000000)))))) ; Break loop
(defun org-agent--handle-message (proc plist)
"Route and execute incoming OACP messages from PROC using PLIST."
(let ((type (plist-get plist :type))
(id (plist-get plist :id))
(payload (plist-get plist :payload)))
(cond
((eq type :REQUEST)
(org-agent--execute-request proc id payload))
((eq type :RESPONSE)
(message "org-agent: Received response for ID %s" id))
(t (message "org-agent: Received unknown message type %s" type)))))
(defun org-agent--execute-request (proc id payload)
"Execute an actuator request from the daemon via PROC with ID and PAYLOAD."
(let ((action (plist-get payload :action)))
(pcase action
(:eval
(let ((code (plist-get payload :code)))
(condition-case err
(let ((result (eval (read code))))
(org-agent-send
`(:type :RESPONSE :id ,id :payload (:status :success :result ,(format "%s" result)))))
(error
(org-agent-send
`(:type :RESPONSE :id ,id :payload (:status :error :message ,(error-message-string err))))))))
(:message
(message "org-agent [DAEMON]: %s" (plist-get payload :text))
(org-agent-send `(:type :RESPONSE :id ,id :payload (:status :success))))
(:insert-at-end
(let ((buf-name (plist-get payload :buffer))
(text (plist-get payload :text)))
(save-excursion
(with-current-buffer (get-buffer-create buf-name)
(goto-char (point-max))
(insert text)
(org-agent-send `(:type :RESPONSE :id ,id :payload (:status :success)))))))
(:refactor-subtree
(let ((target-id (plist-get payload :target-id))
(properties (plist-get payload :properties)))
(condition-case err
(save-excursion
(when target-id (org-id-goto target-id))
(dolist (prop properties)
(org-set-property (car prop) (cdr prop)))
(org-agent-send `(:type :RESPONSE :id ,id :payload (:status :success))))
(error
(org-agent-send
`(:type :RESPONSE :id ,id :payload (:status :error :message ,(error-message-string err))))))))
(_
(message "org-agent: Unknown action %s" action)
(org-agent-send `(:type :RESPONSE :id ,id :payload (:status :unsupported)))))))
(defun org-agent--sentinel (proc event)
"Handle network process PROC lifecycle EVENT."
(when (string-match "finished" event)
(setq org-agent--network-process nil)
(message "org-agent: Connection lost.")))
(defun org-agent-send (plist)
"Send a Lisp PLIST to the daemon using OACP framing."
(let* ((msg (prin1-to-string plist))
(len (length msg))
(framed (format "%06x%s" len msg)))
(if (and org-agent--network-process (process-live-p org-agent--network-process))
(process-send-string org-agent--network-process framed)
(message "org-agent (offline): %s" framed))))
(defun org-agent--buffer-to-sexp ()
"Transform the current Org buffer into a pure Lisp AST (plist)."
(org-agent--clean-element (org-element-parse-buffer)))
(defun org-agent--clean-element (element)
"Recursively transform an Org ELEMENT into a pure Lisp plist."
(cond
((listp element)
(let* ((type (car element))
(props (nth 1 element))
(children (nthcdr 2 element))
(cleaned-props nil))
;; Filter and transform properties
(cl-loop for (key val) on props by 'cddr do
(unless (member key '(:standard-properties :parent :buffer))
(let ((json-val (cond
((stringp val) val)
((numberp val) val)
((booleanp val) val)
(t (format "%s" val)))))
(setq cleaned-props (plist-put cleaned-props key json-val)))))
;; Explicitly capture TODO state
(let ((todo (org-element-property :todo-keyword element)))
(when todo
(setq cleaned-props (plist-put cleaned-props :TODO-STATE (format "%s" todo)))))
(list :type type
:properties cleaned-props
:contents (mapcar #'org-agent--clean-element children))))
((stringp element) element)
(t (format "%s" element))))
;;; Sensors
(defun org-agent-notify-save ()
"Sensor: Notify daemon with full Semantic Perception (AST) when saved."
(when (and org-agent--network-process (derived-mode-p 'org-mode))
(org-agent-send
`(:type :EVENT
:payload (:sensor :buffer-update
:file ,(buffer-file-name)
:state :saved
:ast ,(org-agent--buffer-to-sexp))))))
;;; Interaction Commands
(defun org-agent-set-model-cascade (cascade-string)
"Set the ordered list of LLM providers to use as fallbacks.
CASCADE-STRING should be a comma-separated list of keywords,
e.g., ':gemini,:openai,:ollama'."
(interactive "sEnter model cascade (e.g. :gemini,:openai): ")
(unless org-agent--network-process
(org-agent-connect))
(let ((cascade (mapcar #'intern (split-string cascade-string ","))))
(org-agent-send
`(:type :REQUEST
:id ,(truncate (float-time))
:target :system
:payload (:action :set-cascade :cascade ,cascade)))
(message "org-agent: Requesting model cascade update to %s" cascade)))
(defun org-agent-chat ()
"Switch to the org-agent chat buffer, creating it if necessary."
(interactive)
(let ((buf (get-buffer-create "*org-agent-chat*")))
(with-current-buffer buf
(unless (eq major-mode 'org-mode)
(org-mode)
(local-set-key (kbd "C-c C-c") #'org-agent-chat-send)
(insert "#+TITLE: org-agent Chat\n#+STARTUP: showall\n\n* Welcome to the Neurosymbolic Lisp Machine\n\nType your message below and press `C-c C-c` to send.\n\n")))
(switch-to-buffer buf)
(goto-char (point-max))))
(defun org-agent-chat-send ()
"Send the current chat buffer content to the agent."
(interactive)
(unless org-agent--network-process
(org-agent-connect))
(let* ((text (buffer-substring-no-properties (point-min) (point-max))))
(org-agent-send
`(:type :EVENT
:payload (:sensor :chat-message
:text ,text)))
(save-excursion
(goto-char (point-max))
(insert "\n\n** Thinking...\n"))
(message "org-agent: Message sent.")))
(defun org-agent-organize-subtree ()
"Command: Ask the agent to organize the current Org subtree."
(interactive)
(org-agent-run-command :organize-subtree))
(defun org-agent-summarize-buffer ()
"Command: Ask the agent to summarize the current buffer."
(interactive)
(org-agent-run-command :summarize-buffer))
(defun org-agent-run-command (command-type)
"Generic runner for high-level COMMAND-TYPE."
(unless org-agent--network-process
(org-agent-connect))
(let ((ast (org-agent--buffer-to-sexp)))
(org-agent-send
`(:type :EVENT
:payload (:sensor :user-command
:command ,command-type
:file ,(buffer-file-name)
:ast ,ast)))
(message "org-agent: Requesting '%s'..." command-type)))
;;;###autoload
(define-minor-mode org-agent-mode
"Global minor mode for the org-agent Neurosymbolic kernel.
When enabled, this mode starts the Lisp daemon (if configured)
and establishes the network connection to enable proactive
Org-mode sensing."
:global t
:group 'org-agent
(if org-agent-mode
(progn
(add-hook 'after-save-hook #'org-agent-notify-save)
(add-hook 'kill-emacs-hook #'org-agent-disconnect)
(org-agent-connect))
(remove-hook 'after-save-hook #'org-agent-notify-save)
(remove-hook 'kill-emacs-hook #'org-agent-disconnect)
(org-agent-disconnect)))
(provide 'org-agent)
;;; org-agent.el ends here

64
src/package.lisp Normal file
View File

@@ -0,0 +1,64 @@
(defpackage :org-agent
(:use :cl)
(:export
;; --- OACP Protocol ---
#:frame-message
#:parse-message
#:make-hello-message
;; --- Daemon Lifecycle ---
#:start-daemon
#:stop-daemon
#:kernel-log
#:main
;; --- Object Store (CLOSOS) ---
#:ingest-ast
#:lookup-object
#:list-objects-by-type
#:org-object
#:org-object-id
#:org-object-type
#:org-object-attributes
#:org-object-children
;; --- Context API (Peripheral Vision) ---
#:context-query-store
#:context-get-active-projects
#:context-get-recent-completed-tasks
#:context-list-all-skills
#:context-get-skill-source
#:context-get-system-logs
#:context-filter-sparse-tree
#:context-resolve-path
#:context-get-skill-telemetry
;; --- Cognitive Loop & Event Bus ---
#:perceive
#:think
#:decide
#:act
#:cognitive-loop
#:inject-stimulus
#:dispatch-action
#:register-actuator
;; --- Skill Engine ---
#:load-skill-from-org
#:validate-lisp-syntax
#:find-triggered-skill
#:defskill
#:*skills-registry*
#:skill
#:skill-name
#:skill-priority
#:skill-trigger-fn
#:skill-neuro-prompt
#:skill-symbolic-fn
;; --- Neuro (System 1) ---
#:ask-neuro
#:register-neuro-backend
;; --- AST Helpers ---
#:find-headline-missing-id))

26
src/protocol.lisp Normal file
View File

@@ -0,0 +1,26 @@
(in-package :org-agent)
(defun frame-message (msg-string)
"Prefix MSG-STRING with a 6-character hex length (lowercase)."
(let ((len (length msg-string)))
(format nil "~(~6,'0x~)~a" len msg-string)))
(defun parse-message (framed-string)
"Extract and parse the S-expression from a framed string."
(when (< (length framed-string) 6)
(error "Framed string too short"))
(let* ((len-str (subseq framed-string 0 6))
(actual-msg (subseq framed-string 6))
(expected-len (ignore-errors (parse-integer len-str :radix 16))))
(unless expected-len
(error "Invalid hex length prefix: ~a" len-str))
(unless (= expected-len (length actual-msg))
(error "Message length mismatch. Expected ~a, got ~a" expected-len (length actual-msg)))
(read-from-string actual-msg)))
(defun make-hello-message (version)
"Construct the standard HELLO handshake message."
(list :type :EVENT
:payload (list :action :handshake
:version version
:capabilities '(:auth :swank :org-ast))))

133
src/skills.lisp Normal file
View File

@@ -0,0 +1,133 @@
(in-package :org-agent)
;;; ============================================================================
;;; Org-Native Skill Engine
;;; ============================================================================
;;; This module implements the 'Foundry' for new agent capabilities.
;;; Following the 'Code is Data' philosophy, a skill is defined entirely
;;; within a single .org file. This allows the agent's logic to live
;;; co-located with the user's personal notes.
(defvar *skills-registry* (make-hash-table :test 'equal)
"Global registry of all loaded neurosymbolic skills.
Key is the downcased skill name string.")
(defstruct skill
"The representation of a cognitive capability."
name ; Human-readable name (from #+SKILL_NAME)
priority ; Integer used to resolve conflicts when multiple skills trigger
dependencies ; A list of skill names that this skill depends on (Skill Graph)
trigger-fn ; Lisp function: (context) -> boolean
neuro-prompt ; Lisp function: (context) -> prompt-string (System 1)
symbolic-fn ; Lisp function: (proposed-action context) -> approved-action (System 2)
)
(defmacro defskill (name &key priority dependencies trigger neuro symbolic)
"The primary macro for registering a new skill.
Designed to be called from inside Org-mode Lisp blocks."
`(setf (gethash ,(string-downcase (string name)) *skills-registry*)
(make-skill :name ,(string-downcase (string name))
:priority (or ,priority 10)
:dependencies ,dependencies
:trigger-fn ,trigger
:neuro-prompt ,neuro
:symbolic-fn ,symbolic)))
(defun find-triggered-skill (context)
"The Skill Dispatcher.
Iterates over all loaded skills and returns the one with the
highest priority whose trigger returns true for the current context."
(let ((triggered nil))
(maphash (lambda (name skill)
(declare (ignore name))
;; We catch errors during trigger evaluation to prevent a
;; buggy skill from crashing the main cognitive loop.
(when (ignore-errors (funcall (skill-trigger-fn skill) context))
(push skill triggered)))
*skills-registry*)
;; Return the highest priority match.
(first (sort triggered #'> :key #'skill-priority))))
;;; ============================================================================
;;; Secure Hot-Loading Protocol
;;; ============================================================================
(defun resolve-skill-dependencies (skill-name)
"Recursively resolves all dependencies for a given skill.
Returns a flattened list of skill names in topological order."
(let ((resolved nil)
(seen nil))
(labels ((visit (name)
(unless (member name seen :test #'equal)
(push name seen)
(let ((skill (gethash (string-downcase (string name)) *skills-registry*)))
(when skill
(dolist (dep (skill-dependencies skill))
(visit dep))))
(push name resolved))))
(visit skill-name)
(nreverse resolved))))
(defun load-skill-from-org (filepath)
"Parses an Org file, extracts Lisp source blocks, and hot-loads them into
an isolated namespace. Supports #+DEPENDS_ON: for Skill Graph construction."
(when (uiop:file-exists-p filepath)
(let* ((content (uiop:read-file-string filepath))
(lines (uiop:split-string content :separator '(#\Newline)))
(in-lisp-block nil)
(lisp-code "")
(dependencies nil)
;; We derive the package name from the filename to ensure uniqueness.
(skill-base-name (pathname-name filepath))
(pkg-name (intern (string-upcase (format nil "ORG-AGENT.SKILLS.~a" skill-base-name)) :keyword)))
;; PARSE HEADER: Extract dependencies
(dolist (line lines)
(let ((clean-line (string-trim '(#\Space #\Tab #\Return) line)))
(when (uiop:string-prefix-p "#+DEPENDS_ON:" (string-upcase clean-line))
(let ((deps-str (string-trim '(#\Space #\Tab) (subseq clean-line 13))))
;; Handle both space-separated and [[wikilink]] formats
(setf dependencies
(mapcar (lambda (s) (string-trim "[] " s))
(uiop:split-string deps-str :separator '(#\Space))))))))
;; ROBUST PARSER: Scan for tags at the start of lines, ignoring trailing text like metadata.
(dolist (line lines)
(let ((clean-line (string-trim '(#\Space #\Tab #\Return) line)))
(cond
((uiop:string-prefix-p "#+begin_src lisp" (string-downcase clean-line))
(setf in-lisp-block t))
((uiop:string-prefix-p "#+end_src" (string-downcase clean-line))
(setf in-lisp-block nil))
(in-lisp-block (setf lisp-code (concatenate 'string lisp-code line (string #\Newline)))))))
(when (> (length lisp-code) 0)
(kernel-log "KERNEL: Jailing Org-Native Skill '~a' (Deps: ~a) in package ~a~%"
skill-base-name dependencies pkg-name)
;; DYNAMIC PACKAGE CREATION:
;; We create a sandbox package that :USEs :CL and :ORG-AGENT.
(unless (find-package pkg-name)
(make-package pkg-name :use '(:cl :org-agent)))
;; SECURE EVALUATION:
(let ((*read-eval* nil) ; PREVENT READ-TIME ARBITRARY CODE EXECUTION
(*package* (find-package pkg-name)))
;; We wrap the code in a PROGN so multiple forms can be evaluated at once.
(handler-case
(eval (read-from-string (format nil "(progn ~a)" lisp-code)))
(error (c)
(kernel-log "READER ERROR in skill '~a': ~a~%" skill-base-name c))))))))
(defun validate-lisp-syntax (code-string)
"Verifies that a string of Lisp code is syntactically valid.
Does NOT execute the code. Returns (values boolean error-message)."
(handler-case
(let ((*read-eval* nil))
(with-input-from-string (stream (format nil "(progn ~a)" code-string))
(loop for form = (read stream nil :eof)
until (eq form :eof))
(values t nil)))
(error (c)
(values nil (format nil "~a" c)))))

64
src/symbolic.lisp Normal file
View File

@@ -0,0 +1,64 @@
(in-package :org-agent)
;;; ============================================================================
;;; System 2: The Symbolic Gatekeeper
;;; ============================================================================
;;; This module implements the 'Executive Function' of the kernel.
;;; System 2 is responsible for 'Deterministic Reasoning'—applying strict rules,
;;; safety constraints, and logical checks to verify neural proposals.
;;; It is slow but reliable, and it has the absolute power to overrule System 1.
(defun decide (proposed-action context)
"The System 2 Deciding Stage.
It subjects the proposal from System 1 to a battery of symbolic tests.
1. It applies Global Safety Heuristics (e.g., preventing shell execution).
2. It delegates domain-specific validation to the active skill's verify-fn.
Returns an approved action intent, or a safe fallback (like a log message)."
(let ((active-skill (find-triggered-skill context)))
(if active-skill
(let ((symbolic-gate (skill-symbolic-fn active-skill)))
;; --- GLOBAL SAFETY HEURISTIC #1: Block Shell Execution ---
;; We never allow the LLM to execute raw shell commands via Emacs eval.
(when (and proposed-action (listp proposed-action)
(eq (getf proposed-action :type) :REQUEST)
(eq (getf (getf proposed-action :payload) :action) :eval))
(let ((code (getf (getf proposed-action :payload) :code)))
(when (and code (search "shell-command" code))
(kernel-log "SYSTEM 2 [GLOBAL]: Security violation blocked (shell-command attempt).~%")
(return-from decide '(:type :LOG :payload (:text "Blocked by Global Safety Heuristic"))))))
;; --- SKILL-SPECIFIC VALIDATION ---
;; If the skill provides a specific System 2 verification function, run it.
(if symbolic-gate
(let ((decision (funcall symbolic-gate proposed-action context)))
(if decision
(progn
(kernel-log "SYSTEM 2: Verified by skill '~a'. Proceeding to Act.~%" (skill-name active-skill))
decision)
(progn
;; If the skill's logic returns NIL, the proposal is rejected.
(kernel-log "SYSTEM 2: REJECTED by skill '~a'. Logic violation detected.~%" (skill-name active-skill))
'(:type :LOG :payload (:text "Action rejected by System 2 skill heuristics")))))
;; If the skill has no specific symbolic logic, we allow the proposal to pass.
(progn
(kernel-log "SYSTEM 2: Verified (Implicitly safe for skill '~a').~%" (skill-name active-skill))
proposed-action)))
;; If no skill is active, we return NIL (nothing to decide).
nil)))
(defun list-objects-with-attribute (attr-key attr-val)
"Helper: Returns objects from the symbolic store where ATTR-KEY matches ATTR-VAL.
Used by skills to perform relational checks (e.g., searching for active TODOs)."
(let ((results nil))
(maphash (lambda (id obj)
(declare (ignore id))
(when (equal (getf (org-object-attributes obj) attr-key) attr-val)
(push obj results)))
*object-store*)
results))

View File

@@ -0,0 +1,85 @@
(defpackage :org-agent-cognitive-tests
(:use :cl :fiveam :org-agent))
(in-package :org-agent-cognitive-tests)
(def-suite cognitive-suite
:description "Verification of the Perceive-Think-Decide-Act loop.")
(in-suite cognitive-suite)
(defun setup-mock-skills ()
"Register mock skills for testing."
(clrhash org-agent::*skills-registry*)
(org-agent::defskill :mock-refactor
:priority 100
:trigger (lambda (ctx) (eq (getf (getf ctx :payload) :command) :organize-subtree))
:neuro (lambda (ctx) "Mock neuro prompt")
:symbolic (lambda (action ctx)
`(:type :REQUEST :id 123
:payload (:action :refactor-subtree
:target-id nil
:properties (("ID" . "node-123"))))))
(org-agent::defskill :mock-safety
:priority 50
:trigger (lambda (ctx) t) ; always triggers
:neuro (lambda (ctx) "Mock neuro")
:symbolic (lambda (action ctx) nil))) ; rejects everything
(test test-perceive-ingestion
"Perceive should update the object store and return context."
(clrhash org-agent::*object-store*)
(let* ((stimulus '(:type :EVENT :payload (:sensor :buffer-update :ast (:type :HEADLINE :properties (:ID "test-node" :TITLE "Test") :contents nil))))
(context (perceive stimulus)))
(is (equal stimulus context))
(is (not (null (gethash "test-node" org-agent::*object-store*))))))
(test test-decide-safety-gate
"Decide should block unsafe LLM proposals (System 2 bouncer)."
(setup-mock-skills)
(let ((context '(:type :EVENT :payload (:sensor :buffer-update)))
(unsafe-proposal '(:type :REQUEST :payload (:action :eval :code "(shell-command \"rm -rf /\")"))))
(let ((decision (decide unsafe-proposal context)))
(is (eq :LOG (getf decision :type)))
(is (search "Blocked by Global Safety Heuristic" (getf (getf decision :payload) :text))))))
(test test-decide-deterministic-override
"Decide should pre-empt LLM for deterministic tasks like missing IDs."
(setup-mock-skills)
(let* ((ast '(:type :HEADLINE :properties (:TITLE "No ID") :contents nil))
(context `(:type :EVENT :payload (:sensor :user-command :command :organize-subtree :ast ,ast)))
(dummy-proposal '(:type :LOG :payload (:text "I am thinking..."))))
(let ((decision (decide dummy-proposal context)))
(is (eq :REQUEST (getf decision :type)))
(is (eq :refactor-subtree (getf (getf decision :payload) :action)))
(is (not (null (assoc "ID" (getf (getf decision :payload) :properties) :test #'string=)))))))
(test test-env-loading
"Verify that environment variables are accessible (Phase 2 gating)."
(is (not (null (uiop:getenv "LLM_ENDPOINT"))))
(is (stringp (org-agent::get-env "MEMEX_USER"))))
(test test-path-resolution
"Verify that context-resolve-path expands environment variables."
(let ((path "$MEMEX_USER/test"))
(is (search "Amr/test" (context-resolve-path path)))))
(test test-skill-dependencies
"Verify that resolve-skill-dependencies correctly flattens the graph."
(setup-mock-skills)
;; Add a dependent skill
(org-agent::defskill :mock-dependent
:priority 10
:dependencies '("mock-safety")
:trigger (lambda (ctx) nil)
:neuro nil
:symbolic nil)
(let ((deps (org-agent::resolve-skill-dependencies "mock-dependent")))
(is (member "mock-safety" deps :test #'string-equal))
(is (member "mock-dependent" deps :test #'string-equal))))
(test test-log-buffering
"Verify that kernel-log correctly populates the system logs."
(kernel-log "PSF TEST LOG")
(let ((logs (context-get-system-logs 5)))
(is (cl:some (lambda (line) (search "PSF TEST LOG" line)) logs))))

36
tests/oacp-tests.lisp Normal file
View File

@@ -0,0 +1,36 @@
(defpackage :org-agent-tests
(:use :cl :fiveam :org-agent))
(in-package :org-agent-tests)
(def-suite oacp-suite
:description "Test suite for org-agent Communication Protocol (OACP)")
(in-suite oacp-suite)
(test test-framing
"Verify that messages are correctly prefixed with a 6-character hex length."
(let ((msg "(:type :EVENT :payload (:action :handshake))"))
;; As the Analyst, I expect a function 'frame-message' to exist
(is (string= "00002c(:type :EVENT :payload (:action :handshake))"
(org-agent:frame-message msg)))))
(test test-parse-message
"Verify that incoming framed strings are parsed into Lisp plists."
(let ((framed "00002c(:type :EVENT :payload (:action :handshake))"))
(is (equal '(:type :EVENT :payload (:action :handshake))
(org-agent:parse-message framed)))))
(test test-hello-handshake
"Verify the structure of the HELLO handshake message."
(let ((hello (org-agent:make-hello-message "0.1.0")))
(is (eq :EVENT (getf hello :type)))
(is (eq :handshake (getf (getf hello :payload) :action)))
(is (string= "0.1.0" (getf (getf hello :payload) :version)))))
(test test-find-missing-id
"Verify that the daemon can find a headline missing an ID."
(let* ((ast '(:type :org-data :contents
((:type :HEADLINE :properties (:TITLE "No ID Here") :contents nil)
(:type :HEADLINE :properties (:ID "exists" :TITLE "Has ID") :contents nil))))
(found (org-agent::find-headline-missing-id ast)))
(is (not (null found)))
(is (string= "No ID Here" (getf (getf found :properties) :TITLE)))))

73
tests/org-agent-test.el Normal file
View File

@@ -0,0 +1,73 @@
;;; org-agent-test.el --- Tests for the org-agent Emacs stub
(require 'ert)
(require 'cl-lib)
(require 'org-agent "/home/amr/.openclaw/workspace/memex/5_projects/org-agent/src/org-agent.el")
(ert-deftest test-org-agent-framing ()
"Verify that org-agent-send correctly frames a plist."
(let ((captured-framed nil))
(cl-letf (((symbol-function 'process-send-string)
(lambda (proc string) (setq captured-framed string)))
((symbol-function 'process-live-p) (lambda (proc) t))
(org-agent--process t))
(org-agent-send '(:type :EVENT :id 1))
(should (string= "000014(:type :EVENT :id 1)" captured-framed)))))
(ert-deftest test-org-agent-parsing ()
"Verify that the filter correctly parses OACP framed messages."
(let ((mock-buffer (generate-new-buffer " *org-agent-test*"))
(received-plist nil))
(cl-letf (((symbol-function 'org-agent--handle-message)
(lambda (proc plist) (setq received-plist plist))))
(with-current-buffer mock-buffer
(insert "000014(:type :EVENT :id 1)")
(org-agent--process-buffer mock-buffer)
(should (equal '(:type :EVENT :id 1) received-plist))
(should (= (buffer-size) 0))))))
(ert-deftest test-org-agent-actuator-message ()
"Verify that the :message actuator works."
(let ((org-agent--process nil)
(captured-response nil))
(cl-letf (((symbol-function 'org-agent-send)
(lambda (plist) (setq captured-response plist))))
(org-agent--execute-request nil 101 '(:action :message :text "Hello from Daemon"))
;; Check that we sent a success response back
(should (eq :RESPONSE (plist-get captured-response :type)))
(should (eq :success (plist-get (plist-get captured-response :payload) :status))))))
(ert-deftest test-org-agent-run-command ()
"Verify that org-agent-run-command sends the correct event."
(let ((captured-framed nil))
(cl-letf (((symbol-function 'process-send-string)
(lambda (proc string) (setq captured-framed string)))
((symbol-function 'process-live-p) (lambda (proc) t))
(org-agent--process t))
(org-agent-run-command :test-cmd)
(should (string-match-p ":sensor :user-command" captured-framed))
(should (string-match-p ":command :test-cmd" captured-framed)))))
(ert-deftest test-org-agent-ast-cleaning ()
"Verify that org-agent--clean-element produces a pure plist."
(let* ((org-text "* Hello\nWorld")
(ast (with-temp-buffer
(org-mode)
(insert org-text)
(org-element-parse-buffer)))
(cleaned (org-agent--clean-element ast)))
(should (plist-get cleaned :type))
(should (eq 'org-data (plist-get cleaned :type)))
;; Check that children exist
(should (plist-get (car (plist-get cleaned :contents)) :type))
;; Check that we didn't leak buffer objects
(should-not (plist-get (plist-get cleaned :properties) :buffer))))
(ert-deftest test-org-agent-actuator-eval ()
"Verify that the :eval actuator can execute elisp."
(let ((org-agent--process nil)
(captured-response nil))
(cl-letf (((symbol-function 'org-agent-send)
(lambda (plist) (setq captured-response plist))))
(org-agent--execute-request nil 102 '(:action :eval :code "(+ 1 2)"))
(should (equal "3" (plist-get (plist-get captured-response :payload) :result))))))