From 6abc306c7fb280d46b67941059bf6637fa1e4ac0 Mon Sep 17 00:00:00 2001 From: Amr Gharbeia Date: Mon, 27 Apr 2026 13:10:56 -0400 Subject: [PATCH] build: purge obsolete rca docs and redundant installer scripts - Deleted docs/rca/ as they were early v0.1.0 development artifacts. - Deleted minimal.asd. - Deleted scripts/onboard-baremetal.sh (subsumed by opencortex.sh). - Moved scripts/browser-bridge.py to skills/assets/ for upcoming Web Research skill. - Removed scripts/ directory. --- docs/rca/rca-boot-sequence.org | 42 -------------- docs/rca/rca-bouncer.org | 33 ----------- docs/rca/rca-formal-verification.org | 36 ------------ docs/rca/rca-gateway-matrix.org | 40 ------------- docs/rca/rca-gateway-signal.org | 33 ----------- docs/rca/rca-gateway-telegram.org | 43 -------------- docs/rca/rca-infrastructure-docker.org | 30 ---------- docs/rca/rca-lisp-repair-async.org | 33 ----------- docs/rca/rca-playwright-bridge.org | 39 ------------- docs/rca/rca-provider-verification.org | 40 ------------- docs/rca/rca-self-fix-loop.org | 40 ------------- docs/rca/rca-shell-hardening.org | 33 ----------- docs/rca/rca-task-orchestrator.org | 48 ---------------- minimal.asd | 4 -- scripts/onboard-baremetal.sh | 59 -------------------- {scripts => skills/assets}/browser-bridge.py | 0 16 files changed, 553 deletions(-) delete mode 100644 docs/rca/rca-boot-sequence.org delete mode 100644 docs/rca/rca-bouncer.org delete mode 100644 docs/rca/rca-formal-verification.org delete mode 100644 docs/rca/rca-gateway-matrix.org delete mode 100644 docs/rca/rca-gateway-signal.org delete mode 100644 docs/rca/rca-gateway-telegram.org delete mode 100644 docs/rca/rca-infrastructure-docker.org delete mode 100644 docs/rca/rca-lisp-repair-async.org delete mode 100644 docs/rca/rca-playwright-bridge.org delete mode 100644 docs/rca/rca-provider-verification.org delete mode 100644 docs/rca/rca-self-fix-loop.org delete mode 100644 docs/rca/rca-shell-hardening.org delete mode 100644 docs/rca/rca-task-orchestrator.org delete mode 100644 minimal.asd delete mode 100755 scripts/onboard-baremetal.sh rename {scripts => skills/assets}/browser-bridge.py (100%) diff --git a/docs/rca/rca-boot-sequence.org b/docs/rca/rca-boot-sequence.org deleted file mode 100644 index e18b217..0000000 --- a/docs/rca/rca-boot-sequence.org +++ /dev/null @@ -1,42 +0,0 @@ -#+TITLE: Root Cause Analysis: Micro-Loader & Deterministic Boot Sequence -#+DATE: 2026-04-11 -#+FILETAGS: :rca:boot:loader:topological-sort:autonomy: - -* Executive Summary -Refactored the arbitrary skill loading mechanism into a robust **Micro-Loader**. The system now calculates a deterministic boot sequence based on `#+DEPENDS_ON:` tags and protects the harness from malformed or hanging skills via package-based jailing and execution timeouts. - -* 1. Issue: Fragile Load Order -** Symptoms -Skills that depended on functions or variables from other skills would randomly fail to load depending on the filesystem's directory traversal order. -** Root Cause -`initialize-all-skills` used a simple `dolist` over `uiop:directory-files`, which has no semantic awareness of inter-skill dependencies. -** Resolution -1. **Metadata Scanning:** Implemented `parse-skill-metadata` to extract `:ID:` and `#+DEPENDS_ON:` without executing code. -2. **Topological Sort:** Implemented a DFS-based `topological-sort-skills` to guarantee that prerequisites are loaded before their dependents. -3. **Circular Detection:** Added explicit detection and error reporting for circular dependency loops. - -* 2. Issue: Shared State Corruption (Brain Rot) -** Symptoms -Variables or functions with the same name in different skills would silently overwrite each other, causing unpredictable behavior. -** Root Cause -All skills were being evaluated directly into the `opencortex` package. -** Resolution -**Package-Based Jailing:** Each skill is now evaluated within its own dedicated, shadowed package (e.g., `OPENCORTEX.SKILLS.ORG-SKILL-CHAT`). This ensures logical isolation while still allowing access to kernel exports. - -* 3. Issue: Boot Stall (The Hanging Skill) -** Symptoms -A single skill with an infinite loop or heavy synchronous initialization could hang the entire agent during startup. -** Root Cause -Skill loading was strictly synchronous and blocking on the main thread. -** Resolution -**Execution Timeouts:** Implemented `load-skill-with-timeout`, which wraps the loader in a monitored thread. If a skill takes longer than 5 seconds to initialize, the loader terminates the thread, jails the failure, and continues with the rest of the boot sequence. - -* 4. opencortex Mandate Alignment -** Evolutionary Kernel -The boot sequence is now a verifiable, mathematical process rather than a side-effect of filesystem organization. -** Literate Granularity -The `org-skill-skills.org` source was refactored into a strictly granular "one definition per block" format. - -* 5. Permanent Learnings -- **Reverse Topological Order:** Remember that a DFS-based sort with `push` needs an `nreverse` to place dependencies at the front of the list. -- **Path Portability:** Use `uiop:getcwd` instead of `pwd` for more reliable path resolution across different Lisp implementations and OSes. diff --git a/docs/rca/rca-bouncer.org b/docs/rca/rca-bouncer.org deleted file mode 100644 index 6bca72a..0000000 --- a/docs/rca/rca-bouncer.org +++ /dev/null @@ -1,33 +0,0 @@ -#+TITLE: Root Cause Analysis: Deterministic Engine Bouncer & Authorization Gate -#+DATE: 2026-04-11 -#+FILETAGS: :rca:bouncer:authorization:autonomy:security: - -* Executive Summary -Implemented the "Planning Mode" Bouncer to intercept high-risk Probabilistic Engine proposals (e.g., shell commands, Lisp evaluation). The system now forces these actions into an asynchronous "Flight Plan" Org node for manual Autonomous approval, fulfilling the "everything is a node" and high-integrity mandates. - -* 1. Issue: Automated High-Risk Execution -** Symptoms -Probabilistic Engine proposals involving `shell` or `eval` were executed immediately upon passing the `decide` gate's safety harness. This lacked human-in-the-loop oversight for irreversible or complex operations. -** Root Cause -Architecture gap. The system lacked an authorization state between "Safe" and "Executed". -** Resolution -1. **Interceptor:** Added `bouncer-check` to `deterministic.lisp`. It flags high-risk actions that lack the `:approved t` property. -2. **Asynchronous Event:** If flagged, the harness emits an `:approval-required` event. -3. **Flight Plan Skill:** Created `org-skill-bouncer.org` to: - - Catch the event and create a serialized Org node with state `PLAN`. - - Monitor the Memory for `APPROVED` states. - - Re-inject approved actions with the `:approved t` bypass flag. - -* 2. Design Decision: Org-native Approval -** Requirement -Align with "Homoiconic Memory" and "Lisp Machine Autonomousty". -** Selected Path -State-Based Approval (Org-native). -- *Pros:* Auditable, asynchronous, utilizes existing Org-mode workflows. -- *Cons:* Slightly more latency than an interactive prompt. -** Alignment -Ensures that the agent's "Flight Plans" are first-class citizens in the Memex, allowing the Autonomous to review and approve them using standard GTD tools. - -* 3. Permanent Learnings -- **Serial Bypass:** Always include a specific bypass flag (e.g., `:approved t`) when re-injecting intercepted actions to prevent infinite interception loops. -- **Heartbeat Listeners:** Periodic scanning of the Memory for state transitions is an effective way to implement asynchronous authorization gates without blocking the harness. diff --git a/docs/rca/rca-formal-verification.org b/docs/rca/rca-formal-verification.org deleted file mode 100644 index 919250f..0000000 --- a/docs/rca/rca-formal-verification.org +++ /dev/null @@ -1,36 +0,0 @@ -#+TITLE: Root Cause Analysis: Lisp-Native Formal Verification Gate -#+DATE: 2026-04-11 -#+FILETAGS: :rca:security:formal-verification:autonomy: - -* Executive Summary -Implemented a Lisp-Native Deterministic Prover to replace heuristic whitelisting with formal security invariants. This ensures that every high-impact action (shell, file I/O) is mathematically proven safe against the Autonomous's core mandates. - -* 1. Architectural Shift: Native vs. External -** Issue -The initial draft suggested using `Z3`, an external SMT solver. However, `Z3` was not available in the environment and would add significant complexity/bloat to the Docker image. -** Resolution -Leveraged Common Lisp's inherent strength in symbol manipulation to build a **Lisp-Native Prover**. Invariants are defined as high-order predicates that operate on the structure of proposed actions. This provides a self-contained, high-performance verification layer. - -* 2. Issue: Dependency Fragility -** Symptoms -System failed to load with `Package STR does not exist`. -** Root Cause -Incorrect assumption about the Quicklisp system name vs. the package name. The library is `cl-str` but the Quicklisp system is `str` and the package is `str`. -** Resolution -1. Updated `opencortex.asd` to depend on `:str`. -2. Updated all source code and literate notes to use the `str:` prefix. -3. Verified via explicit `ql:quickload` in the test runner. - -* 3. Formal Invariants Implemented -- **Path Confinement:** Deterministically proves that any file operation or absolute path in a shell command is strictly within the `/home/user/memex` root. -- **No Network Exfiltration:** Prevents the shell from invoking common exfiltration tools (`nc`, `ssh`, etc.) by inspecting the parsed command structure. - -* 4. opencortex Mandate Alignment -** Soundness over Heuristics -By moving to formal invariants, we have moved from "blacklisting bad things" to "proving safety." Any action that cannot be proven to satisfy all invariants is denied by default. -** Literate Granularity -The `org-skill-formal-verification.org` file follows the "one definition per block" mandate, ensuring that the logic of each invariant is individually documented and verifiable. - -* 5. Permanent Learnings -- **Tooling Independence:** Whenever possible, prefer native Lisp logic over external binaries for core security gates to reduce the attack surface and deployment complexity. -- **Environment Consistency:** Always use `(setf (uiop:getenv ...) ...)` for portable environment manipulation in tests. diff --git a/docs/rca/rca-gateway-matrix.org b/docs/rca/rca-gateway-matrix.org deleted file mode 100644 index ccdb7bd..0000000 --- a/docs/rca/rca-gateway-matrix.org +++ /dev/null @@ -1,40 +0,0 @@ -#+TITLE: Root Cause Analysis: Matrix Gateway & Communication Track Completion -#+DATE: 2026-04-11 -#+FILETAGS: :rca:gateway:matrix:chat:autonomy: - -* Executive Summary -Successfully implemented the third and final external communication channel (Matrix) for OpenCortex v1.0. Resolved integration issues related to case-sensitivity in JSON keys and strict header requirements in `dexador`. - -* 1. Issue: Symbol Casing in JSON Keys -** Symptoms -The `TEST-MATRIX-INBOUND-NORMALIZATION` test failed because `room-id` was being extracted as `"!ROOM:HS.ORG"` (uppercase) instead of `"!room:hs.org"`. -** Root Cause -Common Lisp's default reader converts symbol names to uppercase. When `(string car-of-alist)` was called on a symbol generated by `cl-json`, it produced an uppercase string. -** Resolution -Updated the implementation to use `(string-downcase (string ...))` for room IDs and other case-sensitive Matrix identifiers. - -* 2. Issue: Since Token Extraction Failure -** Symptoms -The sync loop failed to update the `*matrix-since-token*`, causing duplicate message processing risk. -** Root Cause -Anticipating `:next-batch` but receiving `:next--batch` (or vice versa) due to inconsistent `cl-json` behavior across different environments or structures. -** Resolution -Implemented a robust `(or (cdr (assoc :next-batch json)) (cdr (assoc :next--batch json)))` lookup to handle both hyphenation styles. - -* 3. Issue: Type Error in Authorization Headers -** Symptoms -`dex:put` crashed with a `TYPE-ERROR`. -** Root Cause -I was passing a single string or an incorrectly nested list where `dexador` expected a strict alist of header pairs `(("Key" . "Value") ...)`. -** Resolution -Standardized all gateway HTTP calls to use proper alist nesting for headers. - -* 4. Completion: Communication Track -With Telegram, Signal, and Matrix gateways now verified and passing tests, the OpenCortex has achieved full multi-channel parity. -- **Telegram:** Polling via Bot API. -- **Signal:** Wrapping `signal-cli`. -- **Matrix:** Polling via `/sync` Client API. - -* 5. Permanent Learnings -- **Case Sensitivity:** Matrix IDs (rooms, users) are case-sensitive; Lisp symbols are not. Always force downcasing or use strings for storage. -- **Header Alists:** Always use dotted pairs `("Key" . "Value")` for `dexador` headers. diff --git a/docs/rca/rca-gateway-signal.org b/docs/rca/rca-gateway-signal.org deleted file mode 100644 index 5de7db7..0000000 --- a/docs/rca/rca-gateway-signal.org +++ /dev/null @@ -1,33 +0,0 @@ -#+TITLE: Root Cause Analysis: Signal Gateway & Multi-Channel Chat -#+DATE: 2026-04-11 -#+FILETAGS: :rca:gateway:signal:chat:autonomy: - -* Executive Summary -Successfully implemented the second external communication channel (Signal) using `signal-cli`. Further hardened the multi-channel chat logic and resolved JSON mapping discrepancies between Common Lisp and external CLI outputs. - -* 1. Issue: JSON Key Mapping Mismatch -** Symptoms -The `TEST-SIGNAL-INBOUND-NORMALIZATION` test failed despite the mock JSON appearing correct. -** Root Cause -`cl-json` default behavior for decoding. It converts camelCase keys from JSON (e.g., `dataMessage`) into kebab-case keywords in Lisp (e.g., `:DATA-MESSAGE`). I had incorrectly anticipated `:DATA--MESSAGE` or `:DATA_MESSAGE`. -** Resolution -1. **Diagnostic:** Added debug output to the test suite to inspect the exact plist structure returned by `cl-json`. -2. **Correction:** Updated both the implementation and the literate note to use the correct `:DATA-MESSAGE` and `:SOURCE` keywords. - -* 2. Implementation: Signal-CLI Wrapper -** Strategy -Unlike Telegram's HTTP API, Signal requires a local binary (`signal-cli`). -- **Sensor:** Uses `uiop:run-program` with `receive --json` in a polling loop (5s interval). -- **Actuator:** Uses `uiop:run-program` with `send -m `. -** Security -The system uses the pre-configured Signal account `+13322690326` discovered in the user's memex. - -* 3. Alignment with opencortex Mandates -** Literate Granularity -Strictly adhered to the "one definition per block" mandate throughout the new `org-skill-gateway-signal.org` file. -** Verification -The `gateway-signal-suite` (10 checks) provides full coverage for inbound parsing and outbound command generation. - -* 4. Permanent Learnings -- **JSON Semantics:** Always verify the specific keyword transformation rules of the JSON library when dealing with external CLI outputs. -- **Process Robustness:** `uiop:run-program` is the reliable standard for CLI-based gateways in SBCL. diff --git a/docs/rca/rca-gateway-telegram.org b/docs/rca/rca-gateway-telegram.org deleted file mode 100644 index 6de12ed..0000000 --- a/docs/rca/rca-gateway-telegram.org +++ /dev/null @@ -1,43 +0,0 @@ -#+TITLE: Root Cause Analysis: Telegram Gateway & Channel-Aware Chat -#+DATE: 2026-04-11 -#+FILETAGS: :rca:gateway:telegram:chat:autonomy: - -* Executive Summary -Successfully implemented the first external communication channel (Telegram) and decoupled the Chat Agent from its Emacs-centric roots. Resolved significant load-order and dependency issues identified during integration. - -* 1. Issue: Undefined Foundational Functions -** Symptoms -During compilation, `gateway-telegram.lisp` failed with `UNDEFINED-FUNCTION` for `register-actuator` and `harness-log`. -** Root Cause -Poorly scoped foundational functions. These were defined in `core.lisp` (the loop orchestrator), which was loaded *after* the gateways in `opencortex.asd`. This created a "Circular Intention" where the gateways needed the harness to exist before the harness could load the gateways. -** Resolution -1. **Relocation:** Moved `*actuator-registry*` and `register-actuator` to `communication.lisp` (the foundation). -2. **Reordering:** Adjusted `opencortex.asd` to load `core.lisp` (containing the stimulus loop) immediately after the deterministic gates but before the physical sensors (gateways). - -* 2. Issue: Hardcoded Chat UI -** Symptoms -The `Chat Agent` could only respond via Emacs buffer insertion, rendering it useless for external channels like Telegram. -** Root Cause -Architectural myopia. The original chat skill assumed the user was always in front of Emacs. -** Resolution -Refactored `org-skill-chat` to be **Channel-Aware**: -- It now extracts `:channel` and `:chat-id` from the inbound stimulus. -- It dynamically generates the Probabilistic Engine mandate, instructing the LLM to use the appropriate `:target` (e.g., `:telegram`) based on the conversation context. - -* 3. Side-Issue: UIOP Portability -** Symptoms -Tests failed with `Symbol "SETENV" not found in the UIOP/DRIVER package`. -** Root Cause -Misinterpretation of the `UIOP` API. `setenv` is not a standard export; the portable way is using `(setf (uiop:getenv ...) ...)`. -** Resolution -Updated all test environment setup to use the `setf` accessor. - -* 4. opencortex Mandate Alignment -** Autonomous Boundary -By moving the Telegram API logic to a user-space skill and communicating with the core via standard stimuli, we have respected the microkernel boundary. -** Homoiconic Memory -All Telegram interactions are now logged as `:chat-message` events, ensuring the agent's history is unified regardless of the platform. - -* 5. Permanent Learnings -- **Foundation First:** Registries and logging macros must reside in the most foundational layers (`protocol` or `package`) to avoid load-order fragility. -- **Instruct the Actuator:** When adding new channels, always update the Chat Agent's neural prompt so it knows how to "speak" back through the new interface. diff --git a/docs/rca/rca-infrastructure-docker.org b/docs/rca/rca-infrastructure-docker.org deleted file mode 100644 index 82bcdbf..0000000 --- a/docs/rca/rca-infrastructure-docker.org +++ /dev/null @@ -1,30 +0,0 @@ -#+TITLE: Root Cause Analysis: Containerized Infrastructure (Docker) -#+DATE: 2026-04-11 -#+FILETAGS: :rca:docker:deployment:infrastructure:autonomy: - -* Executive Summary -Standardized the `opencortex` execution environment by creating a production-grade Docker infrastructure. This ensures that all system dependencies, including the Lisp runtime and external binaries like `signal-cli`, are locked down and portable. - -* 1. Architectural Intent: The "Clean Room" Model -** Problem -The `opencortex` was relying on host-local binaries (`sbcl`, `signal-cli`) and manually configured Quicklisp dists. This made deployment to other environments (e.g., a VPS or a Autonomous Home Server) fragile and prone to version drift. -** Solution -1. **Dockerfile:** Created a multi-step build process that installs Debian Bookworm, SBCL, Java, and `signal-cli 0.14.0`. -2. **Pre-Caching:** The build process triggers a `ql:quickload` of the `:opencortex` system, ensuring all Lisp dependencies are pre-downloaded and stored in the image layer, drastically reducing startup time. -3. **Compose Orchestration:** Standardized the runtime via `docker-compose.yml`, which handles volume mounting of the user's `memex` directory and injection of `.env` secrets. - -* 2. Volume Mapping & Persistence -** Strategy -To maintain the "Autonomous" mandate, the agent's code is isolated, but its memory (the `memex`) remains on the host. -- **Mapping:** `../..` (host) -> `/memex` (container). -- **State:** Created a named Docker volume `signal-state` to ensure that `signal-cli` identities and cryptographic keys survive container restarts and image updates. - -* 3. Alignment with opencortex Mandates -** Evolutionary Completion -By moving to Docker, we have achieved "Evolutionary Completion" for the deployment track. The system is no longer a collection of scripts; it is a deployable appliance. -** Documentation -A new `Deployment Guide` was added to `docs/deployment.org` to ensure standard operating procedures are preserved. - -* 4. Permanent Learnings -- **Lisp Build Layers:** Always push the system to the ASDF registry and quickload during Docker build to bake dependencies into the image. -- **Compose Locality:** Placing the `docker-compose.yml` inside the `projects/opencortex/` folder keeps infrastructure code close to the implementation logic. diff --git a/docs/rca/rca-lisp-repair-async.org b/docs/rca/rca-lisp-repair-async.org deleted file mode 100644 index 8a919f9..0000000 --- a/docs/rca/rca-lisp-repair-async.org +++ /dev/null @@ -1,33 +0,0 @@ -#+TITLE: Root Cause Analysis: Asynchronous Lisp Repair Syntax Gate -#+DATE: 2026-04-11 -#+FILETAGS: :rca:lisp:repair:decoupling:architecture:autonomy: - -* Executive Summary -Reimplemented the `org-skill-lisp-repair` to align with the "Autonomous Boundary" mandate. The previously synchronous, core-blocking repair logic has been replaced with an asynchronous, event-driven architecture using the Reactive Signal Pipeline. - -* 1. Issue: Core Bloat & Synchronous Coupling -** Symptoms -The initial implementation of the Lisp Repair gate placed a `handler-case` and a dynamic function call (`repair-lisp-syntax`) directly inside the core `think` function (`probabilistic.lisp`). This forced the core to wait for repairs and made it "aware" of specific repair logic. -** Root Cause -Architectural shortcutting. By placing repair logic in the core execution path, we violated the microkernel principle which mandates that the core should be a "dumb" signal processor. -** Resolution -1. **Refactored Core:** `think` now only emits a `:syntax-error` stimulus if parsing fails. It no longer attempts to repair. -2. **Asynchronous Skill:** `skill-lisp-repair` now triggers on the `:syntax-error` event. It performs the repair and returns the corrected action, which is then dispatched by the pipeline. - -* 2. Side-Issue: Nested Signal Payloads -** Symptoms -`TYPE-ERROR` during testing when extracting the broken code from the stimulus. -** Root Cause -Mismatched expectations of signal nesting. The skill expected the code at `(getf context :payload)`, but in the `decide-gate`, `context` is the full signal, and the error details were nested inside the `:candidate` field of that signal. -** Resolution -Updated the deterministic logic to correctly traverse the nested signal structure: `(getf (getf context :candidate) :payload)`. - -* 3. opencortex Mandate Alignment -** Autonomous Boundary -The core is now strictly a parser. Repair is an optional, user-space service. -** Reactive Signal Pipeline -Leveraged the pipeline's ability to re-inject `EVENT` signals to flatten the recursion of the repair loop. - -* 4. Permanent Learnings -- **Emit, Don't Call:** In a microkernel, if a non-fatal error occurs, always emit a signal rather than calling a recovery function. This allows the system to remain asynchronous and modular. -- **Signal Inspection:** When writing deterministic gates, always verify the exact shape of the `context` signal being passed by the harness to avoid nesting errors. diff --git a/docs/rca/rca-playwright-bridge.org b/docs/rca/rca-playwright-bridge.org deleted file mode 100644 index e36811b..0000000 --- a/docs/rca/rca-playwright-bridge.org +++ /dev/null @@ -1,39 +0,0 @@ -#+TITLE: Root Cause Analysis: Playwright-Python Bridge (High-Fidelity Browsing) -#+DATE: 2026-04-11 -#+FILETAGS: :rca:intelligence:browsing:automation:autonomy: - -* Executive Summary -Successfully implemented a high-fidelity browsing bridge using Playwright and Python. This allows the `opencortex` to interact with modern, JavaScript-rendered web applications that were previously inaccessible via simple HTTP clients. - -* 1. Architectural Strategy: The I/O Bridge -** Problem -Common Lisp lacks a mature, native Playwright implementation. Direct bindings are complex and fragile. -** Resolution -Implemented a **JSON-over-STDIO Bridge**. -- A standalone Python script (`browser-bridge.py`) manages the Playwright lifecycle and Chromium instance. -- The Lisp kernel communicates with this script using `uiop:run-program`, passing parameters via `stdin` and receiving structured results via `stdout`. This provides a stable, decoupled interface. - -* 2. Environment & Dependency Management -** Issue -Playwright requires a specific version of Chromium and several system-level libraries not present in the base Debian image. -** Resolution -Updated the `Dockerfile` to: -1. Install Python3, pip, and venv. -2. Create a virtual environment for isolated dependency management. -3. Install the `playwright` package and execute `playwright install --with-deps chromium` during the image build. This ensures the production container is ready for high-fidelity browsing immediately upon startup. - -* 3. Cognitive Tooling -Created the `:browser` cognitive tool, which exposes three primary capabilities to Probabilistic Engine: -- **Navigation:** Full JS rendering and waiting for network idle. -- **Extraction:** Targeted text retrieval via CSS selectors. -- **Vision:** Base64-encoded screenshot capture for future multimodal processing. - -* 4. opencortex Mandate Alignment -** Zero-Bloat (Managed) -While adding Playwright increases the image size, it is a "Complexity Earned" trade-off that dramatically expands the agent's capability frontier. -** Literate Granularity -The `org-skill-playwright.org` file strictly follows the "one definition per block" mandate. - -* 5. Permanent Learnings -- **Inter-Process JSON:** JSON is the ideal lingua franca for Lisp-Python bridges. -- **Path Portability:** Always use `uiop:native-namestring` when passing Lisp paths to external shell commands to ensure OS compatibility. diff --git a/docs/rca/rca-provider-verification.org b/docs/rca/rca-provider-verification.org deleted file mode 100644 index f1d8e77..0000000 --- a/docs/rca/rca-provider-verification.org +++ /dev/null @@ -1,40 +0,0 @@ -#+TITLE: Root Cause Analysis: Individual Provider Track Verification -#+DATE: 2026-04-11 -#+FILETAGS: :rca:providers:llm:testing:autonomy: - -* Executive Summary -Verified the unified LLM gateway implementation for all 6 individual provider tracks (Anthropic, Gemini, Groq, OpenAI, OpenRouter, Ollama). Identified and resolved critical parsing failures in the Gemini track and integration gaps in the system build definition. - -* 1. Issue: Fragile Response Parsing (Gemini) -** Symptoms -Gemini API responses were returning `NIL` content during mocked unit tests, despite the JSON structure being seemingly correct. -** Root Cause -Recursive `assoc` / `car` / `cdr` chains were hardcoded and brittle. Specifically, the Gemini extraction logic was incorrectly attempting to treat a single alist pair as a list of pairs, causing `assoc` to fail on the `:TEXT` key. -** Resolution -Implemented a robust `get-nested` helper function that safely traverses both nested objects (alists) and arrays (lists of alists). This normalized the extraction logic across all providers. - -* 2. Issue: Decoupled Build Configuration -** Symptoms -Provider logic was present in the codebase but inaccessible during tests and runtime. -** Root Cause -The `credentials-vault.lisp` and `llm-gateway.lisp` files (consolidated in a previous session) were never added to the `opencortex.asd` system definition. Furthermore, an incorrect loading order caused `UNDEFINED-FUNCTION` errors for `register-probabilistic-backend`. -** Resolution -1. Added both files to `opencortex.asd`. -2. Enforced strict loading order: `probabilistic` (defines registry) -> `credentials-vault` -> `llm-gateway` (uses registry). - -* 3. Issue: Credential Key Mismatch -** Symptoms -Gemini requests failed with "API Key missing" even when environment variables were set. -** Root Cause -`llm-gateway` requested secrets for the `:gemini-api` provider, but the `credentials-vault` fallback logic only recognized the `:gemini` keyword. -** Resolution -Updated `vault-get-secret` to map both `:gemini` and `:gemini-api` to the same `GEMINI_API_KEY` environment variable. - -* 4. opencortex Mandate Alignment -** Invariant Check -- *High-Integrity Memory:* All individual provider tracks are now backed by automated unit tests (`llm-gateway-tests.lisp`). -- *Literate Programming:* Updated `org-skill-llm-gateway.org` to reflect the improved `get-nested` utility. - -* 5. Permanent Learnings -- **Tooling vs Source:** Tangled `.lisp` files are not enough; always ensure new modules are registered in the `.asd` file to be part of the official kernel build. -- **Robustness over Brevity:** Use abstraction helpers like `get-nested` instead of deep `car/cdr` chains when dealing with external JSON structures that may have varying array/object nesting. diff --git a/docs/rca/rca-self-fix-loop.org b/docs/rca/rca-self-fix-loop.org deleted file mode 100644 index 3d61865..0000000 --- a/docs/rca/rca-self-fix-loop.org +++ /dev/null @@ -1,40 +0,0 @@ -#+TITLE: Root Cause Analysis: Autonomous Self-Fix Loop Verification -#+DATE: 2026-04-11 -#+FILETAGS: :rca:self-fix:autonomy:testing: - -* Executive Summary -Verified the autonomous repair capability of the `Self-Fix Agent`. The system successfully detected a deterministic type error in a secondary skill, initiated a repair request, and programmatically patched the source code via the `:repair-file` tool. - -* 1. Issue: Self-Fix Mechanism Verification -** Symptoms -Manual verification was required to prove that `org-skill-self-fix` could transition from "Thinking" about a bug to "Acting" on the file system. -** Root Cause -N/A (Deterministic test injection). -** Resolution -Created `self-fix-tests.lisp` which: -1. Generates `org-skill-broken-math.org` with a `(+ 1 "two")` bug. -2. Triggers the bug to produce a `PIPELINE CRASH`. -3. Injects a `:repair-request` stimulus. -4. Executes `self-fix-apply` to replace the bug with `(+ 1 2)`. -5. Verifies the file content and successful hot-reload. - -* 2. Side-Issue: ASDF Configuration Fragility -** Symptoms -Repeated `LOAD-SYSTEM-DEFINITION-ERROR` and "unmatched close parenthesis" errors during test integration. -** Root Cause -Complexity in the `:components` nesting of `opencortex.asd` led to repeated syntax errors when using automated editing tools. The deep nesting made manual paren counting prone to "off-by-one" errors. -** Resolution -Refactored `opencortex.asd` to use a **Flat Component Structure**. -- *Before:* `:components ((:module "src" :components (...)))` -- *After:* `:components ((:file "src/package") ...)` -This eliminates unnecessary nesting levels and drastically reduces the surface area for syntax errors. - -* 3. opencortex Mandate Alignment -** Invariant Check -- *Lisp Machine Autonomousty:* Verification utilized hot-reloading (`load-skill-from-org`) without restarting the SBCL image. -- *Literate Programming:* Updated `org-skill-self-fix.org` to match the finalized `self-fix.lisp` logic. -- *Institutional Memory:* This RCA documents the decision to flatten the `.asd` structure to prevent future "Parenthesis Hell" incidents. - -* 4. Permanent Learnings -- **Flatten Configuration:** Keep `defsystem` definitions as flat as possible. The overhead of `:module` blocks often outweighs their organizational benefit in a probabilistic-deterministic environment where agents frequently edit these files. -- **Mocking Probabilistic Engine:** For verifying *loop mechanics*, mocking LLM responses is essential to ensure test determinism, while integration tests can use live LLM calls. diff --git a/docs/rca/rca-shell-hardening.org b/docs/rca/rca-shell-hardening.org deleted file mode 100644 index b3dba70..0000000 --- a/docs/rca/rca-shell-hardening.org +++ /dev/null @@ -1,33 +0,0 @@ -#+TITLE: Root Cause Analysis: Shell Actuator Security Hardening -#+DATE: 2026-04-11 -#+FILETAGS: :rca:security:shell:injection:autonomy: - -* Executive Summary -During the formal verification of the `org-skill-shell-actuator`, a critical command injection vulnerability was identified and patched. The previous implementation relied on a naive whitelist check that could be bypassed using shell metacharacters. - -* 1. Issue: Command Injection Vulnerability -** Symptoms -Commands like `ls ; rm -rf /` were potentially executable if the first word (`ls`) was in the whitelist. -** Root Cause -The `execute-shell-safely` function only checked the first space-delimited word of the command string against the `*allowed-commands*` whitelist. Since `uiop:run-program` executes string-based commands via `/bin/sh -c`, the shell would process the entire string, including injected commands following metacharacters like `;`, `&`, or `|`. -** Resolution -1. **Metacharacter Blacklist:** Introduced `*shell-metacharacters*` containing dangerous shell symbols (`; & | > < $ \` \ !`). -2. **Strict Validation:** Updated `execute-shell-safely` to scan the *entire* command string for these characters before performing the whitelist check. -3. **Defense-in-Depth:** Any command containing a metacharacter is now rejected with a "Security Violation" error, even if the primary command is whitelisted. - -* 2. Side-Issue: Missing Package Context -** Symptoms -`UNDEFINED-FUNCTION EXECUTE-SHELL-SAFELY` during unit tests. -** Root Cause -`src/shell-logic.lisp` was missing an `(in-package :opencortex)` declaration, causing symbols to be defined in the default `COMMON-LISP-USER` package instead of the harness package. -** Resolution -Added the `in-package` header to `shell-logic.lisp`. - -* 3. opencortex Mandate Alignment -** Invariant Check -- *High-Integrity Memory:* The shell actuator is now formally verified with 4 new unit tests covering whitelist enforcement and injection blocking. -- *Literate Programming:* Updated `org-skill-shell-actuator.org` Phase A and Build sections to reflect the hardened logic. - -* 4. Permanent Learnings -- **Whole-String Validation:** Never assume that whitelisting the "head" of a command string is sufficient when passing that string to a shell. -- **Subshell Avoidance:** While the current fix blacklists metacharacters, future iterations should move toward passing command arguments as a Lisp list to `uiop:run-program`, bypassing the shell entirely. diff --git a/docs/rca/rca-task-orchestrator.org b/docs/rca/rca-task-orchestrator.org deleted file mode 100644 index 7334a3b..0000000 --- a/docs/rca/rca-task-orchestrator.org +++ /dev/null @@ -1,48 +0,0 @@ -#+TITLE: Root Cause Analysis: Consolidation VI - Task Orchestrator Implementation -#+DATE: 2026-04-11 -#+FILETAGS: :rca:orchestrator:consensus:integrity: - -* Executive Summary -The implementation of Consolidation VI (Task Orchestrator) aimed to introduce parallel multi-backend consensus, GTD task integrity, and delegation. During the build, a critical dependency failure was identified in the `lisp-validator` module. - -* 1. Issue: Undefined `SAFETY-HARNESS-VALIDATE` -** Symptoms -Existing `SAFETY-SUITE` tests failed with `#`. -** Root Cause -The function `lisp-validator-validate` was exported in `package.lisp` but never actually defined in `lisp-validator.lisp`. Only the internal recursive walker `lisp-validator-ast-walk` existed. This represents a "Hollow Export" bug where the interface was designed but the implementation was truncated or skipped in a previous session. -** Resolution -Defined `lisp-validator-validate` as a wrapper around `read-from-string` and `lisp-validator-ast-walk`. - -* 2. Design Decision: Deterministic Consensus -** Requirement -Multi-backend support to reduce hallucinations and increase reliability. -** Solution -Implemented `bt:make-thread` parallel queries in `ask-probabilistic`. -** Trade-off -Selected "Majority Rules" over "First-to-Finish". -- *Pros:* Higher accuracy, mathematically consistent. -- *Cons:* Slower (latency limited by the slowest provider). -** Invariant Alignment -Aligns with opencortex Mandate 4 (Radical Transparency) and Invariant 2 (Technical Mastery) by ensuring decisions are auditable and consistent across multiple brains. - -* 3. Design Decision: Task Integrity Gate -** Requirement -Prevent illegal GTD state transitions. -** Solution -Added `task-integrity-check` in `deterministic.lisp`. -** Invariant Alignment -Enforces the "High-Integrity Memory" mandate by ensuring the Org-mode AST remains semantically valid according to GTD rules (e.g., no orphaned active tasks). - -* 4. opencortex Mandate Violations during Session (Corrected) -** Violations -1. Editing without prior commit. -2. Direct `.lisp` edits vs Literate Org tangling. -3. Multi-function edits per block. -** Correction -1. Performed a retrospective commit. -2. Synchronized `probabilistic-deterministic.org` and `core.org` with source code. -3. Refactored the Markdown flight plan into an Org-mode flight plan. - -* 5. Permanent Learnings -- *Check Exports:* Always verify that symbols exported in `package.lisp` have a corresponding definition in the literate source. -- *Strict opencortex Mode:* Enable a pre-save hook or agent check to ensure all edits are performed within `#+begin_src` blocks in Literate Org files to avoid synchronization debt. diff --git a/minimal.asd b/minimal.asd deleted file mode 100644 index e883d8a..0000000 --- a/minimal.asd +++ /dev/null @@ -1,4 +0,0 @@ -(defsystem :opencortex-minimal - :name "opencortex-minimal" - :depends-on () - :components ((:file "harness/package"))) \ No newline at end of file diff --git a/scripts/onboard-baremetal.sh b/scripts/onboard-baremetal.sh deleted file mode 100755 index 91bc040..0000000 --- a/scripts/onboard-baremetal.sh +++ /dev/null @@ -1,59 +0,0 @@ -#!/bin/bash -# OpenCortex Final-Mile Installer -RED='\033[0;31m'; GREEN='\033[0;32m'; BLUE='\033[0;34m'; YELLOW='\033[0;33m'; NC='\033[0m' -echo -e "${BLUE}=== OpenCortex: Baremetal Power-User Setup ===${NC}" - -prompt_user() { - local prompt="$1" - local default="$2" - local var_name="$3" - local result="" - echo -n -e "${YELLOW}$prompt (default: $default): ${NC}" >&2 - if read -t 5 result; then :; else result="$default"; echo -e "${BLUE} [Auto-Selected: $default]${NC}" >&2; fi - val=${result:-$default} - eval "$var_name=\"$val\"" -} - -# 1. Dependencies -if ! command -v sbcl >/dev/null 2>&1; then - echo -e "${BLUE}Installing dependencies...${NC}" - sudo apt-get update && sudo apt-get install -y sbcl emacs git curl socat || true -fi - -# 2. Quicklisp -if [ ! -d "$HOME/quicklisp" ]; then - curl -O https://beta.quicklisp.org/quicklisp.lisp - sbcl --non-interactive --load quicklisp.lisp --eval "(quicklisp-quickstart:install)" --eval "(ql-util:without-prompting (ql:add-to-init-file))" - rm quicklisp.lisp -fi - -# 3. Tangling -echo -e "${BLUE}Tangling source files...${NC}" -mkdir -p src -for f in literate/*.org; do - echo " - Tangling $f" - emacs --batch --eval "(require 'org)" --eval "(org-babel-tangle-file \"$f\")" >/dev/null 2>&1 -done - -# 4. Config -if [ ! -f .env ]; then cp .env.example .env; fi -prompt_user "What is your name?" "User" "USER_NAME" -prompt_user "What shall we name your Assistant?" "OpenCortex" "AGENT_NAME" -prompt_user "Select provider (1:Gemini, 2:OpenRouter)" "1" "LLM_CHOICE" - -sed -i "s/MEMEX_USER=.*/MEMEX_USER=\"$USER_NAME\"/g" .env -sed -i "s/MEMEX_ASSISTANT=.*/MEMEX_ASSISTANT=\"$AGENT_NAME\"/g" .env - -# 5. Path Alignment -INSTALL_DIR="$(cd "$(dirname "$0")/.." && pwd)" -sed -i "s|MEMEX_DIR=.*|MEMEX_DIR=\"$(dirname "$INSTALL_DIR")\"|g" .env -sed -i "s|SKILLS_DIR=.*|SKILLS_DIR=\"$INSTALL_DIR/skills\"|g" .env - -mkdir -p "$HOME/.local/bin" -ln -sf "$INSTALL_DIR/opencortex.sh" "$HOME/.local/bin/opencortex" -echo -e "${GREEN}✓ Installed 'opencortex' command to ~/.local/bin${NC}" - -echo -e "\n${GREEN}==============================================${NC}" -echo -e " OpenCortex Installation Complete! " -echo -e "==============================================${NC}" -echo -e "To start: opencortex" diff --git a/scripts/browser-bridge.py b/skills/assets/browser-bridge.py similarity index 100% rename from scripts/browser-bridge.py rename to skills/assets/browser-bridge.py