Files
memex/pipeline-refactor.md

38 lines
2.3 KiB
Markdown

# Implementation Plan: Reactive Signal Pipeline Refactor
## Objective
Refactor the monolithic recursive `cognitive-loop` into a functional, gate-based `reactive-signal-pipeline`. This flattens the execution stack, enables multi-backend consensus, and improves observability according to Component VII of the evolutionary roadmap.
## Key Files & Context
- **Target:** `projects/org-agent/literate/core.org` (Source of `src/core.lisp`)
- **Architectural Reference:** `projects/org-agent/literate/evolution.org`
## Implementation Steps
### 1. Define the Signal Structure
Transition from raw plists to a structured `:SIGNAL` format that tracks state as it moves through the pipeline.
- Attributes: `:status`, `:payload`, `:context`, `:depth`, `:proposals`, `:reply-stream`.
### 2. Implement the Pipeline Gates
Extract existing logic from `cognitive-loop` into discrete functional gates:
- **`perceive-gate`**: Normalizes input, updates `*object-store*`.
- **`neuro-gate`**: Invokes System 1. Support for future parallel backend calls.
- **`consensus-gate`**: (New) Selects the best proposal from multiple backends (initially a pass-through for the single proposal).
- **`decide-gate`**: Invokes System 2 safety checks.
- **`dispatch-gate`**: Executes tools or physical actuators.
### 3. Implement the Pipeline Orchestrator (`process-signal`)
Create a function that moves a signal through the sequence of gates.
- **Flattening Recursion:** If `dispatch-gate` results in a tool output or error, it MUST NOT call the pipeline recursively. Instead, it returns a new `:SIGNAL` with `depth + 1`, which the orchestrator then re-injects into the top of the pipe via a loop or queue.
### 4. Refactor `inject-stimulus`
Update `inject-stimulus` to initialize a Signal and hand it to the `process-signal` pipeline instead of `cognitive-loop`.
### 5. Cleanup
Remove the obsolete `cognitive-loop` function and update `literate/core.org` documentation/diagrams to reflect the new architecture.
## Verification & Testing (Phase E: Chaos)
- **Unit Tests:** Verify each gate function in isolation.
- **Pipeline Test:** Simulate a complex multi-turn interaction (e.g., User Prompt -> Tool Call -> Tool Result -> LLM Final Answer) and ensure the stack depth remains flat.
- **Immune System Test:** Verify that the depth limit (Max depth 10) still correctly terminates runaway loops.