amr/org-agent-contrib
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add initial skills to contrib

Browse Source
This commit is contained in:
Amr Gharbeia
2026-04-16 12:00:23 -04:00
parent c5d3d8c5fa
commit a04d8415f3
22 changed files with 2600 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

269
skills/org-skill-state-persistence.org Normal file
View File

@@ -0,0 +1,269 @@
:PROPERTIES:
:ID: state-persistence-skill
:CREATED: [2026-04-09 Thu]
:END:
#+TITLE: SKILL: State Persistence Layer (Universal Literate Note)
#+STARTUP: content
#+FILETAGS: :memory:persistence:closos:ipfs:autonomy:
* Overview
The *State Persistence Layer* ensures the durability and autonomousty of the agent's memory. It unifies local, high-performance Lisp image dumps with decentralized, immutable IPFS checkpointing. This dual-path approach provides both rapid operational recovery and long-term historical integrity.
** Deep Reasoning: Protection Against External Tampering
While the *Prover* and *Bouncer* protect against internal skill failures, the Merkle-Tree architecture within the State Layer protects against **External Threats** (e.g., a hacker or virus modifying your `.org` files directly on disk).
1. **Skill Hashing:** Every code block and headline in a skill file has a unique Merkle hash recorded in the Memory.
2. **Integrity Verification:** Upon loading or reloading a skill, the harness re-calculates the hash and compares it against the "known good" state in the Merkle Tree.
3. **Automatic Lockdown:** If a file has been tampered with externally, the hash mismatch triggers an immediate lockdown. the harness refuses to execute the skill and alerts the Autonomous via Signal/Telegram.
* Phase A: Demand (PRD)
:PROPERTIES:
:STATUS: SIGNED
:END:
** 1. Purpose
Define automated behaviors for knowledge graph serialization, local persistence, and decentralized archival.
** 2. User Needs
- *Instant Recall:* Rapid local loading of the Memory from a persistent image.
- *Decentralized Archival:* Pushing immutable snapshots to IPFS for cross-node sync and autonomousty.
- *Merkle Integrity:* Every save operation must respect and record the Merkle-Tree history.
- *Safety:* Sanitize and validate data during restoration to prevent code injection.
* Phase B: Blueprint (PROTOCOL)
:PROPERTIES:
:STATUS: SIGNED
:END:
** 1. Architectural Intent
The persistence layer acts as a bridge between the volatile RAM-resident Memory and permanent storage backends. It provides two adapters: `LOCAL` (fast, SBCL-native) and `IPFS` (autonomous, content-addressed).
** 2. Semantic Interfaces
#+begin_src lisp
(defun persistence-dump-local ()
"Serializes RAM state to a local Lisp image file.")
(defun persistence-push-ipfs ()
"Pushes an immutable snapshot of the graph to IPFS.")
(defun persistence-restore-ipfs (cid)
"Hydrates the RAM state from an IPFS content identifier.")
#+end_src
* Phase C: Success (QUALITY)
:PROPERTIES:
:STATUS: SIGNED
:END:
** 1. Success Criteria
- [ ] *Speed:* Local image load must be <500ms for a 10k node graph.
- [ ] *Fidelity:* IPFS round-trip must result in a bit-identical Memory.
- [ ] *Validation:* Restoration must block any `read-eval` reader macros in content.
** 2. TDD Plan
Tests in `tests/persistence-tests.lisp` will verify the local dump/load cycle and the JSON serialization format for IPFS.
* Phase D: Build (Implementation)
** Package Context
#+begin_src lisp
;; Skill logic is evaluated in a jailed package by the Harness.
#+end_src
** Helper: Local State Path
Ensures we have a standardized location for local memory images.
#+begin_src lisp
(defun persistence-get-local-path ()
"Returns the path to the local memory image file."
(let ((state-dir (or (uiop:getenv "SYSTEM_DIR") "system/")))
(merge-pathnames "state/memory-image.lisp" state-dir)))
#+end_src
** Local Image Dump (persistence-dump-local)
Serializes the Merkle history and current pointers to a Lisp file.
#+begin_src lisp
(defun persistence-dump-local ()
"Serializes the entire history store and current pointers to a local Lisp image."
(let ((image-file (persistence-get-local-path)))
(ensure-directories-exist image-file)
(harness-log "PERSISTENCE - Dumping local image to ~a..." (uiop:native-namestring image-file))
(with-open-file (out image-file :direction :output :if-exists :supersede)
(format out "~%")
;; 1. Dump all immutable objects in the history store
(maphash (lambda (hash obj)
(print `(setf (gethash ,hash *history-store*) ,obj) out))
*history-store*)
;; 2. Dump the current active pointers
(maphash (lambda (id obj)
(print `(setf (gethash ,id *memory*) (gethash ,(org-object-hash obj) *history-store*)) out))
*memory*))
t))
#+end_src
** Local Image Load (persistence-load-local)
Restores the state from the local disk.
#+begin_src lisp
(defun persistence-load-local ()
"Loads the memory image from local disk."
(let ((image-file (persistence-get-local-path)))
(if (uiop:file-exists-p image-file)
(progn
(harness-log "PERSISTENCE - Loading local image...")
(load image-file)
t)
(progn
(harness-log "PERSISTENCE ERROR - Local image not found.")
nil))))
#+end_src
** IPFS Serialization (persistence-serialize-for-archival)
Converts the live `*memory*` into a list of Lisp Property Lists (Plists) for autonomous, homoiconic transport.
#+begin_src lisp
(defun persistence-serialize-for-archival ()
"Serializes the entire memory for IPFS transport as native S-Expressions."
(let ((objects nil))
(maphash (lambda (id obj)
(declare (ignore id))
(push (list :id (org-object-id obj)
:type (org-object-type obj)
:attributes (org-object-attributes obj)
:content (org-object-content obj)
:parent-id (org-object-parent-id obj)
:children (org-object-children obj)
:version (org-object-version obj)
:last-sync (org-object-last-sync obj)
:hash (org-object-hash obj))
objects))
*memory*)
objects))
#+end_src
** IPFS Push (persistence-push-ipfs)
Pushes the serialized knowledge graph to the decentralized network as a Lisp string.
#+begin_src lisp
(defun persistence-push-ipfs ()
"Serializes the store and pushes it to IPFS as a Lisp text block, returning the CID."
(let* ((data (persistence-serialize-for-archival))
(lisp-payload (format nil "~s" data))
(ipfs-url "http://127.0.0.1:5001/api/v0/add"))
(handler-case
(let* ((response (dex:post ipfs-url
:content `(("file" . ,lisp-payload))
:headers '(("Content-Type" . "multipart/form-data"))))
(result-str (flexi-streams:octets-to-string response))
(start-idx (search "\"Hash\":\"" result-str))
(cid (when start-idx
(let* ((val-start (+ start-idx 8))
(val-end (position #\" result-str :start val-start)))
(subseq result-str val-start val-end)))))
(harness-log "PERSISTENCE - Checkpoint to IPFS successful. CID: ~a" cid)
cid)
(error (c)
(harness-log "PERSISTENCE ERROR - IPFS push failed: ~a" c)
nil))))
#+end_src
** IPFS Restore (persistence-restore-ipfs)
Restores the graph from IPFS, using `read-from-string` with `*read-eval* nil` to prevent injection.
#+begin_src lisp
(defun persistence-restore-ipfs (cid)
"Fetches data from IPFS and safely hydrates the memory."
(let ((ipfs-url (format nil "http://127.0.0.1:5001/api/v0/cat?arg=~a" cid)))
(handler-case
(let* ((response (dex:post ipfs-url))
(payload-str (flexi-streams:octets-to-string response)))
(let ((*read-eval* nil))
(let ((data (read-from-string payload-str)))
(clrhash *memory*)
(dolist (item data)
(let* ((id (getf item :id))
(obj (make-org-object
:id id
:type (getf item :type)
:attributes (getf item :attributes)
:content (getf item :content)
:parent-id (getf item :parent-id)
:children (getf item :children)
:version (getf item :version)
:last-sync (getf item :last-sync)
:hash (getf item :hash))))
(setf (gethash id *memory*) obj)))
(harness-log "PERSISTENCE - Restored from IPFS: ~a" cid)
t)))
(error (c)
(harness-log "PERSISTENCE ERROR - IPFS restoration failed: ~a" c)
nil))))
#+end_src
** Cognitive Tools
Expose persistence capabilities to the neural Probabilistic Engine.
#+begin_src lisp
(progn
(def-cognitive-tool :checkpoint-memory "Creates both a local image and a decentralized IPFS snapshot." nil
:body (lambda (args)
(declare (ignore args))
(persistence-dump-local)
(let ((cid (persistence-push-ipfs)))
(format nil "Local dump complete. IPFS CID: ~a" (or cid "FAILED")))))
(def-cognitive-tool :restore-memory "Restores the state from a specific source."
((:source :type :keyword :description "Either :LOCAL or :IPFS")
(:cid :type :string :description "Required if source is :IPFS"))
:body (lambda (args)
(case (getf args :source)
(:local (if (persistence-load-local) "Restored from disk." "Local restore failed."))
(:ipfs (if (persistence-restore-ipfs (getf args :cid)) "Restored from network." "IPFS restore failed."))))))
#+end_src
** Registration
#+begin_src lisp
(defskill :skill-state-persistence
:priority 100
:trigger (lambda (ctx)
(let ((sensor (getf (getf ctx :payload) :sensor)))
(member sensor '(:heartbeat :manual-persist))))
:probabilistic nil
:deterministic (lambda (action ctx)
(persistence-dump-local)
action))
#+end_src
* Phase E: Chaos (Verification)
** 1. Unit Tests (FiveAM)
#+begin_src lisp
(defpackage :opencortex-persistence-tests
(:use :cl :fiveam :opencortex))
(in-package :opencortex-persistence-tests)
(def-suite persistence-suite :description "Tests for State Persistence Layer.")
(in-suite persistence-suite)
(test test-local-roundtrip
"Ensure RAM -> Disk -> RAM preserves data integrity."
(let ((test-id "persist-test-1")
(test-hash "fake-hash-123"))
(let ((obj (make-org-object :id test-id :content "Integrity Check" :hash test-hash)))
(setf (gethash test-hash *history-store*) obj)
(setf (gethash test-id *memory*) obj))
(opencortex:persistence-dump-local)
(clrhash *memory*)
(clrhash *history-store*)
(opencortex:persistence-load-local)
(is (not (null (gethash test-id *memory*))))
(is (equal "Integrity Check" (org-object-content (gethash test-id *memory*))))))
#+end_src
** 2. Chaos Scenarios
- *Scenario A (IPFS Daemon Down):* Kill the IPFS daemon and verify `persistence-push-ipfs` returns a standardized error instead of hanging the harness.
- *Scenario B (Corrupt Image):* Intentionally mangle the `memory-image.lisp` file and verify the loader catches the error during `load` and falls back to a clean state.
* Phase F: Memory (RCA)
- *[2026-04-09 Thu]:* Unified local SBCL image dumps with IPFS decentralized snapshots. Implemented safety-first restoration logic.