passepartout/skills/org-skill-object-store-persistence.org

SKILL: Object Store Persistence (Universal Literate Note)

• Overview
• Phase A: Demand (PRD)
  • 1. Purpose
  • 2. User Needs
  • 3. Success Criteria
    • Image Dump logic verification (File exists and is readable)
    • Image Load logic verification (Object count matches RAM state)
    • Performance audit (Loading image must be >10x faster than parsing)
• Phase B: Blueprint (PROTOCOL)
  • 1. Architectural Intent
  • 2. Semantic Interfaces
• Phase D: Build (Implementation)
  • Image Serialization
  • Cognitive Tools
  • Registration

Overview

The Object Store Persistence skill ensures that the agent's perceptual memory (the `*object-store*`) is durable. It provides the mechanism to "dump" the in-RAM knowledge graph to a Lisp-native image file and "reload" it upon boot, eliminating the need to re-parse the entire Memex on every restart.

Phase A: Demand (PRD)

1. Purpose

Define automated behaviors for knowledge graph serialization and restoration.

2. User Needs

• Instant Recall: Rapid loading of the Object Store from a persistent image.
• High-Fidelity Serialization: Recursive dumping of `org-object` structs and their relations.
• Atomic Persistence: Save the entire graph state to a single `.el` or `.lisp` file.
• Background Synchronization: Periodically dump the image during heartbeats.

3. Success Criteria

TODO Image Dump logic verification (File exists and is readable)

TODO Image Load logic verification (Object count matches RAM state)

TODO Performance audit (Loading image must be >10x faster than parsing)

Phase B: Blueprint (PROTOCOL)

1. Architectural Intent

Interfaces for state dumping and restoration. Source of truth is the RAM-resident `*object-store*` and the `system/state/memory-image.lisp` file.

2. Semantic Interfaces

(defun memory-dump-image ()
  "Serializes the current *object-store* to disk.")

(defun memory-load-image ()
  "Restores the *object-store* from the persistent image file.")

Phase D: Build (Implementation)

Image Serialization

We serialize the `*history-store*` (immutable objects) and the current `*object-store*` root pointers.

(defun memory-dump-image ()
  "Serializes the entire history store and current pointers to a Lisp image."
  (let* ((state-dir (or (uiop:getenv "SYSTEM_DIR") "system/"))
         (image-file (merge-pathnames "state/memory-image.lisp" state-dir)))
    (ensure-directories-exist image-file)
    (org-agent:kernel-log "MEMORY - Dumping Merkle-Tree history to ~a..." (uiop:native-namestring image-file))
    (with-open-file (out image-file :direction :output :if-exists :supersede)
      (format out "(in-package :org-agent)~%")
      ;; 1. Dump all immutable objects in the history store
      (maphash (lambda (hash obj)
                 (print `(setf (gethash ,hash *history-store*) ,obj) out))
               org-agent:*history-store*)
      ;; 2. Dump the current active pointers (the object store)
      (maphash (lambda (id obj)
                 (print `(setf (gethash ,id *object-store*) (gethash ,(org-agent:org-object-hash obj) *history-store*)) out))
               org-agent:*object-store*))
    '(:target :system :payload (:action :message :text "Merkle-Tree image dumped."))))

(defun memory-load-image ()
  "Loads the memory image from disk."
  (let* ((state-dir (or (uiop:getenv "SYSTEM_DIR") "system/"))
         (image-file (merge-pathnames "state/memory-image.lisp" state-dir)))
    (if (uiop:file-exists-p image-file)
        (progn
          (org-agent:kernel-log "MEMORY - Loading knowledge graph image...")
          (load image-file)
          t)
        (progn
          (org-agent:kernel-log "MEMORY ERROR - Image file not found.")
          nil))))

Cognitive Tools

(org-agent:def-cognitive-tool :persist-memory "Saves the current state of the Memex (Object Store + History) to disk."
  :parameters nil
  :body (lambda (args)
          (declare (ignore args))
          (memory-dump-image)
          "MEMORY PERSISTED SUCCESSFUL."))

(org-agent:def-cognitive-tool :rollback-memory "Reverts the in-memory state to a previous snapshot."
  :parameters ((:index :type :integer :description "The snapshot index to roll back to (0 is most recent)"))
  :body (lambda (args)
          (org-agent:rollback-object-store (getf args :index))
          "ROLLBACK COMPLETE."))

Registration

(org-agent:defskill :skill-object-store-persistence
  :priority 100
  :trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :heartbeat))
  :neuro (lambda (context) nil)
  :symbolic (lambda (action context) (memory-dump-image)))