memex/notes/org-skill-atomic-notes.org

SKILL: Atomic Notes Retrieval (Universal Literate Note)

• Overview
• Phase A: Demand (PRD)
  • 1. Purpose
  • 2. User Needs
  • 3. Success Criteria
    • Concept Discovery
    • Link Resolution
    • Sparse Tree Extraction Verification
• Phase B: Blueprint (PROTOCOL)
  • 1. Architectural Intent
  • 2. Semantic Interfaces
• Phase D: Build (Implementation)
  • Stage 1: Sparse Scan
  • Stage 2: Deep Read
  • Neuro-Cognitive Intelligence
• Registration

Overview

This skill provides the Deep Memory for the agent. it enables Sparse Tree Perception over the Zettelkasten, using semantic search and recursive interlinking to maintain high-signal context without token bloat.

Phase A: Demand (PRD)

1. Purpose

Define the interfaces for knowledge retrieval from the atomic note DAG.

2. User Needs

• Atomicity: Each note represents exactly one concept.
• Sparse Tree Perception: Extract headlines and IDs before deep reading.
• Recursive Deep-Dive: Follow internal links to pull related context.
• Search Efficiency: Optimized searching via `ripgrep`.

3. Success Criteria

TODO Concept Discovery

TODO Link Resolution

TODO Sparse Tree Extraction Verification

Phase B: Blueprint (PROTOCOL)

1. Architectural Intent

Interfaces for scanning and resolving nodes in the Zettelkasten. It implements a two-stage retrieval process: Sparse Perception (Headlines/IDs) followed by Targeted Deep-Reading.

2. Semantic Interfaces

(defun atomic-notes-scan (query)
  "Stage 1: Returns a sparse list of matching headlines and their unique IDs.")

(defun atomic-notes-deep-read (ids)
  "Stage 2: Retrieves the full content for a specific list of node IDs.")

Phase D: Build (Implementation)

Stage 1: Sparse Scan

(defun atomic-notes-scan (query)
  "Uses ripgrep to find matching headlines and extracts their IDs."
  (let ((notes-dir (or (uiop:getenv "MEMEX_NOTES") "notes/")))
    (kernel-log "MEMORY - Sparse Scan for: ~a" query)
    ;; We grep for headlines and include the following line which usually has the ID property
    (uiop:run-program (list "rg" "-i" "-A" "1" (format nil "^\\*+.*~a" query) notes-dir) 
                      :output :string)))

Stage 2: Deep Read

(defun atomic-notes-deep-read (ids)
  "Retrieves the full content subtree for given IDs from the Object Store."
  (let ((results '()))
    (dolist (id ids)
      (let ((obj (org-agent:lookup-object id)))
        (when obj
          (push (list :id id :content (org-agent:org-object-content obj)) results))))
    results))

Neuro-Cognitive Intelligence

(defun neuro-skill-atomic-notes (context)
  "Neural stage of Sparse Perception.
   It analyzes the search results and decides which specific IDs to 'deep read'."
  (let ((query-results (atomic-notes-scan (getf (getf context :payload) :query))))
    (format nil "
      I found the following headlines matching your query:
      ---
      ~a
      ---
      
      TASK: 
      Identify the IDs of the most relevant notes. 
      Return a Lisp plist: (:target :atomic-notes :action :deep-read :ids (\"id1\" \"id2\"))
    " query-results)))

Registration

(defskill :skill-atomic-notes
  :priority 90
  :trigger (lambda (context) nil)
  :neuro #'neuro-skill-atomic-notes
  :symbolic #'atomic-notes-scan)