#+TITLE: SKILL: Atomic Notes Retrieval (Universal Literate Note)
#+ID: skill-atomic-notes
#+STARTUP: content
#+FILETAGS: :knowledge:retrieval:zettelkasten:psf:

* Overview
This skill provides the **Deep Memory** for the agent. it enables **Sparse Tree Perception** over the Zettelkasten, using semantic search and recursive interlinking to maintain high-signal context without token bloat.

* Phase A: Demand (PRD)
:PROPERTIES:
:STATUS: FROZEN
:END:

** 1. Purpose
Define the interfaces for knowledge retrieval from the atomic note DAG.

** 2. User Needs
- **Atomicity:** Each note represents exactly one concept.
- **Sparse Tree Perception:** Extract headlines and IDs before deep reading.
- **Recursive Deep-Dive:** Follow internal links to pull related context.
- **Search Efficiency:** Optimized searching via `ripgrep`.

** 3. Success Criteria
*** TODO Concept Discovery
*** TODO Link Resolution
*** TODO Sparse Tree Extraction Verification

* Phase B: Blueprint (PROTOCOL)
:PROPERTIES:
:STATUS: SIGNED
:END:

** 1. Architectural Intent
Interfaces for scanning and resolving nodes in the Zettelkasten. It implements a two-stage retrieval process: Sparse Perception (Headlines/IDs) followed by Targeted Deep-Reading.

** 2. Semantic Interfaces
#+begin_src lisp
(defun atomic-notes-scan (query)
  "Stage 1: Returns a sparse list of matching headlines and their unique IDs.")

(defun atomic-notes-deep-read (ids)
  "Stage 2: Retrieves the full content for a specific list of node IDs.")
#+end_src

* Phase D: Build (Implementation)

** Stage 1: Sparse Scan
#+begin_src lisp :tangle projects/org-skill-atomic-notes/src/retrieval-logic.lisp
(defun atomic-notes-scan (query)
  "Uses ripgrep to find matching headlines and extracts their IDs."
  (let ((notes-dir (or (uiop:getenv "MEMEX_NOTES") "notes/")))
    (kernel-log "MEMORY - Sparse Scan for: ~a" query)
    ;; We grep for headlines and include the following line which usually has the ID property
    (uiop:run-program (list "rg" "-i" "-A" "1" (format nil "^\\*+.*~a" query) notes-dir) 
                      :output :string)))
#+end_src

** Stage 2: Deep Read
#+begin_src lisp :tangle projects/org-skill-atomic-notes/src/retrieval-logic.lisp
(defun atomic-notes-deep-read (ids)
  "Retrieves the full content subtree for given IDs from the Object Store."
  (let ((results '()))
    (dolist (id ids)
      (let ((obj (org-agent:lookup-object id)))
        (when obj
          (push (list :id id :content (org-agent:org-object-content obj)) results))))
    results))
#+end_src

** Stage 3: Semantic Search (SOTA)
#+begin_src lisp :tangle projects/org-skill-atomic-notes/src/retrieval-logic.lisp
(defun atomic-notes-semantic-search (query &optional (top-k 5))
  "Uses dense vector embeddings to find semantically related notes."
  (let* ((query-vec (org-agent:get-embedding query))
         (matches (when query-vec (org-agent:find-most-similar query-vec top-k))))
    (mapcar (lambda (match)
              (let* ((score (car match))
                     (obj (cdr match))
                     (attrs (org-agent:org-object-attributes obj)))
                (list :score score 
                      :id (org-agent:org-object-id obj)
                      :title (getf attrs :TITLE))))
            matches)))
#+end_src

** Neuro-Cognitive Intelligence
#+begin_src lisp :tangle projects/org-skill-atomic-notes/src/retrieval-logic.lisp
(defun neuro-skill-atomic-notes (context)
  "Neural stage of Sparse and Semantic Perception.
   It combines ripgrep hits and semantic matches to provide high-fidelity context."
  (let* ((query (getf (getf context :payload) :query))
         (sparse-results (atomic-notes-scan query))
         (semantic-results (atomic-notes-semantic-search query)))
    (format nil "
      I have searched your Zettelkasten for '~a'.
      
      KEYWORD MATCHES (Sparse):
      ---
      ~a
      ---
      
      SEMANTIC MATCHES (Dense):
      ---
      ~{~a (Score: ~f) [ID: ~a]~%~}
      ---
      
      TASK: 
      Identify the IDs of the most relevant notes to answer the user's implicit or explicit question.
      Return a Lisp plist: (:target :atomic-notes :action :deep-read :ids (\"id1\" \"id2\"))
    " query sparse-results 
      (loop for m in semantic-results 
            collect (getf m :title) 
            collect (getf m :score) 
            collect (getf m :id)))))
#+end_src

* Registration
#+begin_src lisp
(defskill :skill-atomic-notes
  :priority 90
  :trigger (lambda (context) nil)
  :neuro #'neuro-skill-atomic-notes
  :symbolic #'atomic-notes-scan)
#+end_src