memex/notes/org-skill-embedding-generator.org

SKILL: Embedding Generator (Universal Literate Note)

• Overview
• Phase A: Demand (PRD)
  • 1. Purpose
  • 2. User Needs
• Phase B: Blueprint (PROTOCOL)
  • 1. Architectural Intent
  • 2. Semantic Interfaces
• Phase D: Build (Implementation)
• Registration

Overview

The Embedding Generator provides vector representations of text for semantic search and neural memory.

Phase A: Demand (PRD)

1. Purpose

Generate embeddings for text strings.

2. User Needs

• Provider Choice: Support for local (Ollama) or remote (Gemini, OpenAI) providers.
• Batching: Efficiency through batching text (future-proof).

Phase B: Blueprint (PROTOCOL)

1. Architectural Intent

Unified interface for neural embeddings.

2. Semantic Interfaces

(defun get-embedding (text &key provider) "Acquire vector embedding for TEXT.")

Phase D: Build (Implementation)

(defun get-embedding (text &key (provider :ollama))
  "Retrieves the embedding vector for TEXT using specified PROVIDER."
  (kernel-log "NEURO [Embedding] - Generating via ~a..." provider)
  (case provider
    (:ollama (get-embedding-ollama text))
    (:gemini (get-embedding-gemini text))
    (t (error "Unsupported embedding provider: ~a" provider))))

(defun get-embedding-ollama (text)
  (let* ((url "http://localhost:11434/api/embeddings")
         (payload (cl-json:encode-json-to-string `(("model" . "mxbai-embed-large") ("prompt" . ,text))))
         (response (dex:post url :content payload :headers '(("Content-Type" . "application/json")))))
    (cdr (assoc :embedding (cl-json:decode-json-from-string response)))))

(defun get-embedding-gemini (text)
  (let* ((api-key (getf (org-agent:get-credentials :gemini) :api-key))
         (url (format nil "https://generativelanguage.googleapis.com/v1beta/models/embedding-001:embedContent?key=~a" api-key))
         (payload (cl-json:encode-json-to-string `(("content" . (("parts" . ((("text" . ,text))))))))))
    (let ((response (dex:post url :content payload :headers '(("Content-Type" . "application/json")))))
      (cdr (assoc :values (cdr (assoc :embedding (cl-json:decode-json-from-string response))))))))

Registration

(defskill :skill-embedding-generator
  :priority 50
  :trigger (lambda (context) nil)
  :neuro (lambda (context) nil)
  :symbolic (lambda (action context) action))