81 lines
2.7 KiB
Org Mode
81 lines
2.7 KiB
Org Mode
:PROPERTIES:
|
|
:ID: 6cd0b4d7-e7cf-49ba-a847-e81c4ace0f76
|
|
:CREATED: [2026-04-04 Sat 20:27]
|
|
:EDITED: [2026-04-07 Tue 13:42]
|
|
:END:
|
|
#+TITLE: SKILL: Embedding Generator (Universal Literate Note)
|
|
#+STARTUP: content
|
|
#+FILETAGS: :embeddings:neural:psf:
|
|
|
|
* Overview
|
|
The *Embedding Generator* provides vector representations of text for semantic search and neural memory.
|
|
|
|
* Phase A: Demand (PRD)
|
|
:PROPERTIES:
|
|
:STATUS: FROZEN
|
|
:END:
|
|
|
|
** 1. Purpose
|
|
Generate embeddings for text strings.
|
|
|
|
** 2. User Needs
|
|
- *Provider Choice:* Support for local (Ollama) or remote (Gemini, OpenAI) providers.
|
|
- *Batching:* Efficiency through batching text (future-proof).
|
|
|
|
|
|
* Phase B: Blueprint (PROTOCOL)
|
|
:PROPERTIES:
|
|
:STATUS: SIGNED
|
|
:END:
|
|
|
|
** Phase B: Blueprint (PROTOCOL)
|
|
:PROPERTIES:
|
|
:STATUS: DRAFT
|
|
:END:
|
|
|
|
*** 1. Architectural Intent
|
|
|
|
This system aims for a flexible and extensible architecture that can accommodate different embedding providers (local or remote) while maintaining a consistent interface for the user. It also seeks to optimize for batch processing to improve throughput. The core design principle is *provider abstraction*.
|
|
|
|
*** 2. Semantic Interfaces
|
|
|
|
#+BEGIN_SRC lisp
|
|
;;; Primary Function: Generate Embeddings
|
|
;;; Input: A list of strings, and provider configuration
|
|
;;; Output: A list of embedding vectors (lists of floats).
|
|
|
|
(defun generate-embeddings (texts provider-config)
|
|
"""Generates embeddings for a list of texts using the specified provider.""")
|
|
|
|
;;; Provider Configuration Structure
|
|
;;; This plist defines the provider to use, and any necessary credentials or parameters.
|
|
|
|
;;; Example: Local Ollama provider
|
|
;;; (:provider :ollama :model "mistralai/Mistral-7B-Instruct-v0.2")
|
|
|
|
;;; Example: Remote Gemini provider
|
|
;;; (:provider :gemini :api-key "YOUR_API_KEY" :model "gemini-1.5-pro")
|
|
|
|
;;; Example: Remote OpenAI provider.
|
|
;;; (:provider :openai :api-key "YOUR_API_KEY" :model "text-embedding-ada-002")
|
|
|
|
;;; Sub-Function: (Abstract) Provider-Specific Embedding Generation
|
|
;;; This function is implemented differently for each provider.
|
|
|
|
(defgeneric generate-embeddings-from-provider (texts provider-config))
|
|
|
|
;;; Example implementation for :ollama provider
|
|
(defmethod generate-embeddings-from-provider (texts (provider-config (eql (getf provider-config :provider :ollama))))
|
|
"""Generates embeddings using a local Ollama server.""")
|
|
|
|
;;; Example implementation for :gemini provider
|
|
(defmethod generate-embeddings-from-provider (texts (provider-config (eql (getf provider-config :provider :gemini))))
|
|
"""Generates embeddings using the Gemini API.""")
|
|
|
|
;;; Example implementation for :openai provider
|
|
(defmethod generate-embeddings-from-provider (texts (provider-config (eql (getf provider-config :provider :openai))))
|
|
"""Generates embeddings using the OpenAI API.""")
|
|
|
|
#+END_SRC
|
|
|