passepartout/skills/org-skill-llm-gateway.org

SKILL: LLM Gateway (org-skill-llm-gateway.org)

• Overview
• Implementation
  • Request Execution (execute-llm-request)
  • Skill Registration
• Test Suite

Overview

The LLM Gateway skill provides a unified interface for interacting with multiple Large Language Model providers.

Implementation

Request Execution (execute-llm-request)

(defun execute-llm-request (&key prompt system-prompt (provider :ollama) model)
  "Central dispatcher for LLM requests."
  (let ((backend (gethash provider *probabilistic-backends*)))
    (if backend
        (handler-case
            (funcall backend prompt system-prompt :model model)
          (error (c)
            (list :status :error :message (format nil "~a Failure: ~a" provider c))))
        (list :status :error :message (format nil "Provider ~a not registered" provider)))))

Skill Registration

(defskill :skill-llm-gateway
  :priority 100
  :trigger (lambda (ctx) (getf ctx :user-input))
  :deterministic (lambda (action ctx) (declare (ignore ctx)) action))

Test Suite

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload :fiveam :silent t))

(defpackage :opencortex-llm-gateway-tests
  (:use :cl :opencortex)
  (:export #:llm-gateway-suite))

(in-package :opencortex-llm-gateway-tests)

(fiveam:def-suite llm-gateway-suite :description "Tests for the LLM Gateway skill")
(fiveam:in-suite llm-gateway-suite)

(fiveam:test test-llm-gateway-timeout
  "Tier 2 Chaos: Verify that LLM Gateway handles connection failures gracefully."
  (let ((old-host (uiop:getenv "OLLAMA_HOST")))
    (unwind-protect
         (progn
           (setf (uiop:getenv "OLLAMA_HOST") "localhost:1")
           (let ((fn (or (find-symbol "EXECUTE-LLM-REQUEST" :opencortex.skills.org-skill-llm-gateway)
                         (find-symbol "EXECUTE-LLM-REQUEST" :opencortex))))
             (if fn
                 (let ((result (funcall fn :prompt "hello" :provider :ollama)))
                   (fiveam:is (eq (getf result :status) :error))
                   (fiveam:is (uiop:string-prefix-p "Ollama Failure" (getf result :message))))
                 (fiveam:fail "Could not find EXECUTE-LLM-REQUEST symbol"))))
      (if old-host
          (setf (uiop:getenv "OLLAMA_HOST") old-host)
          (sb-posix:unsetenv "OLLAMA_HOST")))))