41 lines
2.7 KiB
Org Mode
41 lines
2.7 KiB
Org Mode
#+TITLE: Root Cause Analysis: Individual Provider Track Verification
|
|
#+DATE: 2026-04-11
|
|
#+FILETAGS: :rca:providers:llm:testing:autonomy:
|
|
|
|
* Executive Summary
|
|
Verified the unified LLM gateway implementation for all 6 individual provider tracks (Anthropic, Gemini, Groq, OpenAI, OpenRouter, Ollama). Identified and resolved critical parsing failures in the Gemini track and integration gaps in the system build definition.
|
|
|
|
* 1. Issue: Fragile Response Parsing (Gemini)
|
|
** Symptoms
|
|
Gemini API responses were returning `NIL` content during mocked unit tests, despite the JSON structure being seemingly correct.
|
|
** Root Cause
|
|
Recursive `assoc` / `car` / `cdr` chains were hardcoded and brittle. Specifically, the Gemini extraction logic was incorrectly attempting to treat a single alist pair as a list of pairs, causing `assoc` to fail on the `:TEXT` key.
|
|
** Resolution
|
|
Implemented a robust `get-nested` helper function that safely traverses both nested objects (alists) and arrays (lists of alists). This normalized the extraction logic across all providers.
|
|
|
|
* 2. Issue: Decoupled Build Configuration
|
|
** Symptoms
|
|
Provider logic was present in the codebase but inaccessible during tests and runtime.
|
|
** Root Cause
|
|
The `credentials-vault.lisp` and `llm-gateway.lisp` files (consolidated in a previous session) were never added to the `opencortex.asd` system definition. Furthermore, an incorrect loading order caused `UNDEFINED-FUNCTION` errors for `register-probabilistic-backend`.
|
|
** Resolution
|
|
1. Added both files to `opencortex.asd`.
|
|
2. Enforced strict loading order: `probabilistic` (defines registry) -> `credentials-vault` -> `llm-gateway` (uses registry).
|
|
|
|
* 3. Issue: Credential Key Mismatch
|
|
** Symptoms
|
|
Gemini requests failed with "API Key missing" even when environment variables were set.
|
|
** Root Cause
|
|
`llm-gateway` requested secrets for the `:gemini-api` provider, but the `credentials-vault` fallback logic only recognized the `:gemini` keyword.
|
|
** Resolution
|
|
Updated `vault-get-secret` to map both `:gemini` and `:gemini-api` to the same `GEMINI_API_KEY` environment variable.
|
|
|
|
* 4. opencortex Mandate Alignment
|
|
** Invariant Check
|
|
- *High-Integrity Memory:* All individual provider tracks are now backed by automated unit tests (`llm-gateway-tests.lisp`).
|
|
- *Literate Programming:* Updated `org-skill-llm-gateway.org` to reflect the improved `get-nested` utility.
|
|
|
|
* 5. Permanent Learnings
|
|
- **Tooling vs Source:** Tangled `.lisp` files are not enough; always ensure new modules are registered in the `.asd` file to be part of the official kernel build.
|
|
- **Robustness over Brevity:** Use abstraction helpers like `get-nested` instead of deep `car/cdr` chains when dealing with external JSON structures that may have varying array/object nesting.
|