passepartout/docs/rca/rca-provider-verification.org

Root Cause Analysis: Individual Provider Track Verification

• Executive Summary
• 1. Issue: Fragile Response Parsing (Gemini)
  • Symptoms
  • Root Cause
  • Resolution
• 2. Issue: Decoupled Build Configuration
  • Symptoms
  • Root Cause
  • Resolution
• 3. Issue: Credential Key Mismatch
  • Symptoms
  • Root Cause
  • Resolution
• 4. PSF Mandate Alignment
  • Invariant Check
• 5. Permanent Learnings

Executive Summary

Verified the unified LLM gateway implementation for all 6 individual provider tracks (Anthropic, Gemini, Groq, OpenAI, OpenRouter, Ollama). Identified and resolved critical parsing failures in the Gemini track and integration gaps in the system build definition.

1. Issue: Fragile Response Parsing (Gemini)

Symptoms

Gemini API responses were returning `NIL` content during mocked unit tests, despite the JSON structure being seemingly correct.

Root Cause

Recursive `assoc` / `car` / `cdr` chains were hardcoded and brittle. Specifically, the Gemini extraction logic was incorrectly attempting to treat a single alist pair as a list of pairs, causing `assoc` to fail on the `:TEXT` key.

Resolution

Implemented a robust `get-nested` helper function that safely traverses both nested objects (alists) and arrays (lists of alists). This normalized the extraction logic across all providers.

2. Issue: Decoupled Build Configuration

Symptoms

Provider logic was present in the codebase but inaccessible during tests and runtime.

Root Cause

The `credentials-vault.lisp` and `llm-gateway.lisp` files (consolidated in a previous session) were never added to the `org-agent.asd` system definition. Furthermore, an incorrect loading order caused `UNDEFINED-FUNCTION` errors for `register-neuro-backend`.

Resolution

1. Added both files to `org-agent.asd`.
2. Enforced strict loading order: `neuro` (defines registry) -> `credentials-vault` -> `llm-gateway` (uses registry).

3. Issue: Credential Key Mismatch

Symptoms

Gemini requests failed with "API Key missing" even when environment variables were set.

Root Cause

`llm-gateway` requested secrets for the `:gemini-api` provider, but the `credentials-vault` fallback logic only recognized the `:gemini` keyword.

Resolution

Updated `vault-get-secret` to map both `:gemini` and `:gemini-api` to the same `GEMINI_API_KEY` environment variable.

4. PSF Mandate Alignment

Invariant Check

• High-Integrity Memory: All individual provider tracks are now backed by automated unit tests (`llm-gateway-tests.lisp`).
• Literate Programming: Updated `org-skill-llm-gateway.org` to reflect the improved `get-nested` utility.

5. Permanent Learnings

• Tooling vs Source: Tangled `.lisp` files are not enough; always ensure new modules are registered in the `.asd` file to be part of the official kernel build.
• Robustness over Brevity: Use abstraction helpers like `get-nested` instead of deep `car/cdr` chains when dealing with external JSON structures that may have varying array/object nesting.