memex/notes/org-skill-function-calling.org

SKILL: Native Function Calling (Universal Literate Note)

• Overview
• Phase A: Demand (PRD)
  • 1. Purpose
  • 2. User Needs
  • 3. Success Criteria
    • Lisp-to-JSON Schema conversion logic verification
    • Multi-provider schema formatting (Gemini vs OpenAI)
    • Response parsing from tool_call to symbolic action
• Phase B: Blueprint (PROTOCOL)
• Phase B: Blueprint (PROTOCOL)
  • 1. Architectural Intent
  • 2. Semantic Interfaces (Lisp Signatures)
    • `defun json-schema-from-defun (function-name)`
    • `defun parse-tool-call-arguments (function-name arguments tool-provider)`
    • `defun format-json-schema-for-provider (json-schema tool-provider)`
    • `defun validate-arguments (function-name arguments)`
  • 3. Data Structures
    • JSON Schema (Lisp Representation)
    • Tool Call Response

Overview

The Native Function Calling skill provides the translation layer between the system's deterministic Lisp interfaces and the LLM's neural tool-calling capabilities. It ensures that System 1 (the LLM) interacts with the world via structured, validated schemas rather than raw text plists, virtually eliminating "formatting hallucinations."

Phase A: Demand (PRD)

1. Purpose

Define a high-reliability bridge for LLM-native "Tool Use."

2. User Needs

• Schema Generation: Automatically convert Lisp `defun` signatures into JSON Schema tool definitions.
• Reliable Ingress: Parse the LLM's structured `tool_calls` response back into a valid Lisp plist.
• Provider Agnostic: Support schema formats for Gemini, OpenAI, and Anthropic.
• Validation: Ensure arguments match the required types before reaching System 2.

3. Success Criteria

TODO Lisp-to-JSON Schema conversion logic verification

TODO Multi-provider schema formatting (Gemini vs OpenAI)

TODO Response parsing from tool_call to symbolic action

Phase B: Blueprint (PROTOCOL)

Phase B: Blueprint (PROTOCOL)

1. Architectural Intent

The core intent is to create a robust, bi-directional translation layer. This layer guarantees type safety and schema adherence between the LLM's Tool Calling mechanism and the Lisp environment. The design emphasizes clear separation of concerns: schema generation, response parsing, and provider-specific formatting. We should aim for a modular architecture that allows for easier extension to new LLM providers and new data types. The validation process must be explicit and easily auditable. Error handling is critical; parsing failures should yield informative error messages, enabling rapid debugging.

2. Semantic Interfaces (Lisp Signatures)

`defun json-schema-from-defun (function-name)`

• Purpose: Generates a JSON Schema representation from a Lisp function definition.

• Args:

  • `function-name`: A symbol representing the name of the Lisp function.
• Returns: A Lisp plist representing the JSON Schema. Keys should correspond to standard JSON Schema fields (e.g., `:type`, `:properties`, `:required`).
• Side Effects: None. Pure function.

`defun parse-tool-call-arguments (function-name arguments tool-provider)`

• Purpose: Parses the arguments returned by an LLM tool call into a Lisp plist. Validates the arguments against the schema generated by `json-schema-from-defun`.

• Args:

  • `function-name`: A symbol representing the name of the Lisp function being called.
  • `arguments`: A string containing the JSON arguments returned by the LLM's `tool_calls` field.
  • `tool-provider`: A keyword (e.g., `:openai`, `:gemini`, `:anthropic`) indicating the LLM provider.
• Returns: A Lisp plist representing the parsed arguments, or `nil` if parsing fails. On failure, appropriate error messages should be logged.
• Side Effects: May signal errors.

`defun format-json-schema-for-provider (json-schema tool-provider)`

• Purpose: Formats the automatically generated JSON schema to the specific format required by each LLM provider.

• Args:

  • `json-schema`: A Lisp plist containing the generic JSON schema (output of `json-schema-from-defun`).
  • `tool-provider`: A keyword (e.g., `:openai`, `:gemini`, `:anthropic`) indicating the LLM provider.
• Returns: A Lisp plist representing the provider-specific JSON schema.
• Side Effects: None. Pure function.

`defun validate-arguments (function-name arguments)`

• Purpose: Validates that the parsed arguments conform to the expected schema.

• Args:

  • `function-name`: A symbol identifying the function being called. Used to retrieve the function definition and associated JSON schema.
  • `arguments`: A Lisp plist containing the parsed arguments.
• Returns: `T` if validation succeeds, `NIL` if it fails.
• Side Effects: May signal errors if validation fails. Logs validation errors.

3. Data Structures

JSON Schema (Lisp Representation)

A Lisp plist mimicking the structure of a JSON Schema. Keys will generally be keywords mirroring JSON Schema vocabulary (e.g., `:type`, `:properties`, `:required`, `:description`). Values will be Lisp datatypes corresponding to the schema datatypes (e.g., symbols, strings, booleans, numbers, lists of symbols/strings/numbers).

Tool Call Response

The expected format of an LLM's `tool_calls` response will be parsed using a dedicated JSON parsing library. `parse-tool-call-arguments` will handle the conversion to lisp datatypes based upon the `json-schema`