memex/notes/org-skill-web-research.org

SKILL: Web Research Agent (Universal Literate Note)

• Overview
• Phase A: Demand (PRD)
  • 1. Purpose
  • 2. User Needs
  • 3. Success Criteria
    • Engine Fetching Verification (Lynx/Curl)
    • URL vs Query Routing Logic
    • Neural Synthesis Formatting Accuracy
• Phase B: Blueprint (PROTOCOL)
  • 1. Architectural Intent
  • 2. Semantic Interfaces
• Phase D: Build (Implementation)
  • Browser Engines
  • Trigger Perception
• Registration

Overview

The Web Research Agent provides the bridge to the internet. It fetches and synthesizes information from the web using pluggable engines like Lynx and Curl, enabling real-time research and fact verification.

Phase A: Demand (PRD)

1. Purpose

Define the interfaces for internet information retrieval and synthesis.

2. User Needs

• Connectivity: Pluggable engines (Lynx, Curl) for fetching URLs.
• Synthesis: Neural transformation of raw content into factual summaries.
• Efficiency: Default to text-only engines to minimize overhead.
• Search Integration: Automatic DuckDuckGo routing for general queries.

3. Success Criteria

TODO Engine Fetching Verification (Lynx/Curl)

TODO URL vs Query Routing Logic

TODO Neural Synthesis Formatting Accuracy

Phase B: Blueprint (PROTOCOL)

1. Architectural Intent

Interfaces for web I/O and content synthesis. Source of truth is the global internet and local CLI browser engines.

2. Semantic Interfaces

(defun trigger-skill-web-research (context)
  "Triggers on :delegation :target-skill :web.")

(defun web-fetch (url &optional engine)
  "Dispatches fetch request to CLI engines.")

(defun neuro-skill-web-research (context)
  "Neural selection of engine and synthesis of fetched content.")

Phase D: Build (Implementation)

Browser Engines

(defun fetch-with-lynx (url)
  (let ((cmd (format nil "lynx -dump -nolist '~a'" url)))
    (uiop:run-program cmd :output :string :ignore-error-status t)))

(defun fetch-with-curl (url)
  (let ((cmd (format nil "curl -sL '~a'" url)))
    (uiop:run-program cmd :output :string :ignore-error-status t)))

(defun web-fetch (url &optional engine)
  (case engine
    (:curl (fetch-with-curl url))
    (t (fetch-with-lynx url))))

Trigger Perception

(defun trigger-skill-web-research (context)
  (let ((type (getf context :type))
        (payload (getf context :payload)))
    (and (eq type :EVENT)
         (eq (getf payload :sensor) :delegation)
         (eq (getf payload :target-skill) :web))))

Registration

(defskill :skill-web-research
  :priority 80
  :trigger #'trigger-skill-web-research
  :neuro #'neuro-skill-web-research
  :symbolic #'verify-skill-web-research)