4.8 KiB
SKILL: Web Research Agent (Internet Connectivity)
Overview
The Web Research Agent provides the Neurosymbolic Kernel with its primary connection to the global internet. It utilizes multiple pluggable browser engines to fetch, parse, and synthesize information from the web, allowing the agent to perform real-time research, verify facts, and expand its knowledge base beyond the local Memex.
The Research Mandate
- Connectivity: Provide a unified interface for fetching URLs using various engines (Lynx, Curl, etc.).
- Synthesis: The neural layer must translate raw web content into concise, factual summaries aligned with the user's query.
- Pluggability: Support multiple backends to handle different levels of page complexity (e.g., text-only vs. JS-heavy).
- Efficiency: Default to lightweight, text-only engines (Lynx) to minimize latency and token overhead.
Symbolic Implementation (The Logic)
The implementation focuses on the orchestration of external CLI-based browser engines.
Architectural Intent: Delegation Trigger
This skill is engaged when the Router identifies a research-oriented request and delegates it to the `:web` target.
(defun trigger-skill-web-research (context)
(let ((type (getf context :type))
(payload (getf context :payload)))
(and (eq type :EVENT)
(eq (getf payload :sensor) :delegation)
(eq (getf payload :target-skill) :web))))Architectural Intent: Pluggable Browser Engines
These functions define the specific shell commands used to interact with the web. They provide a range of capabilities from fast text extraction (Lynx) to raw inspection (Curl), ensuring the agent has the right tool for the specific research task.
(defun fetch-with-lynx (url)
"Engine: Lynx. Best for fast text extraction from blogs/docs."
(let ((cmd (format nil "lynx -dump -nolist '~a'" url)))
(uiop:run-program cmd :output :string :ignore-error-status t)))
(defun fetch-with-curl (url)
"Engine: Curl. Best for raw HTML or API inspection."
(let ((cmd (format nil "curl -sL '~a'" url)))
(uiop:run-program cmd :output :string :ignore-error-status t)))
(defun fetch-with-playwright (url)
"Engine: Playwright (Placeholder). In the future, this calls a Python bridge."
(format nil "ERROR: Playwright engine not yet implemented. Falling back to Lynx...~%~a"
(fetch-with-lynx url)))
(defun web-fetch (url &optional engine)
"Dispatch the fetch request to the specified engine (defaults to Lynx)."
(case engine
(:lynx (fetch-with-lynx url))
(:curl (fetch-with-curl url))
(:playwright (fetch-with-playwright url))
(t (fetch-with-lynx url))))Architectural Intent: Neuro-Cognitive Synthesis
The neural layer is responsible for engine selection and content synthesis. It determines the target URL (defaulting to DuckDuckGo for general queries) and transforms the potentially massive raw output into a curated, human-readable summary.
(defun neuro-skill-web-research (context)
(let* ((payload (getf context :payload))
(query (getf payload :query))
;; The LLM can specify an engine. If not, we default to Lynx.
(requested-engine (or (getf payload :engine) :lynx))
(is-url (or (search "http://" query) (search "https://" query)))
(target-url (if is-url
query
(format nil "https://duckduckgo.com/html/?q=~a" query)))
(web-text (web-fetch target-url requested-engine)))
(let ((curated (if (and web-text (> (length web-text) 5000))
(format nil "~a... [TRUNCATED]" (subseq web-text 0 5000))
(or web-text "No content fetched."))))
(format nil "
You are the Web Research synthesizer.
USER QUERY - '~a'
ENGINE USED - ~a
TARGET URL - ~a
RAW CONTENT FETCHED -
---
~a
---
Synthesize a concise, factual answer.
Return a Lisp plist - (:target :emacs :action :message :text \"your summary\")
" query requested-engine target-url curated))))Architectural Intent: Symbolic Verification
Ensures the synthesized research is correctly formatted as a user message before delivery.
(defun verify-skill-web-research (proposed-action context)
(if (eq (getf proposed-action :action) :message)
proposed-action
'(:target :emacs :action :message :text "Web skill failed to synthesize message.")))Registration
(defskill :skill-web-research
:priority 80
:trigger #'trigger-skill-web-research
:neuro #'neuro-skill-web-research
:symbolic #'verify-skill-web-research)