110 lines
4.8 KiB
Org Mode
110 lines
4.8 KiB
Org Mode
#+TITLE: SKILL: Web Research Agent (Internet Connectivity)
|
|
#+ID: skill-web-research-agent
|
|
#+STARTUP: content
|
|
|
|
* Overview
|
|
The **Web Research Agent** provides the Neurosymbolic Kernel with its primary connection to the global internet. It utilizes multiple pluggable browser engines to fetch, parse, and synthesize information from the web, allowing the agent to perform real-time research, verify facts, and expand its knowledge base beyond the local Memex.
|
|
|
|
* The Research Mandate
|
|
1. **Connectivity:** Provide a unified interface for fetching URLs using various engines (Lynx, Curl, etc.).
|
|
2. **Synthesis:** The neural layer must translate raw web content into concise, factual summaries aligned with the user's query.
|
|
3. **Pluggability:** Support multiple backends to handle different levels of page complexity (e.g., text-only vs. JS-heavy).
|
|
4. **Efficiency:** Default to lightweight, text-only engines (Lynx) to minimize latency and token overhead.
|
|
|
|
* Symbolic Implementation (The Logic)
|
|
The implementation focuses on the orchestration of external CLI-based browser engines.
|
|
|
|
** Architectural Intent: Delegation Trigger
|
|
This skill is engaged when the Router identifies a research-oriented request and delegates it to the `:web` target.
|
|
|
|
#+begin_src lisp
|
|
(defun trigger-skill-web-research (context)
|
|
(let ((type (getf context :type))
|
|
(payload (getf context :payload)))
|
|
(and (eq type :EVENT)
|
|
(eq (getf payload :sensor) :delegation)
|
|
(eq (getf payload :target-skill) :web))))
|
|
#+end_src
|
|
|
|
** Architectural Intent: Pluggable Browser Engines
|
|
These functions define the specific shell commands used to interact with the web. They provide a range of capabilities from fast text extraction (Lynx) to raw inspection (Curl), ensuring the agent has the right tool for the specific research task.
|
|
|
|
#+begin_src lisp
|
|
(defun fetch-with-lynx (url)
|
|
"Engine: Lynx. Best for fast text extraction from blogs/docs."
|
|
(let ((cmd (format nil "lynx -dump -nolist '~a'" url)))
|
|
(uiop:run-program cmd :output :string :ignore-error-status t)))
|
|
|
|
(defun fetch-with-curl (url)
|
|
"Engine: Curl. Best for raw HTML or API inspection."
|
|
(let ((cmd (format nil "curl -sL '~a'" url)))
|
|
(uiop:run-program cmd :output :string :ignore-error-status t)))
|
|
|
|
(defun fetch-with-playwright (url)
|
|
"Engine: Playwright (Placeholder). In the future, this calls a Python bridge."
|
|
(format nil "ERROR: Playwright engine not yet implemented. Falling back to Lynx...~%~a"
|
|
(fetch-with-lynx url)))
|
|
|
|
(defun web-fetch (url &optional engine)
|
|
"Dispatch the fetch request to the specified engine (defaults to Lynx)."
|
|
(case engine
|
|
(:lynx (fetch-with-lynx url))
|
|
(:curl (fetch-with-curl url))
|
|
(:playwright (fetch-with-playwright url))
|
|
(t (fetch-with-lynx url))))
|
|
#+end_src
|
|
|
|
** Architectural Intent: Neuro-Cognitive Synthesis
|
|
The neural layer is responsible for engine selection and content synthesis. It determines the target URL (defaulting to DuckDuckGo for general queries) and transforms the potentially massive raw output into a curated, human-readable summary.
|
|
|
|
#+begin_src lisp
|
|
(defun neuro-skill-web-research (context)
|
|
(let* ((payload (getf context :payload))
|
|
(query (getf payload :query))
|
|
;; The LLM can specify an engine. If not, we default to Lynx.
|
|
(requested-engine (or (getf payload :engine) :lynx))
|
|
(is-url (or (search "http://" query) (search "https://" query)))
|
|
(target-url (if is-url
|
|
query
|
|
(format nil "https://duckduckgo.com/html/?q=~a" query)))
|
|
(web-text (web-fetch target-url requested-engine)))
|
|
|
|
(let ((curated (if (and web-text (> (length web-text) 5000))
|
|
(format nil "~a... [TRUNCATED]" (subseq web-text 0 5000))
|
|
(or web-text "No content fetched."))))
|
|
|
|
(format nil "
|
|
You are the Web Research synthesizer.
|
|
USER QUERY - '~a'
|
|
ENGINE USED - ~a
|
|
TARGET URL - ~a
|
|
|
|
RAW CONTENT FETCHED -
|
|
---
|
|
~a
|
|
---
|
|
|
|
Synthesize a concise, factual answer.
|
|
Return a Lisp plist - (:target :emacs :action :message :text \"your summary\")
|
|
" query requested-engine target-url curated))))
|
|
#+end_src
|
|
|
|
** Architectural Intent: Symbolic Verification
|
|
Ensures the synthesized research is correctly formatted as a user message before delivery.
|
|
|
|
#+begin_src lisp
|
|
(defun verify-skill-web-research (proposed-action context)
|
|
(if (eq (getf proposed-action :action) :message)
|
|
proposed-action
|
|
'(:target :emacs :action :message :text "Web skill failed to synthesize message.")))
|
|
#+end_src
|
|
|
|
* Registration
|
|
#+begin_src lisp
|
|
(defskill :skill-web-research
|
|
:priority 80
|
|
:trigger #'trigger-skill-web-research
|
|
:neuro #'neuro-skill-web-research
|
|
:symbolic #'verify-skill-web-research)
|
|
#+end_src
|