79 lines
3.1 KiB
Org Mode
79 lines
3.1 KiB
Org Mode
#+TITLE: SKILL: Autonomous Performance Auditor (Universal Literate Note)
|
|
#+ID: skill-performance-auditor
|
|
#+STARTUP: content
|
|
#+FILETAGS: :telemetry:audit:self-improvement:psf:
|
|
|
|
* Overview
|
|
The *Autonomous Performance Auditor* is the system's "Quality Control" agent. It monitors the internal `*skill-telemetry*` registry to identify skills with high failure rates or excessive latency. When a performance threshold is breached, it autonomously triggers the *Scribe-RCA* role to analyze the failure and record it in the Institutional Memory.
|
|
|
|
* Phase A: Demand (PRD)
|
|
:PROPERTIES:
|
|
:STATUS: FROZEN
|
|
:END:
|
|
|
|
** 1. Purpose
|
|
Define automated behaviors for system-wide skill performance monitoring and failure alerting.
|
|
|
|
** 2. User Needs
|
|
- *Continuous Monitoring:* Analyze skill metrics (executions, failures, latency) on every heartbeat.
|
|
- *Threshold Alerts:* Detect skills with failure rates exceeding a defined limit (e.g., >20%).
|
|
- *Loop Closure:* Autonomously trigger Root Cause Analysis (RCA) for offending skills.
|
|
- *Transparency:* Log audit results to the kernel history for user visibility.
|
|
|
|
** 3. Success Criteria
|
|
*** TODO Failure rate calculation logic verification
|
|
*** TODO Autonomous stimulus injection for Scribe-RCA skill
|
|
*** TODO Verified logging of audit results in kernel history
|
|
|
|
* Phase B: Blueprint (PROTOCOL)
|
|
:PROPERTIES:
|
|
:STATUS: SIGNED
|
|
:END:
|
|
|
|
** 1. Architectural Intent
|
|
Interfaces for telemetry inspection and diagnostic dispatch. Source of truth is the kernel's internal `*skill-telemetry*` hash table.
|
|
|
|
** 2. Semantic Interfaces
|
|
#+begin_src lisp
|
|
(defun audit-calculate-failure-rate (skill-name)
|
|
"Returns the percentage of failed executions for a given skill.")
|
|
|
|
(defun audit-scan-all-skills ()
|
|
"Iterates through the telemetry registry and identifies failing components.")
|
|
#+end_src
|
|
|
|
* Phase D: Build (Implementation)
|
|
|
|
** Diagnostic Logic
|
|
#+begin_src lisp :tangle projects/org-skill-performance-auditor/src/audit-logic.lisp
|
|
(defun audit-calculate-failure-rate (skill-name)
|
|
(let ((metrics (org-agent:context-get-skill-telemetry skill-name)))
|
|
(if (and metrics (> (getf metrics :executions) 0))
|
|
(* 100 (/ (getf metrics :failures) (getf metrics :executions)))
|
|
0)))
|
|
|
|
(defun audit-scan-all-skills ()
|
|
(let ((failing-skills '()))
|
|
(dolist (skill-info (org-agent:context-list-all-skills))
|
|
(let* ((name (getf skill-info :name))
|
|
(rate (audit-calculate-failure-rate name)))
|
|
(when (> rate 20) ; Threshold: 20% failure rate
|
|
(kernel-log "AUDITOR - FAILURE DETECTED: Skill '~a' is failing at ~a%" name rate)
|
|
(push name failing-skills))))
|
|
failing-skills))
|
|
#+end_src
|
|
|
|
* Registration
|
|
#+begin_src lisp
|
|
(defskill :skill-performance-auditor
|
|
:priority 95 ; High-priority meta-cognition
|
|
:trigger (lambda (context) (eq (getf (getf context :payload) :sensor) :heartbeat))
|
|
:neuro (lambda (context) nil)
|
|
:symbolic (lambda (action context)
|
|
(let ((failing (audit-scan-all-skills)))
|
|
(dolist (name failing)
|
|
;; Trigger Scribe-RCA for each failing skill
|
|
(org-agent:inject-stimulus
|
|
`(:type :EVENT :payload (:sensor :audit-failure :skill ,name)))))))
|
|
#+end_src
|