Files
hermes-brain/ideas/passepartout-economics/evaluation-harness.org
Hermes 9b2be10c77 Restructure economics doc into 27 org-roam interlinked nodes
Replace monolithic passepartout-economics.org with directory of
org-roam style nodes, each with :ID: property and cross-references
using [[id:uuid][title]] format.

27 nodes organized by theme:
- Core: index, triad overview, agora, stoa
- Revenue: verification appliance, domain gate packages, evaluation
  harness, skill marketplace, agora usernames, PDS service, compute marketplace
- Strategy: investment thesis, moats, licensing, patents, AI industry impact
- Analysis: lisp economics, sufficiency flip, time estimates, cost structure,
  gate rule encoding, upgrade lifecycle, biology parallels, symbolics comparison
- Big money: verification monopoly, infrastructure lock-in

Old file kept as archive with redirect links to new structure.
2026-05-21 19:36:02 +00:00

1.2 KiB

Evaluation Harness as Certification Service

The accumulated regression suite — thousands of edge cases from every deployed instance, every bug fix, every regulatory change — becomes the most comprehensive test of autonomous agent correctness.

Service: "Run our 10,000-task suite against your AI agent and get a Merkle-verified score." Target: AI labs proving their agents' capabilities, enterprise procurement requiring independent verification. Price: $50K-$200K per certification.

The regression suite grows with every deployment, making the certification increasingly valuable over time. The early player's suite is the largest because they started first.

10 certifications in year one = $500K-$2M.

Long-term endpoint: this becomes the UL certification for AI — a third-party verification nobody can ignore. The verification monopoly.

See also: Verification appliance, Infrastructure lock-in, Moats