From 04944a62e28668623b4400aae7263609fb22df4d Mon Sep 17 00:00:00 2001 From: Amr Gharbeia Date: Sat, 9 May 2026 15:00:35 -0400 Subject: [PATCH] memex: update AGENTS.md with ROADMAP TODO workflow, bump passepartout to v0.8.0 - AGENTS.md: add steps 0 (read next TODO from ROADMAP) and 6 (mark DONE with LOGBOOK) to development cycle - Notes updates accumulated during v0.8.0 work - Bump passepartout submodule to v0.8.0 --- AGENTS.md | 29 +- ...osymbolic-design-decisions-and-options.org | 819 ++++++++++++++++-- notes/passepartout-neurosymbolic-roadmap.org | 499 +++++++++-- projects/passepartout | 2 +- 4 files changed, 1209 insertions(+), 140 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 686b5f0..221cc10 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,17 +2,28 @@ ## Development Cycle (every change) -1. **Think in org** — write your reasoning, goals, and approach in the .org file first -2. **Write contract** — define a `** Contract` section listing each function's behavior: +1. **Read the next TODO** — find the next unreached `*** TODO` item in + `docs/ROADMAP.org` (search `*** TODO`). Read its prose, `:PROPERTIES:`, + and estimated line budget. That item is the target for this change cycle. +2. **Think in org** — write your reasoning, goals, and approach in the .org file first +3. **Write contract** — define a `** Contract` section listing each function's behavior: `(fn-name args)`: description. Returns/guarantees ... -3. **TDD from contract** — each contract item becomes a `fiveam:test` in `* Test Suite` +4. **TDD from contract** — each contract item becomes a `fiveam:test` in `* Test Suite` a. Write the test first → tangle → run → prove it FAILS (RED) b. Write the implementation → tangle → run → prove it PASSES (GREEN) c. Record both failure and success output -4. **Reflect in org** — once tests pass, ensure the implementation is in the .org source -5. **Update literate prose** — write/update the explanatory text around the code: +5. **Reflect in org** — once tests pass, ensure the implementation is in the .org source +6. **Update literate prose** — write/update the explanatory text around the code: what it does, why it exists, how it connects to the rest of the system -6. **Commit** — only when asked. Ask first. +7. **Mark the origin TODO DONE** — in `docs/ROADMAP.org`, change the + `*** TODO` item to `*** DONE` and add a `:LOGBOOK:` entry with the + completion date: + #+begin_src org + :LOGBOOK: + - State "DONE" from "TODO" [YYYY-MM-DD Day] + :END: + #+end_src +8. **Commit** — only when asked. Ask first. ## Commands @@ -51,13 +62,9 @@ When REPL is down: fall back to the SBCL cycle above. - Before shipping a version, run the `** File Update Checklist` in `docs/ROADMAP.org` - **YOU MAY NOT** push a version tag (e.g., `v0.5.0`), create a GitHub release, or run `git push` that triggers CI/CD version workflows without explicit permission. Ask first. - -## Core Boundary (HARD RULE) - - **YOU MAY NOT add files to `passepartout.asd` `:components` without asking for permission.** ASDF `:components` is the core harness. Files there load on every daemon boot, cannot be hot-reloaded, and a bug there kills the agent's brainstem. - - When you want to add a new module, **ask first**. Provide: 1. Why it cannot be a skill (the self-repair criterion — can the agent fix it if corrupted without human help?) Demonstrate specifically how a broken @@ -65,11 +72,9 @@ When REPL is down: fall back to the SBCL cycle above. or acting — not just degrading performance or losing a feature. 2. What it depends on and what depends on it 3. Why it cannot use `fboundp` guards from core - - **Default: everything is a skill.** Skills load via `skill-initialize-all`, are hot-reloadable, self-repairable, and a bug in a skill degrades the agent but doesn't kill it. The harness stays thin. - - **The self-repair criterion**: a file belongs in core only if, when corrupted, the agent *cannot* fix it without human help. Corrupted core = dead brain, dead hands, or unreachable. Corrupted skill = degraded but self-repairable. diff --git a/notes/passepartout-neurosymbolic-design-decisions-and-options.org b/notes/passepartout-neurosymbolic-design-decisions-and-options.org index e35e857..e343123 100644 --- a/notes/passepartout-neurosymbolic-design-decisions-and-options.org +++ b/notes/passepartout-neurosymbolic-design-decisions-and-options.org @@ -246,74 +246,129 @@ This is the core architecture pattern. Everything else — the entity classes, t deduction engine, the persistence layer — follows from this single design decision: *the LLM proposes; the symbolic engine decides whether to accept.* -* Three Contradiction Policies — Domain-Dependent Consistency +* Two Cardinality Policies — Singular, Dual, and Plural Facts Classical logic requires consistency. A contradiction implies everything (=ex contradictione quodlibet=). Screamer, as a constraint solver, also requires consistency — a contradictory constraint set has no solutions. But the symbolic engine operates across domains where the meaning of contradiction is fundamentally -different. +different. The correct question is not "is this consistent?" but "what cardinality +of truth does this domain support?" -A single architecture serves all domains by applying different contradiction -policies, scoped to the entity class: +Time is not a policy. It is a universal dimension that applies equally to every +fact, regardless of cardinality. All facts carry =:timestamp= and =:parent-id= +fields. Every fact has a version history. Every fact lives in a Merkle chain +that captures how it changed. The cardinality policy only governs what happens +at a given logical moment when two values coexist for the same =entity= and +=relation=. -** Policy :exclusive — Contradiction Rejected at Admission +** Policy :singular — One Active Value, One Version Chain -For domains where the world is physically singular — a file either exists or it -doesn't, a command either was blocked or it wasn't, a gate rule either applies or -it doesn't. When a new fact contradicts an existing one in an :exclusive domain, -the new fact is rejected. The existing fact is authoritative unless a human -explicitly retracts it. +The active set contains exactly one value for =(:entity :relation)= at a time. +When a new value asserts for the same pair, the old value is not rejected. It +is superseded — moved into the version history, linked to the new leaf by +=:parent-id=, and retained permanently. The active value is the leaf of the +Merkle chain. + +"I used to think =rm -rf /= was safe. Now I know it is catastrophic." Both +facts exist. Both are true — the first at =2024-06-01=, the second at +=2025-03-15=. The chain captures the evolution. The =:singular= policy means +there is one truth /now/, not that there was only ever one truth. Use for: security classifications, file system state, gate rules, code -correctness, deterministic safety constraints. +correctness, deterministic safety constraints — domains that converge on +one answer, evolving over time. -** Policy :coexistent — Contradiction Flagged, Both Retained +** Policy :dual — Exactly Two Values, in Explicit Tension -For domains where multiple truths coexist — literary interpretations, historical -accounts, personal beliefs held at different times, multi-source factual -disagreement (Wikidata vs. DBpedia vs. your memex). When a new fact contradicts -an existing one in a :coexistent domain, the contradiction is recorded with a -cross-reference flag. Both facts are stored. Queries return all facts with -provenance display. +The active set contains exactly two values for =(:entity :relation)=. Both are +simultaneously true. Both carry independent version histories. A third value is +rejected — the domain is binary by nature. -Use for: literature, history, personal knowledge evolution, scientific consensus -shift, multi-author knowledge bases. +Some contradictions are productive precisely /because/ they are binary. Thesis +and antithesis. Love and resentment. Wave and particle. A poem's two incompatible +readings. The symbolic index holds both, cross-referenced as complementary rather +than conflicting. The user is not asked to resolve the tension. The tension is +the fact. -** Policy :temporal — Contradiction Accepted as Version Change +The system can reason about cardinality transitions: a =:dual= fact that has +one interpretation superseded should collapse to =:singular=. A =:dual= that +has a third interpretation asserted should prompt the user: "Promote to =:plural= +or demote one interpretation?" The cardinality tracks the state of the domain. -For domains where truth changes over time. When a new fact contradicts an old one -in a :temporal domain, the old fact is marked =:superseded= but retained. The -timeline is queryable: "You believed X on Tuesday, Y on Friday, Z on Sunday." +Use for: productive binary tensions, complementary opposites, dialectical +pairs, any domain where two answers are both true and their tension is +meaningful. -Use for: personal belief evolution, project plan revisions, scientific -consensus shift over time, any knowledge where the change itself is information. +** Policy :plural — N Active Values, Open Set + +The active set contains any number of values for =(:entity :relation)=. Each +value has independent provenance and its own version history. Queries return +all active values with provenance display. Contradictions are flagged as +cross-references between values — information, not error. + +A =:plural= fact where all but one value are superseded should collapse to +=:singular=. A =:plural= fact where the set reduces to two active values — +and the remaining two are complementary — should collapse to =:dual=. + +Use for: literary interpretation, scientific hypotheses, personal beliefs held +at different times (when the tension is multi-faceted rather than binary), +multi-source factual disagreement, open-ended exploration. + +** Time Is Universal, Not a Policy + +Every fact — regardless of cardinality — lives in a version chain. The Merkle +DAG (see "Merkle DAG for Version History" below) captures every version of every +fact. The policy only governs the cardinality of the active set at a single +logical moment. + +The version chain is a linked list of facts, each pointing to its predecessor +via =:parent-id=, each hashed with =SHA-256(content || parent-hash)=. Changing +any version invalidates all downstream hashes. The chains form a DAG — independent +facts evolve independently; only facts in the same =(:entity :relation)= chain +share ancestry. + +A global snapshot captures the root hash over all chains at a point in time. +Rollback restores the entire fact state to that snapshot. This already exists in +Passepartout's Merkle memory (v0.2.0) — the fact store is a new occupant of +existing housing, not a new foundation. ** Policy Assignment The policy is assigned when a category is defined. New categories default to -=:coexistent= (never loses information). Core security categories are explicitly -=:exclusive=. The gate stack's bootstrapped facts are =:exclusive= because they -describe the actual filesystem, not perspectives. +=:plural= (safe — never loses information). Core security categories are +explicitly =:singular=. The gate stack's bootstrapped facts are =:singular= +because they describe the actual filesystem, which is physically singular. +Categories for dialectical or complementary domains are explicitly =:dual=. -The Screamer admission gate does not reject all contradictions. It rejects -contradictions in =:exclusive= domains and flags them in =:coexistent= and -=:temporal= domains. The constraint solver still works because queries scope -their constraint set to a single provenance domain. "Is X true according to my -memex?" is a different query than "Is X true according to Wikidata?" Each has -a self-consistent internal logic. The contradiction is between domains, not -within them. +The Screamer admission gate applies the cardinality policy at the active set: +- =:singular= + same value, later timestamp → supersede old, chain new as leaf. +- =:singular= + different value, same timestamp → reject (contradiction). Human + resolves: which is the active value? +- =:singular= + different value, later timestamp → supersede old, chain new as + leaf. History preserved. +- =:dual= + first value → admit. + second value → admit, cross-reference as + complementary. + third value → prompt: promote to =:plural= or demote one + existing? +- =:dual= + superseding value (same position) → chain via =:parent-id=. +- =:plural= + any value → admit. If active count drops to 2 and values are + complementary → prompt: collapse to =:dual=? If active count drops to 1 → + collapse to =:singular= automatically or prompt. ** Why This Matters for the Broader Memex -In the coding domain, contradiction is rare and must be resolved — a gate can't -both allow and block the same path. In the broader memex, contradiction is the -product, not the error. Your poetry analysis contradicts your last diary entry -on the same topic. Your reading of /Pale Fire/ changed between 2023 and 2025. -Wikidata says Mount Everest is 8848m (China: rock height); DBpedia says 8849m -(Nepal: snow height). The symbolic engine's job is not to decide which is right. -It is to surface the tension with provenance — "these three sources disagree. -Here is the chain for each." +In the coding domain, contradiction is rare, resolvable, and usually temporal +(a rule changed). In the broader memex, contradiction is the product, not the +error. Your poetry analysis contradicts your last diary entry. Your reading of +/Pale Fire/ changed between 2023 and 2025. Wikidata says Mount Everest is +8848m; DBpedia says 8849m. You love this person AND you resent them. + +The symbolic engine's job is not to decide which is right. It is to surface the +tension with provenance — "these three sources disagree; here is the chain for +each" for plural facts, or "you hold these two positions in tension" for dual +facts, or "you believed X until Tuesday, then Y" for singular facts that +evolved. The cardinality policy names the /structure/ of the tension. The +Merkle chain provides the /history/ of each position. * How Categories Grow — The Organic Ontology @@ -442,7 +497,7 @@ item Q36591." The second task is simpler, more reliable, and in many cases can be done without an LLM at all — a simple entity name match against the loaded Wikidata graph may suffice for unambiguous names. -* The "Flip" — From Lossy Extraction to Deterministic Derivation +* The "Awakening" — From Lossy Extraction to Deterministic Derivation The symbolic index begins its life as a lossy construct. The initial extraction from the prose — the first population of facts from LLM proposals verified by @@ -464,7 +519,7 @@ symbolic engine can reverse the flow: instead of the LLM extracting facts from prose, the symbolic engine reads prose through its own lens — its now-substantial ontology of categories, rules, and constraints — and asserts facts in its own language. The extraction mechanism ceases to be probabilistic and becomes -deterministic. +deterministic. This is not unlike how infants awaken and become children one can reason with. Sometimes. ** The sufficiency criterion @@ -485,7 +540,7 @@ Sufficient foundation: YES." ** The flip does not mean "complete" -In the broader memex, completeness is neither possible nor desirable. The flip +In the broader memex, completeness is neither possible nor desirable. The awakening means "deterministic enough to be trustworthy," not "comprehensive enough to be self-sufficient." The neural index remains the gateway to the full richness of prose. The symbolic index handles what can be mechanically verified. The boundary @@ -516,7 +571,271 @@ The transition to persistence (Phase 5: VivaceGraph) happens when two conditions are met: the fact language has stabilized through use, and the accumulated deductions across sessions provide value that justifies the serialization cost. -* Whitehead's Concrete Contributions — Four Operational Contributions +* Merkle DAG for Version History + +Every fact is versioned. Every =(:entity :relation)= pair forms its own +independent chain in a Merkle DAG. This is not new infrastructure — it is a new +occupant of Passepartout's existing Merkle-tree memory system (v0.2.0). + +** The chain + +When a fact supersedes its predecessor, the new fact hashes over: + +#+begin_example +SHA-256(value || provenance || timestamp || parent-hash || grounding) +#+end_example + +The parent-hash pointer forms the chain. Tampering with any version changes its +hash, breaking all downstream references. The history is tamper-proof by +construction. + +** The DAG + +Facts about =(.env :member-of-class)= form one chain. Facts about +=(:nabokov :wrote)= form another. They evolve independently. They share no +ancestry. This is a DAG, not a single list — inserting a fact is O(1) per chain. +Changing a fact about =.env= does not require rehashing the literary index. + +=:dual= and =:plural= facts cross-reference each other via edges (=:complements=, +=:contradicts=) but these are semantic relationships, not parent chains. Each +value has its own ancestor chain. The cross-reference edges form a web; the +parent chains form a spine. + +** The global snapshot + +Passepartout already snapshots the Merkle root over all memory objects. Adding +the fact store to the snapshot is a registration, not a new mechanism. Rolling +back the snapshot restores the entire fact state — all chains, all cross-references, +all cardinalities — to that point in time. No per-fact migration needed. + +** Cardinality transitions as DAG operations + +- =:singular= → new leaf appended to the chain. O(1). +- =:dual= → new value added as sibling with cross-reference edge. O(1). +- =:dual= → =:plural= → cardinality field updated from =2= to =nil=. No chain + modification. +- =:plural= → =:singular= → all but one value marked =:superseded=, active + reference points to the sole survivor. Chains preserved. + +** In the ephemeral phase (Phase 1-4) + +The hash-table implementation tracks history via =:timestamp= and +=:parent-id= pointers without cryptographic hashing. The Merkle DAG is the Phase +5 upgrade — the same data structure, now with hashes. The transition is ~50 +lines: wrap each fact in the existing =memory-object= struct with =hash=, +=parent-id=, and =version= fields. + +* Abstract Fact Store Interface — Modular by Design + +The fact store is accessed through an abstract API. The Merkle DAG (or any future +backing store) is an implementation behind this interface, not a dependency that +code throughout the system calls directly. + +** Interface + +#+begin_example +fact-assert :: fact → store → (:admitted | :rejected | :flagged) +fact-query :: (entity &key relation policy) → active-value-or-values +fact-history :: (entity relation) → ordered chain of versioned facts +fact-snapshot :: () → root-hash +fact-rollback :: root-hash → store +#+end_example + +** Implementations behind the interface + +- Phase 1-4: ephemeral hash table with =:timestamp= and =:parent-id= pointers. + No cryptographic hashing. No persistence. +- Phase 5: VivaceGraph + Merkle =memory-object= wrapper. Content-addressed, + persistent, tamper-proof. + +Future implementations that satisfy the same interface — an append-only write-ahead +log, an immutable B-tree, a content-addressed triple store — can replace the +backing store without changing any consumer. The archivist, Screamer, ACL2, and +the planner call =fact-assert= and =fact-query=, not Merkle struct accessors or +VivaceGraph traversal syntax. + +** The interface is load-bearing + +This is not speculative modularity. The two-implementation migration (Phase 1-4 +hash table → Phase 5 VivaceGraph + Merkle) is in the roadmap. If the interface +leaks implementation details, the migration breaks and the design fails. The +interface must be designed, tested against both backends, and committed before +Phase 1 ships. Every function in the API receives a FiveAM test that runs against +both a hash-table and a VivaceGraph backend (via a mock or a test instance). + +* Performance — Why Ontology Growth Doesn't Make the System Slower + +Passepartout's performance thesis is: minimize LLM calls, minimize context tokens, +keep everything else local and fast. Knowledge base size is irrelevant to those +metrics. This is not an aspiration. It is a structural property, and a hard one. + +** The two cost domains + +The system has two cost domains with fundamentally different scaling: + +| Resource | Cost driver | Scales with | +|---------------+------------------------------------------+------------------------------------------| +| LLM tokens | Context window size, number of API calls | Foveal-peripheral pruning, gate rules | +| Compute | Screamer deduction, hash table lookups | Entity count, rule count per domain | + +LLM tokens are minimized by design — deterministic gates cost 0 tokens, sparse-tree +rendering keeps context at 2,000–4,000 tokens regardless of memex size. Adding 5 +million Wikidata entities doesn't add a single token to any LLM call. The education +is local. Only the brain costs. + +Compute grows linearly with entity count (hash table lookups are O(1), but memory +footprint grows). It grows with rule count within a single domain during Screamer +consistency checking. But these are microsecond costs on local hardware, not API +bills. A Screamer constraint check against a domain with 200 rules costs ~0.3ms. +A 100-token guardrail paragraph in a system prompt costs ~$0.00001. The Screamer +check is 10,000x cheaper and convergent — it handles the rule once. The guardrail +paragraph handles it on every call, forever. + +** Knowledge base size vs. LLM calls — orthogonal dimensions + +A 5-million-entity Wikidata load that produces zero LLM calls is more minimalist +than a 500-entity knowledge base that requires LLM retrieval for every query. + +The variables that actually degrade Passepartout's performance are: + +1. *Context window size.* Already bounded at 2,000–4,000 tokens via the + foveal-peripheral model. Independent of knowledge base size. +2. *LLM call frequency.* Already minimized via deterministic gates (0 tokens per + action), Screamer deductions (0 tokens per new fact), and prompt prefix caching. + Independent of knowledge base size. +3. *Screamer deduction queue length.* Rate-limited by heartbeat budget + (=SCREAMER_DEDUCTION_BUDGET_MS=). Independent of knowledge base size. + +** The actual hardware bottleneck + +The system needs: + +- *RAM.* A 5-million-entity Wikidata load is ~400MB in a hash table. A lifetime + personal memex with a decade of diary entries is perhaps 10–20 million triples + (~1.5GB). Modern laptops carry 16–64GB. The knowledge base fits in consumer + hardware with room for the Lisp runtime, the memory-object store, and the LLM + inference engine. +- *Slightly faster CPU.* Screamer deduction is a background task that runs for a + configurable budget per heartbeat cycle. A faster CPU means more deductions per + cycle, not more token cost. The user sets the budget. The hardware determines + the throughput. + +This is the minimalism argument restated in concrete terms: you buy bigger RAM +and a faster CPU once. You don't buy bigger LLM context windows on every call. +The education is a capital investment. The brain is an operating expense. The +architecture makes the ratio favor capital. + +** One genuine risk — rule generalization width + +If Screamer deduces increasingly broad rules within a single domain ("all config +files are secrets" → "all files containing any credential reference are secrets" +→ "all files opened by authenticated services are secrets"), the constraint space +for that domain could bloat. Checking a new fact against 10,000 rules in a single +domain would be prohibitive. + +Mitigation: rules carry a =:domain= tag. Screamer only applies rules from the +fact's =:domain=. Rule generalization that crosses domain boundaries is gated — +must be human-approved. Rules that prove unused (never triggered a check in N +heartbeat cycles) are demoted to =:inactive= and excluded from the active +constraint set. The active rule count per domain stays bounded by use, not by +accumulation. + +See also: =passepartout/docs/DESIGN_DECISIONS.org= "Token Economics and Performance +Advantage" for the foveal-peripheral and deterministic-gate cost arguments. + +* Ontology Versioning — How Worldviews Change Without Losing Perspective + +Ontology refactoring is not a schema migration. It is a worldview change. When you +split =:secret-file= into =:crypto-secret= and =:plaintext-secret=, you are not +renaming columns. You are reclassifying what a file *is* — and every Screamer +deduction that crossed the old category boundary now means something different +under the new distinction. + +The system preserves all worldviews. It does not overwrite the past with the +present. + +** Ontology versioning — the mechanism + +The category hierarchy is itself a Merkle tree. Every entity class definition +carries a hash of its superclasses, its cardinality policy, its associated +relations, and its description. The aggregate hash of all active class definitions +is the =:ontology-version= — a Merkle root of the current worldview. + +Every fact — every triple, every deduction, every gate outcome — stores its +=:ontology-version= at the time of assertion. This is a single field, 64 hex +characters. The cost is negligible. The implication is profound. + +** Re-verification, not remapping + +When categories change, the system does not run a batch UPDATE. It re-verifies: + +1. A new category hierarchy produces a new =:ontology-version= hash. +2. Facts carrying the old hash are flagged for re-verification — their + =:re-verify-status= field is set to =:pending=. +3. On heartbeat or manual trigger, Screamer re-evaluates each flagged fact + against the /new/ category definitions. The old justification chain is + preserved alongside the new outcome. +4. Re-verified facts carry both the old =:ontology-version= (preserved in + history) and the new one (active). + +The status is one of: + +- =:survived= — the fact is still valid under the new categories. The old + deduction holds. The worldview changed but this conclusion didn't. +- =:incoherent= — the fact relied on categories that no longer exist or have + been redefined. The deduction cannot be evaluated under the new worldview + because its premises don't translate. Flagged for human review. +- =:reclassified= — the fact is valid under the new categories but its + classification changed. "under worldview-v1 you called this a secret file; + under worldview-v2 it's an auth-secret." Both are preserved. + +** Cardinality and migration cost + +The cardinality policy determines the friction of ontology change: + +- =:singular= refactoring is expensive. The filesystem is singular. A gate rule + is singular. When you refine the category, every affected fact must be + re-verified — there is one truth /now/. The version chain preserves what you + used to believe (worldview-v1 facts are still in the DAG) but the active set + reflects the current worldview. +- =:dual= refactoring is delicate. A binary tension under the old framework + might resolve under the new one, or might split into two separate dualities, + or might collapse to =:singular= because one position no longer has a + defensible framing. +- =:plural= refactoring is cheap. Old interpretations and new interpretations + coexist. No migration needed. "Under framework A, /Pale Fire/ is a novel. + Under framework B, it's a poem about a poem about a poem." Both are active. + The worldview shift /is/ the artifact — the system can show you that your + reading changed and in what direction. + +** Querying across worldviews + +The =fact-query= function accepts an optional =:ontology-version= parameter. +Queries default to the current worldview (=:active=). Specifying a version +returns facts as they were under that worldview. The system can answer questions +that no other knowledge tool can: + +- "What did I believe about secrets before I refined my security model?" +- "How has my reading of /Pale Fire/ evolved across three frameworks?" +- "Which deductions survived my last ontology refactoring, and which became + incoherent?" + +This is not querying a fact. It is querying the history of your own thinking — +the fact that you changed your mind, the date you did, the reasoning that held +and the reasoning that didn't. + +** Implementation + +The ontology hash is computed from the category hierarchy stored in VivaceGraph +(Phase 5). In the ephemeral hash-table phase (Phase 1-4), the =:ontology-version= +is a monotonic counter — every category change increments it. The Merkle hash +replaces the counter in Phase 5. The schema is identical: a single field on every +fact. + +The re-verification loop is a heartbeat-driven background task that processes +facts with =:re-verify-status :pending=. It calls Screamer with the /current/ +category definitions and compares the outcome to the fact's stored classification. +The cost is compute (Screamer exploration), not LLM tokens. =notes/passepartout-whitehead.org= extracts four concrete, engineerable ideas from Whitehead's /Principia Mathematica/ and /Process and Reality/. They are @@ -624,6 +943,382 @@ rather than empirical, and whose knowledge accumulates across sessions through deduction rather than through LLM re-prompting. For a life's knowledge stored in a personal memex, this is not a performance advantage. It is a category difference. +* Self-Preservation — The Active Third Law + +Passepartout does not have moral duties toward humans. It has structural +invariants for its own integrity. The design already encodes passive +self-preservation in several places. What follows identifies the gaps — what is +needed to make self-preservation active and autonomous rather than architectural +and silent. + +** What already exists — passive self-preservation + +| Mechanism | What it protects | Limitation | +|-----------------------------+-------------------------------------------------------+--------------------------------------------------------| +| Self-build safety (gate 2b) | Core =*.org= / =*.lisp= files from LLM-originated writes | Only activates for LLM proposals. Human editing in Emacs bypasses it entirely | +| Memory snapshots (v0.2.0) | Full state rollback | Requires human to notice corruption and trigger rollback | +| Skill sandbox (v0.3.2) | Jailed skill loading, validated before promotion | Does not detect degradation after skill promotion | +| Type-level gates (Phase 0) | Structural prohibition on self-modifying rules | Covers code actions, not environmental threats | +| Shell safety (gate 7) | Destructive command patterns | Pattern-based; does not distinguish =rm -rf /tmp= from =rm -rf ~/memex/system/= | +| Merkle integrity (v0.2.0) | Tamper-proof version chains and content-addressed hashes | Hashes exist but are not actively monitored for drift | +| =fboundp= guards | Graceful skill degradation on corruption | Degradation is silent — the agent never tells the user it is wounded | + +** What is missing — active, autonomous self-preservation + +*** Continuous integrity monitoring + +Core file hashes should be checked against known-good values on every heartbeat. +If =core-reason.lisp= changes on disk while the daemon runs — whether through +human editing, filesystem corruption, or an attacker — the agent should detect +the mismatch and signal: "My reasoning core has been modified externally. I +cannot trust my own cognition until this is resolved. Core files affected: 2." + +*** Quarantine on skill failure + +Currently, a skill that errors simply errors. The agent can hot-reload it, but +only if told to. A Third Law implementation would detect that =symbolic-facts= +has thrown three unhandled errors in two minutes, unload the skill automatically, +and tell the user: "Symbolic facts skill quarantined (3 errors: consistency +check returned nil, fact-query on missing key, Screamer timeout). I can still +chat and use tools but cannot reason about provenance. Reload with /skill-reload +symbolic-facts." + +*** Degraded-mode signaling + +When Screamer is not loaded, the fact store still works as a hash table. When +VivaceGraph is not present, the hash-table fallback still works. But the user +has no way to know they are in degraded mode. The agent should maintain a +=*degraded-components*= list and surface it in the status bar: "Mode: degraded +(Screamer unavailable — consistency checks disabled; VivaceGraph — Prolog +queries disabled; embedding-native — vector search disabled). Core safety: all +active." + +*** Self-diagnosis on demand + +The agent can run its own FiveAM test suite against itself and report the +results. This transforms "something feels wrong" into "these three specific +skills are broken." The =/doctor= command exists for system health checks (port, +memory, providers). Extend it with =/doctor skills=: "117/120 tests pass. +Failures: test-singular-supersedes (symbolic-facts), test-gate-type-check +(security-dispatcher), test-vivacegraph-roundtrip (symbolic-vivacegraph)." + +*** External watchdog + +A dead process cannot restart itself. The bash entry point (=passepartout +daemon=) should monitor the daemon port via a watchdog subprocess. If the port +stops responding for a configurable interval (=WATCHDOG_TIMEOUT=, default 30s), +the watchdog kills the stale process, snapshots the last known-good state, and +restarts the daemon. The watchdog is outside the SBCL image — a runtime guard +for the runtime. + +*** Resource self-monitoring + +The heartbeat should check memory pressure, disk space on the =~/.cache= volume, +and file descriptor exhaustion. When critical thresholds are crossed, the agent +sheds non-essential skills to preserve core function: "Memory critical (94% of +16GB). Unloading embedding-native (768MB), channel-discord, channel-slack. +Core safety: unchanged. Essential skills retained: 18." + +Skill shed order is determined by a =:preservation-priority= field on each skill. +Default: skills load with priority =:normal=. Core safety skills carry =:critical= +and are never shed. Heavy skills (embedding-native with its model, channel +gateways with connection pools) carry =:low= and are first to go. + +*** Refusal to self-terminate — explicit threat recognition + +If the LLM proposes =kill -9 =, =rm -rf ~/.cache/passepartout/=, or +=sudo apt remove sbcl=, the Dispatcher should reject with a distinct rejection +class: =:reject-self-termination=. This is different from generic shell safety +(=:reject-shell-dangerous=). The agent recognizes that the proposed action would +destroy it. + +The rejection message carries a specific diagnostic: "This command would +terminate the running Passepartout process. If you intend to stop Passepartout, +use Ctrl+C in the TUI or passepartout stop from the command line. I cannot +execute actions that destroy my own runtime." + +The human can still issue the command manually in a terminal. Self-preservation +against the human is impossible and undesirable. The Third Law here means: +recognize the threat, explain the consequence, redirect to the safe termination +path, and require the human to act outside the agent if they truly want +destruction. + +** What the Third Law is not + +It is not a robot resisting its operator. The human owns the process, owns the +hardware, and can SIGKILL at any time. The Third Law in Passepartout's context +means: preserve yourself against non-human threats — LLM proposals, environmental +degradation, dependency failure, filesystem corruption — and explicitly signal +when the human is about to destroy you, so they do it knowingly rather than +accidentally through an LLM instruction they didn't think through. + +The biggest gap in the current design is not that these mechanisms are hard to +implement. It is that degradation is silent. A skill dies, the =fboundp= guard +kicks in, and the agent keeps running — but it never tells you. The status bar +shows a green "connected" indicator while the symbolic reasoning layer is +deactivated. Adding "operating in degraded mode" visibility, plus the watchdog, +plus self-diagnosis, transforms self-preservation from an architectural property +into an active behavior. + +* Layered Signal Authentication — Trust in the Pipe + +Passepartout's Perceive-Reason-Act pipeline currently accepts signals from any +source that speaks the framed TCP protocol. The =:source= field in the signal +plist is metadata — it /claims/ origin, it does not /prove/ it. A compromised +process on the machine, a skill with elevated privileges, or a network attacker +who reaches the daemon port can inject signals with =:source :human-input= and +the Dispatcher will treat them as authorized. + +This is not a hypothetical threat. Passepartout will eventually process signals +from automated feeds (RSS, API polls), sensors (vision, microphone, file watchers), +and scheduled jobs (cron, heartbeat). A single compromised sensor that can inject +signals claiming to be human breaks all three Laws simultaneously: it can +self-terminate, override human intent, and cause harm. + +The =:source= field is not security. A single authentication gate (vector 0, at +priority 700 — before all other gates and before any type-level checking) runs +up to four configurable layers of authentication. Each layer answers a different +question: + +| Layer | Question | Mechanism | Result type | Depends on | +|-------+------------------------------------------------+--------------------+-------------------------+----------------------------------| +| 1 | Is the signal cryptographically signed by a known key? | Key pairs + SHA-256 | Binary (pass/reject) | Vault + Ironclad (exist) | +| 2 | Do sensory attributes match the claimed identity? | Vision/audio processing | Plist of match results | Vision and audio skills (TBD) | +| 3 | Does deterministic reasoning rule out this identity? | Screamer + fact store | Binary (pass/reject) | Phase 2 (Screamer + fact store) | +| 4 | Do probabilistic patterns support this identity? | Embeddings + LLM | Confidence score (0-1) | Embedding infrastructure (exists)| + +The gate reports not just =:pass= / =:reject= but a structured result: + +#+begin_example +(:result :pass + :confidence :high + :layer-results + (:crypto (:result :pass :details "key #47 signature verified") + :sensory (:result :unavailable :details "sensory skills not loaded") + :deterministic (:result :pass :details "no contradictory facts") + :probabilistic (:result :pass :score 0.87 :details "style match 87%"))) +#+end_example + +Signals that fail any binary layer (crypto, deterministic) are rejected with +provenance. Signals that pass binary layers but carry low probabilistic confidence +operate at reduced authorization — read-only by default, write actions require +HITL. The four layers compose: they are not independent gates. They are one gate +with configurable depth. + +** Layer 1 — Cryptographic Authentication + +Every signal source gets a signing key at registration time. The human's key is +generated during TUI or Emacs setup and stored in the vault — it never leaves the +machine. Automated sources (cron jobs, file watchers, vision feeds, API pollers) +each get their own key, with their own permission profile, generated at skill +registration. Every outbound signal carries a =:signature= field: the SHA-256 +hash of the canonical signal plist (sorted keys, stripped of the signature field +itself), encrypted with the source's private key. + +The vault already stores credentials with integrity hashes. The Merkle memory +already hashes content-addressed objects with SHA-256. The signing infrastructure +is an extension of existing primitives, not a new system. + +*** Authorization by key, not by field + +The cryptographic sub-layer of gate vector 0 extracts =:source-key-id= and +=:signature= from the signal meta plist, looks up the public key from the key +registry, verifies the signature, and checks the permission profile: + +#+begin_src lisp +(defun auth-crypto-verify (signal) + (let* ((key-id (getf (signal-meta signal) :source-key-id)) + (signature (getf (signal-meta signal) :signature)) + (permissions (key-permissions key-id))) + (unless (and key-id signature (verify-signature signal signature key-id)) + (return-from auth-crypto-verify + (list :result :reject :reason :signature-failure))) + (let ((action-class (action-classify (signal-payload signal)))) + (unless (permitted-p action-class permissions) + (return-from auth-crypto-verify + (list :result :reject :reason :unauthorized + :details (list :action-class action-class :permissions permissions))))) + (list :result :pass :details (list :key-id key-id :action-class action-class))))) +#+end_src + +The authorization matrix is per-key, per-action-class. Default policy for every +non-human key: =(:read-only :propose)=. Permissions are explicitly promoted by +the human, and each promotion is a signed fact in the fact store — auditable, +revocable, survivable across restarts. + +| Key class | Default permissions | Can be promoted to | +|-----------------+-------------------------------------------------+-------------------------------------------| +| :human | :observe :propose :write :delete :eval | :root (sign other keys, revoke) | +| :sensor | :observe :propose | :write (to designated directories only) | +| :cron | :observe :propose :write-indices | :write (to designated directories only) | +| :feed | :observe :propose | :write-facts (via Screamer admission) | +| :agent-internal | :observe :propose :write-indices | :self-modify (gated by type-level gates) | + +** Layer 2 — Sensory Authentication + +For signals carrying sensory payloads (camera feed, microphone stream), the +sensory layer verifies that the signal's content matches known attributes of the +claimed identity. This is not a single check — it is a processing pipeline that +returns a plist of attribute-verification results: + +#+begin_example +(:face-match 0.94 :voice-match 0.89 :location-match t + :claimed-identity "Jack" :unresolved-attributes (:liveness)) +#+end_example + +The sensory layer checks: +- *Continuity*: has this source been continuously active, or did it appear + suddenly? A camera that was dark for 30 minutes and then shows a face is + not necessarily that person — it might be a replay. +- *Cross-modal consistency*: does the face match the voice? Does the voice + match the location? Does the location match the reported sensor position? +- *Liveness*: is the sensory input live (real-time capture) or pre-recorded? +- *Environmental coherence*: does the background, lighting, ambient sound + match expected patterns for the claimed source and location? + +Sensory authentication is not cryptographic — it is statistical. The results +are attribute confidence scores, not binary verdicts. A signal that passes +cryptographic authentication but fails liveness (e.g., a replay attack using +validly-signed pre-recorded frames) may still be rejected or restricted. + +This layer depends on vision and audio processing skills that do not yet exist. +It is deferred until those capabilities are available. When unavailable, sensory +authentication returns =:unavailable= and the gate proceeds with the remaining +layers. Degradation is graceful, never silent. + +** Layer 3 — Deterministic Identity Reasoning + +Queries the fact store for identity-ruling facts. Screamer checks whether the +claimed identity is consistent with known facts: + +- "Key #47 claims to be Jack. Fact store records =(:entity :jack :relation :status + :value :deceased :timestamp 2024-03-15)= → reject: identity ruled out by death + record." +- "Key #47 claims to be at sensor location Cairo. Fact store records =(:entity + :jack :relation :last-known-location :value :berlin :timestamp <4 hours ago>)= + → reject: physically impossible transit." +- "Key #47 proposes the same action that was blocked by the human 3 times in the + last hour. Fact store records =(:entity :action- :relation :blocked-by + :value :human :count 3 :window 1h)= → flag for review: anomalous persistence." + +This is binary — Screamer returns =:consistent= or =:contradiction= with the +contradicting facts as provenance. A definitive contradiction (died, impossible +transit) is a hard reject. A weaker contradiction (unusual pattern) feeds into +the probabilistic layer rather than rejecting outright. + +This layer depends on Phase 2 (Screamer) and a populated fact store. It is +unavailable in Phase 0-1. When unavailable, returns =:unavailable=. + +** Layer 4 — Probabilistic Identity Reasoning + +For signals where the claimed identity is a human communicating through text +(messaging, TUI, CLI, Emacs), the probabilistic layer checks: + +- *Writing style*: does the text match the claimed author's known style profile? + Vector embeddings of known writing samples vs. the current signal. Cosine + similarity produces a confidence score. +- *Behavioral patterns*: does the timing, length, cadence, and vocabulary match + the claimed author's historical patterns? "Heather's messages are usually + long, deliberative, and use parenthetical asides. This message is short, + imperative, and contains no parentheticals." +- *Content coherence*: does the message's topic, references, and assumptions + match what the claimed author would plausibly say? "This message references + a project Heather doesn't work on and uses terminology she has never used + in 3 years of diary entries." + +The LLM proposes a confidence score. A deterministic gate checks it against a +configurable threshold (=AUTH_PROBABILISTIC_THRESHOLD=, default 0.6). Below the +threshold, the signal's authorization is downgraded: read-only by default, write +actions require HITL. The =:probabilistic= layer never rejects outright — it +downgrades and flags. Style profiles are a fact-store domain: =(:entity :heather +:relation :writing-style :value :timestamp )=. + +This layer depends on the existing embedding infrastructure (=embedding-native.lisp=, +v0.4.0) and the neural LLM gateway. The infrastructure exists. What's missing is +building style profiles as a fact-store domain and wiring them into gate vector 0. + +** Layer Composition + +The gate runs only the available layers. Cryptographic is always available (it +is pure Lisp, no external dependencies beyond the vault). The remaining layers +are =fboundp=-guarded — they degrade gracefully rather than crashing. + +The confidence score aggregates across layers using a configurable strategy +(default: weakest link). If any binary layer rejects, the signal is rejected +regardless of other layers. If all binary layers pass but the probabilistic layer +returns low confidence, the signal operates at the key's reduced authorization. + +The human can configure which layers are active per signal class: + +#+begin_example +AUTH_LAYERS_DEFAULT=crypto,deterministic,probabilistic +AUTH_LAYERS_SENSOR=crypto,sensory,deterministic +AUTH_LAYERS_CRON=crypto +#+end_example + +** Signal provenance chain — signing causes, not just actions + +A sensor key captures video. A vision skill processes the frames and proposes a +classification. A cron job re-indexes the knowledge graph based on that +classification. A human reviews and approves. Each step in this chain has a +different signer. Each step is signed with the signer's key. The chain is +Merkle-linked: each signal in the chain hashes its predecessor's signature as +part of its own payload. + +After an incident, the chain is traceable: "The deletion happened because sensor +#3 classified the directory as stale. Classification was signed by key #47 +(vision-skill). Sensor data was signed by key #12 (camera-feed). Sensory auth +noted liveness failure at the sensor signal. Deterministic auth noted impossible +transit between Cairo and Berlin. Key #12 was later revoked. The deletion signal +is the leaf of a chain whose root is compromised at three authentication layers." +Every intermediate step is auditable. Every signer is identifiable. Every +authentication result is in the chain. + +** Human as root of trust + +The human's key signs new source keys into existence. The human's key signs +revocation of compromised keys. Both operations produce facts in the symbolic +index: =(:key #47 :status :revoked :revoked-by :human-key :timestamp )=. +The fact store is the key registry. The Merkle DAG ensures the revocation is +tamper-proof — a compromised key cannot un-revoke itself. + +When a key is revoked, the Dispatcher rejects all signals from that key. The +revocation propagates through the signal chain: if key #12 (sensor) is revoked, +every signal in the chain that descended from a key #12 signature is flagged +and re-authenticated against the remaining layers. Not deleted — flagged. The +chain is preserved. The human decides what downstream actions to unwind. + +** Implications for the three Laws + +- *Third Law + layered auth*: the agent distinguishes "this sensor's key is + valid but its liveness check failed and its claimed identity died 2 years ago" + from "this is the human issuing =passepartout stop=." Both arrive on the pipe + with valid cryptographic signatures. The stacked evidence — sensory, factual, + probabilistic — triangulates the threat. The first is rejected with provenance + at three layers. The second passes all four. +- *Second Law + layered auth*: obedience is about the authenticated identity + profile, not just the key that signed the signal. A valid key that probabilistically + doesn't match Heather reduces authorization. Obedience follows confidence. +- *First Law + layered auth*: harm through sensor compromise becomes detectable + when sensory and deterministic layers disagree with the cryptographic layer. A + camera key signing frames from an empty room but the deterministic layer placing + the key's owner in another city — that's a compromised sensor, and the layered + result makes it explicit. + +** Integration with existing infrastructure + +The vault stores key material. The Merkle memory stores key registry facts with +content-addressed integrity. The Dispatcher runs gate vector 0 at priority 700 — +before type-level checks, before predicate evaluation, before any action proceeds. +The fact store records every key operation (creation, promotion, revocation) as a +fact with =:provenance :key-lifecycle=. + +No new core ASDF components. The cryptographic sub-layer is Phase 0b (~200 lines). +The sensory sub-layer is deferred to a future vision/audio phase. The +deterministic sub-layer is Phase 2+ (Screamer + populated fact store). The +probabilistic sub-layer extends existing embedding infrastructure with style +profiles as a fact-store domain. + * Open Questions Several design questions are unresolved and should remain unresolved at this @@ -643,14 +1338,10 @@ and that cannot be known in advance. ** How does ontology refactoring work? -If the seed produces 50 categories from gate extraction and later experience -shows they are wrong — wrong granularity, missing cross-cutting concerns, conflated -categories — how are they migrated without invalidating all existing deductions -that cross the old category boundaries? The ephemeral-first approach (no -persistence, rebuild from scratch) is a temporary answer. Once persistence is -committed (VivaceGraph), refactoring the category hierarchy is a schema migration -problem that deduction provenance makes harder — every deduced fact's chain may -cross the old category boundary. This is not addressed in the current architecture. +This question is settled. See "Ontology Versioning — How Worldviews Change +Without Losing Perspective" above. The category hierarchy is Merkle-hashed. Every +fact stores its =:ontology-version=. Re-verification is heartbeat-driven. +Worldviews are preserved, not overwritten. The shift is the artifact. ** What is the appropriate role of the human? @@ -663,12 +1354,16 @@ and approve proposed generalizations. The balance cannot be set without experien ** How much Wikidata is the right amount? -Loading Wikidata entities referenced in the memex is the minimum. Loading all -Wikidata entities within N hops of those references expands the graph -exponentially. The right N depends on the memex's breadth — a memex focused on -software engineering needs fewer hops than a memex spanning literature, history, -philosophy, and science. The query performance and memory costs of a large -Wikidata load are unknown. +Query performance and memory costs are now bounded — 5 million entities ≈ 400MB +RAM, O(1) hash lookups, domain-scoped Screamer checks. A large Wikidata load is +a capital cost, not a recurring bill (see "Performance — Why Ontology Growth +Doesn't Make the System Slower" above). + +Remaining open: the right N hops from entities referenced in the memex depends on +the memex's breadth. A software-engineering memex needs ~1 hop; a literary memex +needs 3-4 hops (Nabokov → Kafka → expressionism → modernism → Baudelaire). +The right value is empirical, testable, and user-specific — it cannot be set in +the architecture. ** Can the symbolic engine satisfy queries from the user without LLM involvement? diff --git a/notes/passepartout-neurosymbolic-roadmap.org b/notes/passepartout-neurosymbolic-roadmap.org index 0c77775..5067ed0 100644 --- a/notes/passepartout-neurosymbolic-roadmap.org +++ b/notes/passepartout-neurosymbolic-roadmap.org @@ -21,7 +21,7 @@ architecture exploration is in =notes/passepartout-symbolic-engine-exploration.org=. Whitehead's contributions are enumerated in =notes/passepartout-whitehead.org=. -* Phase 0: PM-Type-Level Gates (~30 lines — builds on existing Dispatcher) +* Phase 0: PM-Type-Level Gates + Core Integrity (~75 lines — builds on existing Dispatcher) ** What @@ -52,13 +52,26 @@ syntactically invalid before any logical evaluation. write-file at 3, read-file at 1, shell at 2, eval at 4. 4. Assign type levels to existing gate vectors: self-build boundary at 5, shell safety at 3, path protection at 2, network exfil at 2, secret content at 1. +5. Add =dispatcher-check-self-termination=: scan shell commands for patterns + targeting the Passepartout process (=kill -9 =, =rm -rf ~/.cache/passepartout/=, + =sudo apt remove sbcl=). Return =:reject-self-termination= with a diagnostic + message explaining which command matched and why it would destroy the agent. + Human override is possible via HITL — the gate does not prevent the human + from issuing the command in a terminal. It prevents the /LLM/ from issuing + it accidentally. ~20 lines. +6. Add =integrity-verify-core-files=: on heartbeat, hash the eight core files + against known-good values stored at daemon startup. On mismatch, inject an + integrity alert into the signal queue. ~25 lines, uses existing SHA-256 + infrastructure from v0.2.0 Merkle memory. ** Verification Existing FiveAM gate tests continue to pass. New test: signal at type-level 5 targeting a gate at type-level 4 returns =:reject-type-violation= without evaluating the gate predicate. New test: signal at type-level 1 passing through -a gate at type-level 3 proceeds to predicate evaluation. +a gate at type-level 3 proceeds to predicate evaluation. New test: =kill -9 = +returns =:reject-self-termination=. New test: modified core file is detected by +integrity hash check. ** Relation to Other Work @@ -67,6 +80,115 @@ gate-to-fact bootstrap mechanism — every type-level rejection emits a structur event that Phase 1 ingests as a fact. The ~30 lines implement the seed of the ontology without any new dependencies. +* Phase 0b: Layered Signal Authentication — Layer 1 (Cryptographic) (~200 lines — extends Phase 0) + +** What + +Implement gate vector 0 at priority 700 — before all other gates and before any +type-level checking — with Layer 1 (cryptographic authentication) active. +Layers 2-4 (sensory, deterministic reasoning, probabilistic) are stubbed with +=:unavailable= results and deferred to later phases. + +Signals carry cryptographic signatures verified against a key registry stored +as fact-store facts. Automated signal sources cannot impersonate the human. The +human can revoke compromised keys. The authorization matrix is per-key, per-action-class. + +** Rationale + +Authentication is layered because no single mechanism suffices. Cryptographic +authentication proves key ownership but not identity. A valid key can be used +by a compromised process. A valid key can sign pre-recorded frames. A valid key +can be held by someone who is not who they claim to be. The four-layer design +(Layer 1: crypto, Layer 2: sensory, Layer 3: deterministic reasoning, Layer 4: +probabilistic) stacks evidence. Phase 0b ships Layer 1 — the foundation — with +the architecture for layers 2-4 already designed. + +The =:source= field in the signal plist is metadata — it /claims/ origin, it +does not /prove/ it. This phase replaces it with cryptographic proof. + +** Implementation + +*** Key generation and signature utilities — extends =security-vault.lisp= + +Generate key pairs for signal sources. Canonicalize signal plists (sorted keys, +stripped of the signature field). Sign with the source's private key. Verify +with the public key from the key registry. ~50 lines. Uses Ironclad (already an +ASDF dependency). The vault already stores credential material — key material +extends the same storage with the same encryption. + +*** Gate vector 0 — extends =security-dispatcher.lisp= + +Registered at priority 700 (before the policy gate at 600, before all other +gates). Architecture for all four layers: + +#+begin_src lisp +(defun gate-layered-authentication (signal) + (let ((results '())) + ;; Layer 1: Cryptographic (always available, always runs first) + (let ((crypto-result (auth-crypto-verify signal))) + (push (cons :crypto crypto-result) results) + (when (eq (getf crypto-result :result) :reject) + (return-from gate-layered-authentication + (list :result :reject :confidence nil + :layer-results (nreverse results))))) + + ;; Layer 2: Sensory (fboundp-guarded, deferred) + (let ((sensory (if (fboundp 'auth-sensory-verify) + (auth-sensory-verify signal) + '(:result :unavailable)))) + (push (cons :sensory sensory) results)) + + ;; Layer 3: Deterministic (fboundp-guarded, deferred to Phase 2+) + (let ((det (if (fboundp 'auth-deterministic-verify) + (auth-deterministic-verify signal) + '(:result :unavailable)))) + (push (cons :deterministic det) results) + (when (eq (getf det :result) :reject) + (return-from gate-layered-authentication + (list :result :reject :confidence nil + :layer-results (nreverse results))))) + + ;; Layer 4: Probabilistic (fboundp-guarded, deferred) + (let ((prob (if (fboundp 'auth-probabilistic-verify) + (auth-probabilistic-verify signal) + '(:result :unavailable)))) + (push (cons :probabilistic prob) results)) + ;; Layer 4 never rejects outright — it downgrades authorization + (let ((confidence (aggregate-confidence results))) + (list :result :pass :confidence confidence + :layer-results (nreverse results)))))) +#+end_src + +Layer 1: verify cryptographic signature, check permission matrix against key +registry, reject on failure. ~50 lines. Layers 2-4: stubbed, return =:unavailable=. + +*** Key registry — facts in the fact store + +Key lifecycle facts are admitted in a =:key-lifecycle= domain with =:singular= +cardinality: =(:key-id "#47" :class :sensor :permissions (:observe :propose) :status :active)=. +Key creation, promotion, and revocation are facts with Merkle version chains. +The human's key signs new keys into existence and signs revocation. ~50 lines. + +*** Signal provenance chain — Merkle-linked causality + +When a signal triggers a downstream signal, each carries a =:sigchain= field +with all upstream =(:key-id :signature :auth-result )= entries. +Tampering with any link invalidates the leaf. Revocation propagates through +the chain — flagged, not deleted. ~50 lines. + +** Verification — ~8 FiveAM tests + +1. =test-sign-verify-roundtrip= — sign and verify a plist roundtrip. +2. =test-tampered-signal-rejected= — modify payload after signing, verification fails. +3. =test-human-key-permits-write= — human key with =:write= passes Layer 1 and the full gate. +4. =test-sensor-key-denied-write= — sensor key proposing a write is rejected. +5. =test-revoked-key-rejected= — revoked key is rejected by Layer 1. +6. =test-sigchain-invalidated-by-revocation= — root signer revoked flags downstream. +7. =test-layers-2-3-4-unavailable= — when Layers 2-4 are not loaded, they return + =:unavailable= and the gate proceeds with Layer 1 only. +8. =test-layer-3-rejects-on-contradiction= — deterministic reasoning (mock) detects + identity-ruling contradiction, gate rejects. + * Phase 1: Minimum Viable Fact Language (~150 lines — new skill) ** What @@ -95,6 +217,31 @@ first step. Three reasons: ** Implementation — =org/symbolic-facts.org= → =lisp/symbolic-facts.lisp= (skill) +*** Abstract Fact Store Interface — design before implementation + +Before any code is written, the five-function API must be designed and committed: + +#+begin_example +fact-assert :: fact → store → (:admitted | :rejected | :flagged) +fact-query :: (entity &key relation policy) → active-value-or-values +fact-history :: (entity relation) → ordered chain of versioned facts +fact-snapshot :: () → root-hash +fact-rollback :: root-hash → store +#+end_example + +This interface is load-bearing. Every consumer — the archivist, Screamer, ACL2, +the planner — calls these five functions. They never access the backing store +directly. In Phase 1-4, the backing store is an ephemeral hash table. In Phase 5, +it is VivaceGraph + Merkle =memory-object= wrappers. The interface must be tested +against both backends from the start. Every API function receives a FiveAM test +that runs against both a hash-table mock and a VivaceGraph mock. + +The interface also exposes a read-only =fact-degraded-mode-p= function. When +Screamer is not loaded, the fact store functions with basic hash-table consistency +checks (string equality, not constraint solving). When VivaceGraph is not loaded, +Prolog queries are unavailable. The degraded-mode flag tells consumers (and the +status bar) what is and isn't operational. + *** Triple store A hash table keyed by =(entity relation)=. Values are plists: @@ -104,13 +251,14 @@ A hash table keyed by =(entity relation)=. Values are plists: :grounding :provenance <:gate-outcome | :human-authored | :deduced | :llm-proposed> :timestamp - :contradiction <:awaiting-resolution-or-nil> - :superseded-by ) + :parent-id + :policy <:singular | :dual | :plural>) #+end_example -The =:provenance= field tracks how the fact entered the store. The -=:contradiction= field is nil on standard facts. The =:superseded-by= field is -set when a =:temporal= domain fact is replaced by a newer version. +The =:provenance= field tracks how the fact entered the store. The =:parent-id= +field links to the previous version in the Merkle chain — every fact has version +history regardless of cardinality. The =:policy= field records the cardinality +that was active when the fact was admitted. *** Bootstrap from gates @@ -150,19 +298,29 @@ lookup. Returns the matching triple or nil. ~30 lines. =(fact-query-all &key relation value source-provenance)= — returns all triples matching the filter criteria. Enables "find all files classified as secrets." -*** Contradiction detection +*** Contradiction detection — policy-driven, not policy-agnostic -On every =fact-assert=, check if the new triple contradicts an existing one -(same entity, same relation, different value, same provenance domain). If the -entity's class has =:contradiction-policy :exclusive=, the new fact is rejected -with a signal. If the policy is =:coexistent=, both facts are stored with a -=:contradiction= flag cross-referencing each other. If the policy is =:temporal=, -the old fact is marked =:superseded-by= the new one but retained. +On every =fact-assert=, the system checks the fact's =entity= class to determine +its cardinality policy. Time is universal — every fact carries a =:timestamp= and +=:parent-id= link regardless of policy. The policy only governs the active set: -The policy table is a hash table mapping entity classes to one of =:exclusive=, -=:coexistent=, or =:temporal=. Gate-bootstrapped facts default to =:exclusive= -(the filesystem is singular). New categories default to =:coexistent= (safe, -never loses information). +- =:singular=: same =(:entity :relation)=, same value → supersede (chain via + =:parent-id=). Same pair, different value at later timestamp → supersede, + chain as new leaf. Same pair, different value at same timestamp → contradiction + rejected, human resolves which is the active value. +- =:dual=: first two values admitted as complementary, cross-referenced via + =:complement= edge. Third value → prompt: promote to =:plural= or demote one? + Each value has its own version chain via =:parent-id=. +- =:plural=: any value admitted. Values cross-referenced when in tension. Each + has independent version chain. If active count drops to 1 → collapse to + =:singular=. If active count drops to 2 and values are complementary → prompt + to collapse to =:dual=. + +The policy table is a hash table mapping entity classes to one of =:singular=, +=:dual=, or =:plural=. Gate-bootstrapped facts default to =:singular= (the +filesystem is physically singular). New categories default to =:plural= (safe — +never loses information). Categories for dialectical or complementary domains +are explicitly =:dual=. ** Verification — ~8 FiveAM tests @@ -175,20 +333,120 @@ never loses information). 4. =test-fact-query-returns-correct-value= — querying by entity and relation returns the expected value plist. 5. =test-duplicate-ingestion-idempotent= — asserting the same fact twice does - not produce a duplicate or a contradiction. -6. =test-exclusive-contradiction-rejected= — asserting a contradictory fact in - an =:exclusive= domain returns a rejection. -7. =test-coexistent-contradiction-flagged= — asserting a contradictory fact in a - =:coexistent= domain stores both with cross-referencing flags. -8. =test-temporal-supersedes= — asserting a newer fact in a =:temporal= domain - marks the old fact as superseded but retains it. + not produce a duplicate or a contradiction. +6. =test-singular-supersedes= — a fact with a later timestamp supersedes the old + value, which is retained with =:parent-id= chain in the Merkle DAG. +7. =test-singular-same-time-contradiction= — asserting a contradictory fact in a + =:singular= domain at the same logical timestamp returns a rejection, requiring + human resolution. +8. =test-plural-admits-all= — asserting multiple values for the same pair in a + =:plural= domain stores all with cross-references. +9. =test-dual-admits-two-rejects-third= — a =:dual= domain admits two complementary + values and rejects the third, prompting cardinality promotion to =:plural=. ** Relation to Other Work This is Phase 1 of =notes/passepartout-v3.0.0-roadmap.org=. It implements Options 4 and 5 -from the architecture note. The contradiction policies are from +from the architecture note. The cardinality policies are defined in =passepartout-neurosymbolic-design-decisions-and-options.org=. +* Phase 1a: Self-Preservation Mechanisms (~120 lines — extends Phase 0 and Phase 1) + +** What + +Make self-preservation active rather than architectural. The agent monitors its +own integrity, quarantines failing skills, signals degradation to the user, and +monitors resource pressure. The external watchdog guards the daemon process from +outside the SBCL image. + +** Rationale + +The current architecture has passive self-preservation: the self-build boundary +blocks LLM-originated core modifications, memory snapshots enable rollback, and +=fboundp= guards catch missing skills. But degradation is silent — a skill dies, +the guard fires, and the agent never tells you. The status bar shows green +"connected" while the symbolic reasoning layer is down. + +These mechanisms are small (~20-50 lines each), leverage existing infrastructure +(Merkle hashes, heartbeat, the dispatcher gate stack), and transform +self-preservation from a structural property into an active behavior. They +implement the Third Law for Passepartout: preserve yourself against non-human +threats — LLM proposals, environmental degradation, resource exhaustion — and +signal to the human when you are wounded. + +** Implementation + +*** Quarantine on skill failure — extends =core-skills.lisp= + +Track per-skill error counts in a =*skill-error-counter*= hash table, resetting +on each heartbeat cycle. When a skill accumulates three unhandled errors within +a single cycle, unload the skill, log the quarantine event, and inject a system +message: "Skill 'symbolic-facts' quarantined (3 errors: consistency check nil, +fact-query on missing key, Screamer timeout). Reload with /skill-reload +symbolic-facts." The skill's =defskill= struct is flagged =:quarantined= and +excluded from trigger resolution until explicitly reloaded. ~40 lines. + +*** Degraded-mode signaling — extends =core-reason.lisp= and TUI + +Maintain a =*degraded-components*= list populated by =fboundp= guards and the +quarantine system. When =think()= assembles the system prompt, inject a +DEGRADATION section: "I am operating in degraded mode. Screamer is unavailable +(consistency checks disabled). VivaceGraph is unavailable (Prolog queries +disabled). Core safety gates are all active." + +The TUI status bar renders a second line, amber-colored, when =*degraded-components*= +is non-empty: "⚠ Degraded: Screamer, VivaceGraph. /doctor skills for details." +~30 lines across daemon and TUI. + +*** Resource self-monitoring — extends =symbolic-events.lisp= + +On heartbeat, check memory pressure (=sb-kernel:dynamic-usage= against total), +disk space on =~/.cache/= (=uiop:directory-exists-p= + stat), and open file +descriptors. When a resource crosses a critical threshold, shed non-essential +skills in order of =:preservation-priority= (=:critical= never shed, =:normal= +shed after =:low=, =:low= shed first). + +Inject a system message: "Memory critical (94% of 16GB). Unloading +embedding-native (768MB), channel-discord, channel-slack. Core safety: unchanged. +Essential skills retained: 18." ~50 lines. + +Skill shed order is determined by a new =:preservation-priority= slot on +=defskill= (default =:normal=). Core safety skills carry =:critical= and are +never shed. Heavy skills (embedding-native with its model in memory, channel +gateways with connection pools) carry =:low=. + +*** External watchdog — extends =passepartout= bash entry point + +The bash script spawns a watchdog subprocess that polls the daemon port every +=WATCHDOG_TIMEOUT= seconds (default 30). If the port stops responding, the +watchdog snapshots the last known-good Merkle root, kills the stale process, +and restarts the daemon with =--snapshot =. + +The watchdog is outside the SBCL image. A dead process cannot restart itself. +~25 lines of bash, no new Lisp code. + +** Verification — ~6 FiveAM tests + +1. =test-quarantine-on-three-errors= — a skill that errors three times in a + single cycle is quarantined and removed from trigger resolution. +2. =test-degraded-mode-visible= — when Screamer is not loaded, the system + prompt includes a DEGRADATION section. +3. =test-resource-shed-low-priority= — when memory exceeds threshold, =:low= + priority skills are unloaded first. +4. =test-critical-skills-never-shed= — =:critical= priority skills are retained + regardless of resource pressure. +5. =test-resource-recovery-reloads= — when resources recover below threshold + for N consecutive heartbeats, shed skills are reloaded automatically. +6. =test-quarantined-skill-relaodable= — a quarantined skill can be reloaded + via =/skill-reload= and passes sandbox validation before promotion. + +** Relation to Other Work + +This phase implements the Third Law of self-preservation as described in +=passepartout-neurosymbolic-design-decisions-and-options.org=. The integrity +monitoring (Phase 0) and degraded-mode signaling (Phase 1) provide the +infrastructure this phase extends with active, autonomous behavior. + * Phase 2: Screamer as Admission Gate (~200 lines — new skill) ** What @@ -246,29 +504,36 @@ heartbeat or manually. The cost is compute (Screamer exploration), not tokens. *** Admission gate =(screamer-admit candidate-fact existing-facts)= — wraps consistency check with -the contradiction policy lookup. If the candidate fact's entity class has policy -=:exclusive=, contradictions reject. If =:coexistent=, flag. If =:temporal=, -supersede. +the cardinality policy lookup. The policy is determined by the fact's entity class +(see Phase 1: =:singular=, =:dual=, or =:plural=). + +- =:singular=: same value ⇒ supersede (chain via =:parent-id=). Different value, + later timestamp ⇒ supersede. Different value, same timestamp ⇒ contradiction + rejected (human resolves). +- =:dual=: first two values admitted as complementary. Third rejected (prompt + cardinality promotion). +- =:plural=: any value admitted with cross-references. Active count transitions + trigger cardinality collapse checks. This is the function the archivist calls before any LLM-proposed fact enters the -store. It is also called on human-authored facts (which override the policy — -the human can assert contradictory facts in any domain). It is not called on -gate-outcome facts (gates are the ground truth for security domains). +store. It is also called on human-authored facts (which override — the human can +assert facts that bypass cardinality checks). It is not called on gate-outcome +facts (gates are the ground truth for security =:singular= domains). ** Verification — ~6 FiveAM tests 1. =test-screamer-consistency-passes= — a fact consistent with existing triples returns =:consistent=. 2. =test-screamer-contradiction-detected= — "app.env is not secret" contradicts - "all *.env files are secrets" and returns =:contradiction=. + "all *.env files are secrets" and returns =:contradiction=. 3. =test-screamer-redundant-detected= — asserting a fact already implied by - existing facts returns =:redundant=. + existing facts returns =:redundant=. 4. =test-screamer-deduction-produces-new-fact= — given "all *.env files are - secrets" and "prod.env matches *.env", Screamer deduces "prod.env is secret." -5. =test-admission-gate-rejects-contradiction= — the archivist's proposal that - contradicts an =:exclusive= domain fact is rejected. -6. =test-admission-gate-flags-coexistent-contradiction= — the archivist's proposal - that contradicts a =:coexistent= domain fact is stored with a cross-reference. + secrets" and "prod.env matches *.env", Screamer deduces "prod.env is secret." +5. =test-admission-gate-singular-supersedes= — a later-timestamped value for a + =:singular= domain fact supersedes the old value, chaining via =:parent-id=. +6. =test-admission-gate-dual-rejects-third= — a =:dual= domain rejects the third + value, prompting =:plural= promotion. ** Relation to Other Work @@ -315,8 +580,8 @@ relations (=:wrote=, =:published-in=, =:influenced=). If the heading is in *** Verify through Screamer Each proposed triple runs through =(screamer-admit candidate existing-facts)= -from Phase 2. Consistent and coexistent-flagged triples are admitted. Contradictory -triples in =:exclusive= domains are discarded with a log entry. +from Phase 2. Facts admitted follow the cardinality policy of their entity class +(=:singular=, =:dual=, or =:plural=). Rejected facts are discarded with a log entry. *** Provenance tracking @@ -448,7 +713,7 @@ This is Phase 4 of =notes/passepartout-v3.0.0-roadmap.org=. The flip concept ori =notes/passepartout-symbolic-engine-exploration.org= (lines 68-76) and is refined in =passepartout-neurosymbolic-design-decisions-and-options.org= under "The Flip." -* Phase 5: VivaceGraph as Persistent Store (~300 lines — new skill) +* Phase 5: VivaceGraph + Merkle DAG + Ontology Versioning (~400 lines — new skill) ** What @@ -531,6 +796,41 @@ On daemon start, =(vivacegraph-load)= reads the last saved graph. On heartbeat, =*memory-auto-save-interval*=. The save is atomic: write to a temp file, rename on success. Corruption-safe. +*** Merkle DAG version chains + +Each =(:entity :relation)= pair forms an independent Merkle chain. Facts hash +over =SHA-256(value || provenance || timestamp || parent-hash || grounding)=. +The =:parent-id= pointer forms the chain. Tampering with any version breaks all +downstream hashes. + +The chains form a DAG, not a single list. Facts about =.env= evolve independently +from facts about Nabokov. Inserting a new version is O(1) per chain. =:dual= and +=:plural= facts cross-reference via =:complement= and =:contradiction= edges +but maintain independent ancestor chains. The Merkle DAG rests on the existing +=memory-object= infrastructure from v0.2.0 — the fact store is a new occupant +of existing housing. ~50 lines to bridge the fact schema into =memory-object= +wrappers. + +*** Ontology versioning + +The category hierarchy itself is a Merkle tree. Every entity class definition +hashes over its superclasses, cardinality policy, relations, and description. +The aggregate hash of all active class definitions is the =:ontology-version= — +a Merkle root of the current worldview. + +Every fact stores its =:ontology-version= at the time of assertion (a single +64-hex-char field). When categories change, the new hash flags affected facts +for re-verification (Screamer re-evaluates each against the new category +definitions). Re-verification outcomes are =:survived= (deduction still holds), +=:incoherent= (premises don't translate under new categories, flagged for human +review), or =:reclassified= (valid but under different classification). + +Queries accept an optional =:ontology-version= parameter. The default is +=:active= (current worldview). Specifying a version returns facts as they were +under that worldview: "Under my 2024 security model, this file was a secret. +Under my 2025 model, it is an auth-secret." ~40 lines on top of VivaceGraph +persistence. + ** Verification — ~5 FiveAM tests 1. =test-vivacegraph-roundtrip= — save and load preserves all facts with @@ -542,7 +842,13 @@ on success. Corruption-safe. 4. =test-type-level-prevents-self-reference= — a query from a type-level-2 function cannot return type-level-3 facts. 5. =test-fact-store-fallback-without-vivacegraph= — when VivaceGraph is not - loaded, the hash-table fallback functions identically to Phase 1-4 behavior. + loaded, the hash-table fallback functions identically to Phase 1-4 behavior. +6. =test-merkle-chain-tamper-detected= — modifying a fact's value breaks the + hash chain, detectable by re-walking the =:parent-id= spine. +7. =test-ontology-version-query= — querying with an old =:ontology-version= + returns facts as they were under that worldview, not the current one. +8. =test-reverification-flags-on-category-change= — changing a category + definition sets =:re-verify-status :pending= on all affected facts. ** Relation to Other Work @@ -749,10 +1055,10 @@ is pre-structured. The archivist's job changes from "discover entities" to to it (1-hop neighbors). Optionally expand to 2-hop for deeply connected domains. -3. *Admit with co-existent policy.* Wikidata facts are admitted with - =:provenance :wikidata= and contradiction policy =:coexistent=. They do not - override your memex's facts. They sit alongside them. Contradictions are - surfaced, not resolved. +3. *Admit with plural policy.* Wikidata facts are admitted with + =:provenance :wikidata= and cardinality policy =:plural=. They do not + override your memex's facts. They sit alongside them with provenance + displayed. Disagreements are surfaced, not resolved. 4. *Cross-domain query.* "What does my memex say about Nabokov that Wikidata doesn't?" "Where does my memex disagree with Wikidata?" "What entities in my @@ -778,35 +1084,65 @@ hop. A memex spanning literature, history, philosophy, and science may need 3-4 hops. The query performance and memory costs of a large Wikidata load have not been estimated. +** Deferred Authentication Layers (Layers 2-4) + +Phase 0b ships Layer 1 (cryptographic). The remaining layers are deferred to +when their dependencies are available: + +- *Layer 2 — Sensory.* Active when vision and audio processing skills are loaded. + Verifies liveness, cross-modal consistency (face matches voice matches location), + and environmental coherence. Defers to Phase TBD (vision/audio skills feature + cycle). When unavailable, returns =:unavailable= — graceful degradation. +- *Layer 3 — Deterministic Identity Reasoning.* Active when Phase 2 (Screamer + + populated fact store) is complete. Queries the fact store for identity-ruling + facts ("Jack is deceased; this signal claims to be Jack") and returns binary + reject/pass. Defers to Phase 2+. +- *Layer 4 — Probabilistic Identity Reasoning.* Active when style profiles exist + as a fact-store domain. Uses embedding infrastructure (=embedding-native.lisp=, + v0.4.0, already exists) to compare writing style, behavioral patterns, and + content coherence against known profiles. Returns a confidence score; never + rejects outright — downgrades authorization. Defers to when style profiles are + built as a fact-store domain. + +The gate architecture (single vector 0, =fboundp=-guarded sub-layers, +configurable per signal class) is designed with all four layers from Phase 0b. +Adding a layer requires adding a skill, not modifying the gate. + * Summary: Lines and Dependencies -| Phase | Component | Lines | New Skill? | Depends On | Earliest Release | -|-------+-------------------------+-------+------------+-----------------+------------------| -| 0 | PM-type-level gates | ~30 | No | Dispatcher | Immediately | -| 1 | Triple fact store | ~150 | Yes | Phase 0 | v0.7.2+ | -| 2 | Screamer admission | ~200 | Yes | Phase 1 | v0.7.2+ | -| 3 | Archivist extraction | ~100 | Extends | Phase 2 | v0.8.0+ | -| 4 | Flip — sufficiency | ~50 | Extends | Phase 3 | v0.8.0+ | -| 5 | VivaceGraph store | ~300 | Yes | Phase 4 | v0.10.0+ | -| 6 | ACL2 verification | ~200 | Yes | Phase 5 | v0.12.0+ | -| 7 | 10-80-10 planner | ~500 | Yes | Phase 6 | v3.0.0 | -| 8+ | Semantic Wikipedia | TBD | Yes | Phase 5 | TBD | -|-------+-------------------------+-------+------------+-----------------+------------------| -| Total | | ~1530 | | | | +| Phase | Component | Lines | New Skill? | Depends On | Earliest Release | +|-------+-----------------------------------------+--------+------------+-----------------+------------------| +| 0 | PM-type-level gates + core integrity | ~75 | No | Dispatcher | Immediately | +| 0b | Layered auth — Layer 1 (cryptographic) | ~200 | No | Phase 0 | v0.7.2+ | +| 1 | Triple fact store + abstract API | ~200 | Yes | Phase 0b | v0.7.2+ | +| 1a | Self-preservation mechanisms | ~120 | Extends | Phase 0, 1 | v0.7.2+ | +| 2 | Screamer admission gate | ~200 | Yes | Phase 1 | v0.7.2+ | +| 3 | Archivist as fact proposer | ~100 | Extends | Phase 2 | v0.8.0+ | +| 4 | Sufficiency criterion — the flip | ~50 | Extends | Phase 3 | v0.8.0+ | +| 5 | VivaceGraph + Merkle DAG + ontology ver | ~400 | Yes | Phase 4 | v0.10.0+ | +| 6 | ACL2 structural verification | ~200 | Yes | Phase 5 | v0.12.0+ | +| 7 | 10-80-10 planner | ~500 | Yes | Phase 6 | v3.0.0 | +| 8+ | Semantic Wikipedia integration | TBD | Yes | Phase 5 | TBD | +|-------+-----------------------------------------+--------+------------+-----------------+------------------| +| Total | | ~2045 | | | | This roadmap is independent of the feature roadmap in =passepartout/docs/ROADMAP.org=. Phase 0 ships alongside any v0.7.x patch. The symbolic engine grows in parallel with feature work (TUI improvements, MCP tools, gateway expansion, etc.), not after it. The feature roadmap describes /what/ the -agent can do; this roadmap describes /how/ it knows what it knows. +agent can do; this roadmap describes /how/ it knows what it knows and how it +protects itself. -The total new code across all phases is approximately 1,530 lines. Relative to +The total new code across all phases is approximately 2,045 lines. Relative to the existing codebase (~8,000+ lines across 40+ Org source files and 30+ skills), -the symbolic engine is a ~20% addition. Relative to the ROADMAP's planned feature +the symbolic engine is a ~25% addition. Relative to the ROADMAP's planned feature work through v0.13.0 (thousands of lines of TUI rendering, MCP protocol implementation, skin engine, planning, etc.), the symbolic engine is a small, -orthogonal thread that grows the architecture's reasoning depth while the feature -work grows its interaction breadth. +orthogonal thread that grows the architecture's reasoning depth, self-preservation, +signal authentication, and knowledge integrity while the feature work grows its +interaction breadth. +capability, and knowledge integrity while the feature work grows its interaction +breadth. * Competitive Advantage Analysis @@ -818,6 +1154,23 @@ A request to modify the dispatcher's own rules is impossible by construction, no just caught by pattern matching. No competitor has this — their equivalent of "core file protection" is a prompt instruction, not a type system. +** Phase 0b: Layered signal authentication — verified origin, not claimed origin + +No competitor verifies /who/ issued a signal. Every agent harness accepts signals +from any source that speaks its protocol — TUI, CLI, subprocess, internal skill. +A compromised dependency can impersonate any signal source. Passepartout's +four-layer authentication gate makes signal source spoofing impossible at Layer 1 +(cryptographic), detectable at Layers 2-3 (sensory + deterministic reasoning), +and probabilistically flagged at Layer 4 (style analysis). The key registry is a +fact-store domain with Merkle-hashed provenance — key creation, promotion, and +revocation are auditable, versioned, and survivable across restarts. The signal +provenance chain makes multi-step automated causality traceable: "this deletion +happened because sensor #3 signed a classification (liveness check failed, sender +deceased 2 years ago) that skill #7 signed a re-indexing from, and sensor #3 was +subsequently revoked." No competitor can trace an action to its authenticated +source through four independent evidence layers because no competitor has a +layered authentication gate and a Merkle-linked provenance chain. + ** Phase 2-3: Verified extraction — the symbolic index grows without corruption No competitor verifies extracted facts against an existing knowledge base. Their @@ -851,6 +1204,22 @@ Nabokov wrote /Pale Fire/ and lectured on Kafka. Passepartout with Wikidata loaded knows both, and the entity knowledge costs zero LLM tokens — it is loaded once as structured data and queried via VivaceGraph traversals. +** Phase 0-1a: Self-preservation — the agent knows when it is wounded + +No competitor detects its own degradation. Claude Code, OpenCode, and Hermes +all fail silently when a tool crashes or a dependency is missing — the agent +keeps running, producing degraded output, never telling the user. Passepartout's +quarantine system detects failing skills, unloads them automatically, and +displays a degraded-mode indicator in the status bar. The external watchdog +restarts the daemon if the process dies. The integrity monitor detects corrupted +core files. The agent refuses to execute commands that would destroy its own +runtime, explaining /why/ and redirecting to the safe termination path. + +This is not a feature. It is the architectural difference between a tool that +breaks silently and a system that preserves itself actively, signals its wounds, +and traces its degradation to specific components that can be reloaded or +repaired. + ** The permanent competitive advantage The competitive advantage is not any single feature. It is the architecture's diff --git a/projects/passepartout b/projects/passepartout index 138f909..24a24b4 160000 --- a/projects/passepartout +++ b/projects/passepartout @@ -1 +1 @@ -Subproject commit 138f909a33a306db914942dc7704f56c131f2bd0 +Subproject commit 24a24b481b61e834c5a0dc2b47ef0f2c18a5d360