From 43c8c886e97588b6969c0e327a5e389caacd5083 Mon Sep 17 00:00:00 2001
From: Hermes <hermes@hermes.local>
Date: Sat, 23 May 2026 06:21:29 +0000
Subject: [PATCH] roadmap: Passepartout native Org-mode knowledge base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Adds roadmap item for Passepartout to ingest and query org files
directly — no pandoc/gbrain bridge. Replaces the current org→md→gbrain
pipeline with native org parsing, heading-level vector embeddings,
property-based entity extraction, and org-id cross-references.
Target v0.8.0-v0.9.0 after gate stack and Screamer planner.
---
 ideas/native-org-knowledge-base.org | 81 +++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 ideas/native-org-knowledge-base.org

diff --git a/ideas/native-org-knowledge-base.org b/ideas/native-org-knowledge-base.org
new file mode 100644
index 0000000..a1bc995
--- /dev/null
+++ b/ideas/native-org-knowledge-base.org
@@ -0,0 +1,81 @@
+:PROPERTIES:
+:ID:       7f4e6b9a-2c1d-5e8f-9a3b-6d7c4e5f2a1b
+:CREATED:  [2026-05-23 Sat]
+:END:
+#+title: Passepartout Native Org-Mode Knowledge Base
+#+filetags: :passepartout:roadmap:knowledge:org:gbrain:
+
+** What
+
+Passepartout should be able to use Org-mode files directly as its
+knowledge base — no pandoc conversion, no markdown intermediary.
+
+Currently gbrain provides vector search + entity linking over markdown,
+but we bridge via a conversion layer (org → pandoc → markdown → gbrain).
+This loses Org-mode semantics: properties drawers become flat YAML, tag
+inheritance is lost, file: links become relative markdown links, TODO
+states vanish, and the tree structure (headings with content subtrees)
+collapses into flat markdown headings.
+
+** Why
+
+Org-mode's data model is strictly richer than markdown's. A Passepartout
+that can ingest, index, and query org files natively has:
+- Property-based entity extraction (no separate links: frontmatter needed)
+- Tag-inheritance for automatic categorization
+- TODO/priority/timestamps for knowledge freshness signals
+- ID-based stable cross-references (org-id) that survive file moves
+- Heading-level chunking (one heading = one knowledge unit)
+- The same file format for everything — no split between "authoring format"
+  and "knowledge base format"
+
+** What it replaces
+
+The current pipeline: org file → pandoc → markdown file → gbrain import →
+
+gbrain embed → gbrain query. This is four serial steps with a conversion
+at each boundary that degrades the data model.
+
+The target: org file → (Passepartout-native indexer) → query. Zero
+conversion, zero data loss.
+
+** Architecture sketch
+
+A Passepartout-native knowledge module that directly ingests
+ideas/*.org:
+
+- Parser: extract each heading as a chunk. Preserve:
+  - Heading path (H1 → H2 → H3) as a hierarchical path
+  - Properties drawer as structured metadata
+  - file: links as typed entity references
+  - org-id as stable identifier
+  - Tags (inherited from parent headings)
+  - TODO state, priority, timestamps
+
+- Embedder: vector-embed each heading chunk with metadata prefix
+
+- Query: hybrid search over headings + full-text over content.
+  Result includes the heading path + sibling headings for context.
+
+- Cross-reference graph: build a typed entity graph from:
+  - file: links → typed reference
+  - org-id links → stable cross-doc reference
+  - Tag co-occurrence → implicit relationship
+  - Same-property values → attribute-based grouping
+
+- Dream cycle: auto-discover entities from org properties and file:
+  links. Enrich thin sections. Flag sections with stale timestamps.
+
+** Priority
+
+Below the gate stack and ACL2 planner (v1.0.0 dependencies) but above
+the Lisp Machine hardware. Target: v0.8.0-v0.9.0 range, once Screamer
+planner is stable enough to route queries through the knowledge base.
+
+The short-term bridge (current) is gbrain with nightly org→md sync.
+This is adequate while the gate stack and planner are the priority.
+The native org module replaces gbrain entirely once built.
+
+** See also
+[[file:../../concepts/compliance-framework-mapping.org][Compliance framework mapping]]
+[[file:../../ideas/passepartout-economics.org][Passepartout economics]]