tools: add repl-block, check-tangle; move existing tools into memex

New tools (projects/<tool>/ — standalone, git-committed): - repl-block: extract and pipe lisp blocks from org files to the REPL - check-tangle: tangle + compile in one step, reports errors Existing tools moved from ~/.opencode/bin/ into memex (survives reinstalls): - repl, tangle, org-eval, verify-repl AGENTS.md updated: - Tool reference table with all 7 tools - Package reference table for passepartout and cl-tty - Updated tangle command to use project-local tools .opencode/commands/ added: check-parens, repl-block, check-tangle commands
2026-05-13 12:54:38 -04:00
parent b0b9a25fb3
commit c9cc874e53
10 changed files with 526 additions and 4 deletions
--- a/projects/AGENTS.md
+++ b/projects/AGENTS.md
@@ -125,13 +125,53 @@
  goes to main. If it spans sessions or might be abandoned for a better
  approach, it gets a branch.

+## Tools
+
+All tools are standalone scripts in `projects/<tool>/`, committed to the memex.
+They survive upgrades, reinstalls, and new clones.
+
+| Tool | What | Usage |
+|------|------|-------|
+| `check-parens` | Validate paren balance in org lisp blocks (uses SBCL reader) | `projects/check-parens/check-parens org/file.org` |
+| `repl-block` | Extract and send an org lisp block to the daemon REPL | `projects/repl-block/repl-block org/file.org --function foo` |
+| `check-tangle` | Tangle an org file and compile the result in SBCL | `projects/check-tangle/check-tangle org/file.org` |
+| `repl` | Send lisp code to the running daemon | `projects/repl-tool/repl "(+ 1 2)"` |
+| `tangle` | Tangle an org file via Emacs batch | `projects/tangle-tool/tangle org/file.org` |
+| `org-eval` | Evaluate an org src block via Emacs | `projects/org-eval-tool/org-eval org/file.org` |
+| `verify-repl` | Compliance: check defuns have REPL-VERIFIED comments | `projects/verify-repl-tool/verify-repl org/` |
+
+## Package reference
+
+When sending code to the REPL, use the correct `(in-package ...)` form first.
+
+### Passepartout
+
+| Package | When to use |
+|---------|-------------|
+| `:passepartout` | Core system — skills, gates, memory, dispatcher |
+| `:passepartout.channel-tui` | TUI — event handlers, view, state |
+| `<project>-tests` | Test suites — run with `(fiveam:run (intern ...))` |
+
+### cl-tty
+
+| Package | When to use |
+|---------|-------------|
+| `:cl-tty.backend` | Backend protocol — terminal init, draw-text, read-event |
+| `:cl-tty.rendering` | Framebuffer — make-framebuffer, flush-framebuffer |
+| `:cl-tty.input` | Input — key-event, defkeymap, dispatch-key-event |
+| `:cl-tty.layout` | Layout — vbox, hbox, spacer, layout-calculate |
+| `:cl-tty.dialog` | Dialog system — dialog stack, select-dialog |
+| `:cl-tty.select` | Select widget — filter, handle-key |
+| `:cl-tty.slot` | Slot/plugin system |
+| `:cl-tty.markdown` | Markdown rendering |
+
 ## Commands

 Tangle a single file:
-  emacs --batch --eval "(progn (require 'org) (find-file \"org/FILE.org\") (org-babel-tangle) (kill-buffer))"
+  projects/tangle-tool/tangle org/FILE.org

-Validate structural integrity (org/ source files only):
-  emacs --batch -Q --eval '(progn (find-file "org/FILE.org") (check-parens) (kill-buffer))'
+Or:
+  emacs --batch --eval "(progn (require 'org) (find-file \"org/FILE.org\") (org-babel-tangle) (kill-buffer))"

 Run tests (from REPL):
  (fiveam:run (intern "SUITE-NAME" :project-TESTS))
@@ -139,7 +179,6 @@ Run tests (from REPL):

 Run tests (SBCL fallback — only when the runtime cannot start):
  sbcl --noinform \
-    --eval '(load (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname)))' \
    --eval '(ql:quickload :your-project :silent t)' \
    --eval '(load "lisp/FILE.lisp")' \
    --eval '(fiveam:run (intern "SUITE-NAME" :project-TESTS))' --quit
--- a/projects/check-tangle/check-tangle
+++ b/projects/check-tangle/check-tangle
@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+# Tangle an .org file and compile the resulting .lisp with SBCL.
+# Reports the first compile error with line numbers.
+#
+# Usage: check-tangle <file.org>
+#   Exit 0 if compile succeeds, 1 if tangled or compilation fails
+
+set -euo pipefail
+
+if [ $# -ne 1 ]; then
+    echo "Usage: check-tangle <file.org>" >&2
+    exit 2
+fi
+
+ORG_FILE="$1"
+
+if [ ! -f "$ORG_FILE" ]; then
+    echo "Error: File not found: $ORG_FILE" >&2
+    exit 1
+fi
+
+# Determine the tangle target from the org file
+TANGLE=$(grep -m1 'header-args.*:tangle' "$ORG_FILE" | sed 's/.*:tangle //' | head -1 || true)
+if [ -z "$TANGLE" ]; then
+    echo "SKIP: $ORG_FILE — no :tangle header" >&2
+    exit 0
+fi
+
+# Resolve relative tangle path
+ORG_DIR=$(dirname "$ORG_FILE")
+LISP_FILE=$(cd "$ORG_DIR" && realpath -m "$TANGLE" 2>/dev/null || echo "$ORG_DIR/$TANGLE")
+
+echo "Tangling: $ORG_FILE → $LISP_FILE" >&2
+
+# Tangle using opencode's tangle tool
+TANGLE_CMD=$(command -v tangle 2>/dev/null || echo "/home/user/.opencode/bin/tangle")
+if ! "$TANGLE_CMD" "$ORG_FILE" 2>/dev/null; then
+    echo "FAIL: Tangling $ORG_FILE failed" >&2
+    exit 1
+fi
+
+if [ ! -f "$LISP_FILE" ]; then
+    echo "SKIP: Tangle target $LISP_FILE not found after tangling" >&2
+    exit 0
+fi
+
+echo "Compiling: $LISP_FILE" >&2
+
+# Compile with SBCL, capturing errors
+OUTPUT=$(sbcl --noinform --no-userinit --disable-debugger --quit \
+    --eval "(handler-case
+                (progn (compile-file \"$LISP_FILE\") (print :OK))
+              (error (c) (print c) (sb-ext:exit :code 1)))" 2>&1 || true)
+
+if echo "$OUTPUT" | grep -q ":OK"; then
+    echo "OK: $ORG_FILE compiles cleanly" >&2
+    exit 0
+else
+    echo "$OUTPUT" | grep -v '^;' | grep -v '^$' | head -5
+    echo "FAIL: $ORG_FILE — compilation error" >&2
+    exit 1
+fi
--- a/projects/org-eval-tool/org-eval
+++ b/projects/org-eval-tool/org-eval
@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+# Evaluate an org src block using Emacs batch mode
+# Usage: org-eval <org-file> [block-index]
+# If block-index is not provided, evaluates all blocks
+
+set -e
+
+if [ $# -lt 1 ]; then
+    echo "Usage: org-eval <org-file> [block-index]"
+    echo "  Evaluates src blocks in the org file"
+    echo "  If block-index is omitted, evaluates ALL blocks"
+    exit 1
+fi
+
+ORG_FILE="$1"
+BLOCK_INDEX="${2:-}"
+
+if [ ! -f "$ORG_FILE" ]; then
+    echo "Error: File not found: $ORG_FILE"
+    exit 1
+fi
+
+echo "Evaluating: $ORG_FILE"
+
+if [ -n "$BLOCK_INDEX" ]; then
+    # Evaluate specific block
+    emacs --batch \
+        --load org \
+        --eval "(setq org-confirm-babel-evaluate nil)" \
+        --eval "(with-current-buffer (find-file-noselect \"$ORG_FILE\") \
+                 (goto-char (point-min)) \
+                 (dotimes (_ $BLOCK_INDEX) \
+                   (org-babel-next-src-block)) \
+                 (org-babel-execute-src-block))"
+else
+    # Evaluate all blocks
+    emacs --batch \
+        --load org \
+        --eval "(setq org-confirm-babel-evaluate nil)" \
+        --eval "(org-babel-execute-buffer)"
+fi
+
+echo "Done."
--- a/projects/repl-block/repl-block
+++ b/projects/repl-block/repl-block
@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+"""Extract a #+begin_src lisp block from an .org file and print it.
+
+Identify the block by:
+  - index: --block 3  (1-based, counting all #+begin_src lisp blocks)
+  - function: --function view-status  (finds a defun/demacro matching the name)
+
+Output is the block content between begin and end markers, sans markers.
+
+Usage:
+  repl-block org/file.org --function foo | repl
+  repl-block org/file.org --block 3
+  repl-block org/file.org --function foo --package :my-package  (adds in-package prefix)
+"""
+
+import sys
+import re
+import argparse
+
+
+LISP_BEGIN = re.compile(r"#\+begin_src\s+lisp\b", re.IGNORECASE)
+END_SRC = re.compile(r"#\+end_src\b", re.IGNORECASE)
+
+
+def extract_blocks(lines):
+    blocks = []
+    start = None
+    buf = None
+    for i, line in enumerate(lines, start=1):
+        if start is None:
+            if LISP_BEGIN.match(line.lstrip()):
+                start = i
+                buf = []
+        else:
+            if END_SRC.match(line.lstrip()):
+                blocks.append((start, buf))
+                start = None
+                buf = None
+            else:
+                buf.append(line.rstrip("\n"))
+    return blocks
+
+
+def find_by_function(blocks, name, lines):
+    for line_no, body in blocks:
+        for bline in body:
+            if re.match(rf"\(def(un|macro|method|var|parameter|class|struct|package)\s+{re.escape(name)}\b", bline):
+                return line_no, body
+    return None, None
+
+
+def find_by_index(blocks, idx):
+    if 1 <= idx <= len(blocks):
+        return blocks[idx - 1]
+    return None, None
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Extract lisp blocks from org files")
+    parser.add_argument("file", help=".org file to extract from")
+    parser.add_argument("--block", type=int, default=None, help="Block number (1-based)")
+    parser.add_argument("--function", type=str, default=None, help="Function name to find")
+    parser.add_argument("--package", type=str, default=None, help="(in-package ...) prefix")
+    args = parser.parse_args()
+
+    try:
+        with open(args.file, encoding="utf-8") as f:
+            lines = f.readlines()
+    except FileNotFoundError:
+        print(f"File not found: {args.file}", file=sys.stderr)
+        return 1
+    except Exception as e:
+        print(f"Error reading {args.file}: {e}", file=sys.stderr)
+        return 1
+
+    blocks = extract_blocks(lines)
+
+    if args.function:
+        line_no, body = find_by_function(blocks, args.function, lines)
+        if body is None:
+            print(f"No block found containing function '{args.function}'", file=sys.stderr)
+            return 1
+    elif args.block:
+        line_no, body = find_by_index(blocks, args.block)
+        if body is None:
+            print(f"Block {args.block} not found (file has {len(blocks)} blocks)", file=sys.stderr)
+            return 1
+    else:
+        # Print listing
+        for idx, (line_no, body) in enumerate(blocks, 1):
+            first = (body or [""])[0][:60]
+            print(f"  {idx}: line {line_no}: {first}")
+        print(f"\n{len(blocks)} total blocks", file=sys.stderr)
+        return 1
+
+    if args.package:
+        pkg = args.package
+        if not pkg.startswith(":"):
+            pkg = f":{pkg}"
+        print(f"(in-package {pkg})")
+        print()
+
+    for line in body:
+        print(line)
+
+
+if __name__ == "__main__":
+    sys.exit(main())
--- a/projects/repl-tool/repl
+++ b/projects/repl-tool/repl
@@ -0,0 +1,74 @@
+#!/usr/bin/env perl
+# repl — evaluate Lisp forms against the running Passepartout daemon
+# Usage: repl <lisp-form>
+#   or: echo '<lisp-form>' | repl
+#
+# Connects to the daemon on 127.0.0.1:9105, sends a repl-eval request
+# via the framed TCP protocol, reads and prints the response.
+
+use strict;
+use warnings;
+use IO::Socket::INET;
+
+my $HOST  = $ENV{PASSEPARTOUT_HOST} || "127.0.0.1";
+my $PORT  = $ENV{PASSEPARTOUT_PORT} || "9105";
+my $TIMEOUT = $ENV{PASSEPARTOUT_REPL_TIMEOUT} || 10;
+
+my $expr = join(" ", @ARGV);
+if (!$expr) {
+    $expr = do { local $/; <STDIN> };
+}
+chomp($expr);
+# Quote the expression for embedding in a Lisp string
+$expr =~ s/\\/\\\\/g;   # backslash → doubled
+$expr =~ s/"/\\"/g;     # " → \"
+if (!$expr) {
+    die "Usage: repl <lisp-form>\n   or: echo '(+ 1 2)' | repl\n";
+}
+
+my $sock = IO::Socket::INET->new(
+    PeerHost  => $HOST,
+    PeerPort  => $PORT,
+    Proto     => "tcp",
+    Timeout   => 2
+) or die "Cannot connect to $HOST:$PORT: $!\n";
+
+sub read_frame {
+    my ($sock) = @_;
+    my $hex;
+    $sock->read($hex, 6) or return undef;
+    my $len = hex($hex);
+    my $content;
+    $sock->read($content, $len);
+    return $content;
+}
+
+sub write_frame {
+    my ($sock, $content) = @_;
+    my $len = sprintf("%06X", length($content));
+    $sock->send($len . $content);
+}
+
+# Read handshake (discard)
+my $handshake = read_frame($sock);
+
+# Build framed message
+my $msg = '(:TYPE :EVENT :PAYLOAD (:SENSOR :repl-eval :CODE "' . $expr . '"))';
+write_frame($sock, $msg);
+
+# Read response
+my $response = read_frame($sock);
+if ($response) {
+    if ($response =~ /:VALUE "([^"]*)"/s) {
+        print "$1\n";
+    } elsif ($response =~ /:message "([^"]*)"/s) {
+        print STDERR "$1\n";
+        exit 1;
+    } else {
+        print "Response: $response\n";
+    }
+} else {
+    print STDERR "No response from daemon (is it running on $HOST:$PORT?)\n";
+    exit 1;
+}
+$sock->close();
--- a/projects/tangle-tool/tangle
+++ b/projects/tangle-tool/tangle
@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# Tangle an org file using Emacs batch mode
+# Usage: tangle <org-file>
+
+set -e
+
+if [ $# -ne 1 ]; then
+    echo "Usage: tangle <org-file>"
+    echo "  Tangles all src blocks with :tangle directives to their targets"
+    exit 1
+fi
+
+ORG_FILE="$1"
+
+if [ ! -f "$ORG_FILE" ]; then
+    echo "Error: File not found: $ORG_FILE"
+    exit 1
+fi
+
+echo "Tangling: $ORG_FILE"
+
+emacs --batch \
+    --load org \
+    --eval "(setq org-confirm-babel-evaluate nil)" \
+    --eval "(org-babel-tangle-file \"$ORG_FILE\")"
+
+echo "Done."
--- a/projects/verify-repl-tool/verify-repl
+++ b/projects/verify-repl-tool/verify-repl
@@ -0,0 +1,140 @@
+#!/bin/bash
+# verify-repl — compliance checker for the OpenCode Engineering Discipline
+#
+# Usage: verify-repl <org-directory>
+#   Scans all .org files in the given directory for violations of:
+#   1. REPL-First: every (defun|defmacro|defvar|defparameter|defstruct|defmethod|defclass)
+#      block must have ";; REPL-VERIFIED:" on the line above #+begin_src lisp
+#   2. One-per-block: each #+begin_src lisp block must contain exactly one top-level form
+#   3. Prose-before-code: each code block must be preceded by an Org headline
+#
+# Returns 0 if all checks pass, 1 if violations found.
+
+set -euo pipefail
+
+ORG_DIR="${1:-}"
+if [ -z "$ORG_DIR" ] || [ ! -d "$ORG_DIR" ]; then
+    echo "Usage: verify-repl <org-directory>"
+    exit 1
+fi
+
+VIOLATIONS=0
+FILES_CHECKED=0
+
+# Blacklist: files exempt from REPL verification (core infrastructure, tests)
+BLACKLIST=(
+    "core-defpackage.org"
+    "core-manifest.org"
+    "core-skills.org"
+    "core-communication.org"
+    "package.lisp"
+    "setup.org"
+)
+
+is_blacklisted() {
+    local fname
+    fname=$(basename "$1")
+    for bl in "${BLACKLIST[@]}"; do
+        [ "$fname" = "$bl" ] && return 0
+    done
+    return 1
+}
+
+check_file() {
+    local file="$1"
+    local fname
+    fname=$(basename "$file")
+    local in_block=0
+    local block_start=0
+    local prev_line=""
+    local prev_was_headline=0
+    local block_content=""
+    local line_no=0
+    local has_repl_verify=0
+    local def_count=0
+    local violations_in_file=0
+
+    is_blacklisted "$file" && return 0
+
+    FILES_CHECKED=$((FILES_CHECKED + 1))
+
+    while IFS= read -r line || [ -n "$line" ]; do
+        line_no=$((line_no + 1))
+        local trimmed="${line#"${line%%[![:space:]]*}"}"
+
+        # Track headlines
+        if echo "$trimmed" | grep -qE '^\*+[[:space:]]'; then
+            prev_was_headline=1
+        fi
+
+        # Enter code block
+        if echo "$trimmed" | grep -qE '^#\+begin_src[[:space:]]+lisp'; then
+            in_block=1
+            block_start=$line_no
+            block_content=""
+            def_count=0
+            has_repl_verify=0
+            # Check for REPL-VERIFIED comment on previous line(s)
+            if echo "$prev_line" | grep -qE ';;.REPL.VERIFIED:'; then
+                has_repl_verify=1
+            fi
+            # Check for prose requirement: was there a headline before this block?
+            if [ "$prev_was_headline" -eq 0 ]; then
+                echo "  $fname:$block_start: PROSE-BEFORE-CODE: no Org headline before code block"
+                violations_in_file=$((violations_in_file + 1))
+            fi
+        fi
+
+        # Inside code block: collect content
+        if [ "$in_block" -eq 1 ]; then
+            if echo "$trimmed" | grep -qE '^#\+end_src'; then
+                in_block=0
+                # Check: REPL verification for definition blocks
+                if [ "$def_count" -gt 0 ] && [ "$has_repl_verify" -eq 0 ]; then
+                    echo "  $fname:$block_start: REPL-FIRST: $(if [ "$def_count" -gt 1 ]; then echo "$def_count definitions"; else echo "defun/defmacro"; fi) without REPL-VERIFIED comment"
+                    violations_in_file=$((violations_in_file + 1))
+                fi
+                # Check: one-per-block
+                if [ "$def_count" -gt 1 ]; then
+                    echo "  $fname:$block_start: ONE-PER-BLOCK: $def_count definitions in a single block (must be exactly 1)"
+                    violations_in_file=$((violations_in_file + 1))
+                fi
+                prev_was_headline=0
+                block_content=""
+            else
+                # Count definitions in block
+                if echo "$trimmed" | grep -qE '^\((defun|defmacro|defvar|defparameter|defstruct|defmethod|defclass)[[:space:](]'; then
+                    def_count=$((def_count + 1))
+                fi
+            fi
+        fi
+
+        prev_line="$line"
+    done < "$file"
+
+    if [ "$violations_in_file" -gt 0 ]; then
+        echo "  ✗ $fname: $violations_in_file violation(s)"
+        VIOLATIONS=$((VIOLATIONS + violations_in_file))
+    fi
+}
+
+echo "=== REPL Compliance Check ==="
+echo "Directory: $ORG_DIR"
+echo ""
+
+for file in "$ORG_DIR"/*.org; do
+    [ -f "$file" ] || continue
+    check_file "$file"
+done
+
+echo ""
+echo "Files checked: $FILES_CHECKED"
+echo "Violations:    $VIOLATIONS"
+
+if [ "$VIOLATIONS" -eq 0 ]; then
+    echo "Status:        ✓ PASS — all checks satisfied"
+    exit 0
+else
+    echo "Status:        ✗ FAIL — fix violations before committing"
+    exit 1
+fi