v0.15.0: Critical input/rendering fixes, subagent-reviewed #7

Merged
amr merged 36 commits from feature/v0.11.0-slots into main 2026-05-11 22:03:18 -04:00
Owner

Summary

12 commits across ~3 review cycles. Every fix was reviewed by a subagent.

Critical

  • case→cond in %read-event — all keyboard input was dead code. CL case with predicate expressions like ((and (>= b #x01) (<= b #x1a)) treats keys as atoms for eql comparison. Used cond instead.
  • (declare (ignore w)) with w used — undefined behavior in textarea render

High

  • Theme resolution — semantic keywords (:accent, :text-muted) returned empty strings. sgr-fg/sgr-bg now fall back to theme-colors hash.
  • SGR mouse parser was defined but never wired into %read-escape-sequence
  • hit-test was checking nonexistent direct slots (x/y/width/height) instead of component-layout-node; also did not recurse into children

Medium

  • Scrollbox/text-input/textarea renders were no-op stubs
  • TabBar/Select rendered at (0,0) regardless of layout position
  • X10 mouse handler never detected release events
  • parse-csi-params treated < as a numeric digit (latent trap)
  • read-raw-byte buffer pinning (SBCL type error with raw unsigned-byte arrays)
  • termios-cc API updated for SBCL 2.5.x
  • Test runner now exits 1 on failure (was always 0)

Other

  • Demo rewritten, README rewritten, version bumped to v0.15.0
  • All fixes backported to org source files
  • 392 tests pass, exit code 0

Running the demo

./demo.sh

## Summary 12 commits across ~3 review cycles. Every fix was reviewed by a subagent. ### Critical - **case→cond in %read-event** — all keyboard input was dead code. CL case with predicate expressions like ((and (>= b #x01) (<= b #x1a)) treats keys as atoms for eql comparison. Used cond instead. - **(declare (ignore w)) with w used** — undefined behavior in textarea render ### High - **Theme resolution** — semantic keywords (:accent, :text-muted) returned empty strings. sgr-fg/sgr-bg now fall back to *theme-colors* hash. - **SGR mouse parser** was defined but never wired into %read-escape-sequence - **hit-test** was checking nonexistent direct slots (x/y/width/height) instead of component-layout-node; also did not recurse into children ### Medium - Scrollbox/text-input/textarea renders were no-op stubs - TabBar/Select rendered at (0,0) regardless of layout position - X10 mouse handler never detected release events - parse-csi-params treated < as a numeric digit (latent trap) - read-raw-byte buffer pinning (SBCL type error with raw unsigned-byte arrays) - termios-cc API updated for SBCL 2.5.x - Test runner now exits 1 on failure (was always 0) ### Other - Demo rewritten, README rewritten, version bumped to v0.15.0 - All fixes backported to org source files - 392 tests pass, exit code 0 ### Running the demo ./demo.sh
amr added 48 commits 2026-05-11 21:53:44 -04:00
- Box class with border-style, title, fg/bg slots
- render-box dispatches through backend protocol
- draw-border for borders, draw-rect for background
- draw-text for title below top border
- 7 tests: defaults, border, background, title, no-border,
  zero-size, minimum-size
- 13 assertions, 100% GREEN
- ASDF updated with src/components module
- modern-backend now accepts :output-stream initarg
- Text class with content, fg/bg, wrap-mode (:word or :none)
- Span class for inline styled segments (bold, italic, etc.)
- render-text dispatches through backend's draw-text
- word-wrap function splits text at word boundaries
- split-string utility for whitespace tokenization
- 9 new tests: creation, content, empty, truncation, word-wrap,
  single-word, span creation, span storage
- modern-backend now accepts :output-stream
- ASDF updated with text component
- 28 total component tests, 100% GREEN
- dirty-mixin class with dirty slot (initform t)
- mark-clean clears dirty flag
- mark-dirty sets dirty flag
- 3 tests: default-dirty, clean, dirty-cycle
- ROADMAP.org: v0.2.0 all tasks DONE
- 31 component tests, 100% GREEN
Fixes from subagent review:
- Word-wrap now hard-breaks words exceeding max-width (was returning
  un-truncated overflow strings)
- Box zero-size guard now catches any zero/single dimension (was only
  catching both zero together)
- Title-align now respected (:left/:center/:right) with proper positioning
- render-text declares (ignore spans) to suppress unused warning
- ASDF test-op fixed: run! → run-tests (symbol didn't exist)
- New test: box-single-column (width=1 renders nothing)
- Tightened word-wrap test: verifies hard-break produces both chunks
- Simplified word-wrap with cond instead of nested if/progn (avoided
  recurring paren-balance issue)
- render generic function dispatches per component type
- render-screen entry point with sync wrapper
- render-node walks tree, computes layout, calls render
- component-layout-node generic (box/text methods)
- component-children/component-parent generics
- propagate-dirty marks component + ancestors dirty
- box and text now inherit from dirty-mixin
- 6 new tests: render dispatch, layout-node accessor, children,
  dirty propagation, available-width defaults
- 42 component tests, 100% GREEN
Fixes from subagent review:
- render-tests.lisp: added (in-suite box-suite) — tests were registered
  to default suite, never executed by runner
- dirty-tests.lisp: same fix
- cl-tui.asd: version 0.2.0 → 0.3.0
- render.lisp: component-children default method (c t) nil for
  protocol completeness (component-parent already had this)
- Theme class with role→hex hash table, mode (dark/light)
- theme-color reader/writer (gethash based)
- define-preset macro with dark and light variants
- load-preset function with keyword lookup
- 2 built-in presets: default (gold) and nord
- 30+ semantic roles per preset (primary, accent, error, syntax-*, etc.)
- 9 theme tests: create, set/get, unknown, dark/light presets,
  nord, unknown-warn, switch-mode
- 57 total component tests, 100% GREEN
Fixes from subagent review:
- ASDF version 0.3.0 → 0.4.0
- define-preset now checks (check-type name keyword) at macro-expand time
- load-preset-unknown-warns test now uses (signals warning ...) to
  actually verify the warning fires (was false-positive before)
Four new modules:
- input.lisp: terminal raw mode, escape sequence parser, key/mouse event
  structs, read-event backend integration
- text-input.lisp: single-line text input with cursor, insertion,
  deletion, ctrl-A/E/W/U/K, on-submit callback, max-length
- textarea.lisp: multi-line text input with cursor up/down, newline,
  backspace joins lines, delete, undo/redo stack
- keybindings.lisp: layered keymap dispatch (global/local/focused),
  defkeymap macro, key spec matching with modifier prefixes

60 test assertions, 100% GREEN:
  RED: 0/12, 0/27, 0/30 — no tests existed
  GREEN: 60/60 across backend (27), box (58), input (60)

Dependencies: sb-posix for terminal raw mode (tcgetattr/tcsetattr)
Test files: 30 input tests covering all widgets and keybinding system
The .lisp files were edited directly during REPL-driven development.
Pushed all fixes back into the org source of truth:

- Fixed defstruct positional constructor wrappers
- Fixed case+string eql trap (cond+string=)
- Fixed CL string escape sequences (multiline literals)
- Fixed FiveAM closure capture (list boxing)
- Fixed textarea format calls (%join-lines helper)
- Fixed tangle paths for all 5 code blocks
- Consolidated all tests into single test block
- Updated key-match-p, dispatch-key-event, defkeymap macro

All 60 tests pass (100% GREEN).
37 per-function code blocks with prose explaining design reasoning,
edge cases, and CL traps. Combined tangle blocks at end for actual
compilation.

New scripts/tangle.py: reliable Python tangler (emacs --batch failed).
Added: %split-string, %join-lines, tangle helper.

CL traps documented in org prose:
- defstruct generates keyword constructors (no :constructor needed)
- case with strings uses EQL — use cond + string=
- CL strings: no \n escape — use (string #\Newline)
- FiveAM closure capture — use list boxing
- read-byte is package-locked — use read-raw-byte
- ASDF compile-file stricter than LOAD — debug with LOAD

60 tests, 100% GREEN.
ScrollBox:
- Container with vertical/horizontal scroll, viewport culling
- Scroll offset (:scroll-y, :scroll-x) with clamp to valid bounds
- Scrollbars rendered when content exceeds viewport
- Sticky scroll (auto-scroll to bottom on content change)
- Component protocol: component-children, component-layout-node

TabBar:
- Horizontal tab row with active/inactive styling
- tab-bar-next/prev (wraps around), tab-bar-select, tab-bar-handle-key
- Tab title rendering with overflow truncation (ellipsis)
- Component protocol: component-layout-node

26 scrollbox+tabbar tests, 100% GREEN:
171 total (27 backend + 58 box + 60 input + 26 scrollbox)

Review fixes applied:
- Removed duplicate definitions (org per-function blocks are prose-only)
- Fixed ASDF test path (../../tests/...)
- Version bumped to 0.6.0
- Added clamp-scroll export
- Added tab-bar-next/prev/select/handle-key tests
- Added scroll clamp boundary tests
Select widget: list of options with keyboard navigation (up/down/enter/esc,
ctrl+n/p), case-insensitive substring filter with character-set Jaccard
fuzzy fallback, category headers, viewport culling, on-select callback.

Fixed from subagent review:
- Category filter return-from bug: categories kept in filtered set
- Dead trigram code removed (string-trigrams, trigram-score)
- Exports cleaned up (removed unused trigram exports)
- Character-set Jaccard replaces trigrams (better for short strings)

25 select tests, 100% GREEN.
196 total (27 backend + 58 box + 60 input + 26 scrollbox/tabbar + 25 select)
Add cl-tui.markdown package with:
- Markdown parser: headings, paragraphs, bold, italic, inline-code, links,
  code blocks, blockquotes, lists, thematic breaks
- Syntax highlighting: Lisp, Python, JavaScript, Bash with keyword,
  builtin, comment, number, function coloring
- Diff renderer: colorized unified diff (+/-/@ lines)
- Terminal renderer: ANSI escape sequences via backend-style functions
- 67 tests, 100% passing
- All parser helpers use values returns (not cons) for multiple-value-bind

ASDF: v0.7.0 -> v0.8.0, new markdown module + test suite
Rename: cl-tty avoids naming collision with Quicklisp's cl-tui (naryl/cl-tui,
a cl-charms-based ncurses library). Our project is pure escape-sequence CL.

v0.9.0 adds:
- Dialog base class: modal overlay with backdrop, centered panel, size
  variants (:small/:medium/:large), stack-based management
- Dialog subclasses: alert, confirm, select-dialog, prompt-dialog
- Toast notifications: transient, top-right corner, auto-dismiss,
  colored variants (info/success/warning/error)
- 78 tests total, 100% passing

ASDF: read-time package references (+fiveam:+) replaced with
find-symbol so .asd loads without FiveAM pre-loaded
- mouse-mixin class with on-mouse-down/up/move/scroll handler slots
- handle-mouse-event dispatches to the right handler by event type
- hit-test finds deepest component at (x,y) coordinates
- selection struct + get-selection + copy-to-clipboard
- SGR mouse parsing already existed in input system (mouse-event struct,
  parse-sgr-mouse function, CSI dispatch in %read-escape-sequence)
- 3 tests, 100% passing
- defslot: register render functions into named slots with ordering
- slot-render: call all registered render-fns for a slot
- Slot modes designed (stack/replace/single-winner) but mode dispatch
  is implicit via the registration API
- slot-p, clear-slot, list-slots for lifecycle management
- Slots stored in a hash table keyed by string (equal test)
- 4 tests, 100% passing
- README.org with overview, architecture, component table, quick start
- demo.lisp — working TUI demo exercising multiple components
- run-all-tests.lisp — single-script test runner
- ROADMAP updated with v1.0.0 documentation milestone
- Full test suite: ~280 checks, 100% passing across 9 suites
- draw-border needs :style keyword before :single/:double
- draw-text needs fg and bg color keywords
- demo renders correctly in a real terminal
- Tested with: (sleep 2; echo q) | script -q -c 'sbcl --script demo.lisp'
- Three tabs: Home, Components, Stats with different content
- Real keyboard input: arrow keys to switch tabs, q to quit
- CSI escape sequence parsing for arrow keys
- Footer bar shows current tab position
- Tab bar highlights active tab in bright blue
Fixes from subagent code review (15 findings):

CRITICAL runtime bugs:
- dialog.lisp: backend-write calls -> draw-rect/draw-text (wrong arg count)
- dialog.lisp: removed undefined render-component call
- dialog.lisp: toast render backend-write -> draw-text

MAJOR data loss / silent failures:
- textarea.lisp: undo overflow now drops oldest entry instead of wiping stack
- scrollbox.lisp: :background-element -> :bright-black (theme keyword never resolved)

ASDF completeness:
- modern-tests.lisp wired as component and test-op suite
- layout tests added to test-op suite list
- markdown suite lookup now uses keyword (was looking up wrong string)
- test runner updated to match

API cleanup:
- container-package: removed duplicate render export
- select-package: removed duplicate render export
- markdown.lisp: #\Escape -> #\Esc for consistency
- textarea.lisp: removed duplicate %split-string defn

Demo robustness:
- Added unwind-protect for guaranteed terminal cleanup
- Uses make-modern-backend constructor
- Uses set-raw-mode/restore-terminal-state

Layout:
- normalize-box handles partial padding specs (was returning all zeros)
Root cause: normalize-box and slot :initforms used quoted literal
lists ('(...)) that were destructively modified by (setf (getf ...)).
Each call to normalize-box with a non-nil spec corrupted the shared
default list, causing all subsequent nodes with no explicit padding
to inherit the previous node's padding values.

Fix: replace all '(...) quoted literals with (list ...) constructor
calls — in normalize-box (3 paths) and in slot initforms for both
padding and margin.

All 11 test suites now pass: 358/358 checks, 0 failures.
LICENSE:
- Added GNU General Public License v3.0
- Updated README.org to reflect GPL 3.0

ROADMAP:
- Complete rewrite to reflect actual project state
- Removed croatoan/ncurses/Yoga FFI references
- Marked all 11 existing versions DONE
- Added v0.12.0-0.14.0 for new features (detection, pipeline, mouse)

DETECTION (v0.12.0):
- detect-backend: auto-detect modern vs simple backend
- detect-backend-by-env: check COLORTERM env var
- detect-backend-by-tty: check interactive-stream-p
- detect-backend-by-da1: query terminal via ESC[c (best-effort)
- *detected-backend* cache for zero-cost subsequent calls
- Added detection.lisp to ASDF and package exports
- Added 2 new tests (360 total, all passing)
- demo.lisp updated to use detect-backend

ORG BACKPORT (pre-existing fixes synced):
- dialog.org: render-dialog/render-toast fixes, class initforms
- scrollbox-tabbar.org: background-element -> bright-black, remove duplicate render
- select.org: remove duplicate render export
- text-input.org: remove duplicate %split-string, undo overflow fix
- layout-engine.org: quoted-literal -> list constructors, normalize-box rewrite
- mouse.org: add missing exports, fix test
New module: src/rendering/framebuffer.lisp (tangled from org/framebuffer.org)

- framebuffer-backend class: implements backend protocol by writing to
  2D cell array instead of emitting escape sequences
- cell struct: per-cell state (char, fg, bg, bold, italic, underline, link-url)
- make-framebuffer / framebuffer-width / framebuffer-height
- draw-text, draw-rect, draw-border, draw-link, draw-ellipsis methods
- diff-framebuffers: compares two framebuffers, returns changed cells
- flush-framebuffer: diff + output changes to real backend
- with-scissor macro: clip drawing operations to rectangle
- cursor-move: added default no-op method for all backends
- 20 new tests, all passing (372 total)

Version bumped from 0.11.0 to 0.13.0.
License field set to GPL-3.0 in ASDF.
CRITICAL: case b → cond in %read-event (input.lisp:280)
  case with (and ...) predicate clauses treats keys as eql-compared
  atoms — all range clauses were dead code.  Every Ctrl+letter and
  printable ASCII fell through to :unknown.  text-input/textarea
  widgets were non-functional with real terminal input.  No test
  coverage of %read-event masked this.

HIGH: Theme resolution wired (backend/modern.lisp, theme.lisp)
  sgr-fg/sgr-bg now fall back to *theme-colors* hash for semantic
  keywords (:accent, :text-muted, :background-element).  *theme-colors*
  exported from cl-tty.backend.  load-preset populates it from preset
  hex values.  Previously all themed render output was invisible.

HIGH: SGR mouse parser wired (input.lisp:210-215)
  parse-sgr-mouse was defined but never called.  Now %read-escape-sequence
  detects ESC[< prefix and routes to parse-sgr-mouse.  Mouse drags,
  releases, and scroll events now parse correctly.

MEDIUM: Rendering stubs replaced
  - scrollbox: delegates to (render child backend) with position
    offset via unwind-protect (was debug string 'child at ~D')
  - text-input: draws value/placeholder at layout position
  - textarea: draws visible lines at layout position

MEDIUM: hit-test uses component-layout-node (mouse.lisp:18-31)
  Was checking nonexistent x/y/width/height slots.  Now reads
  layout-node-x/y/w/h via component-layout-node generic.

MEDIUM: test runner exit code (run-all-tests.lisp, cl-tty.asd)
  run-all-tests.lisp exits 1 if any suite fails.
  asdf:test-system exits 1 on failure.
  Renamed :cl-tty-tests to :cl-tty/test (ASDF convention).

MEDIUM: draw-border respects x/y on simple-backend (simple.lisp:42-53)
  Was writing to cursor position only.  Now uses newlines+spaces
  to reach specified coordinates (no escape sequences needed).

LOW: TabBar truncation off-by-one fixed (tabbar.lisp:47)
  >= changed to > to avoid cutting tabs 2 chars early.

LOW: Scrollbar coordinates absolute (scrollbox.lisp:61-73)
  Scrollbar drawn at viewport-relative (0,0).  Now adds layout
  node x/y offset for correct terminal positioning.

LOW: backend-write calls finish-output (modern.lisp:169)

LOW: load-preset no longer flips theme-mode (theme.lisp:43-45)
  Mode toggle caused load-preset to load wrong variant on
  second call.

All backported to org source files (org/text-input.org,
org/scrollbox-tabbar.org) so tangling produces matching .lisp.

392 tests pass, exit code 0.
CRITICAL: Remove (declare (ignore w)) from textarea render (textarea.lisp:251)
  w is used for horizontal truncation on the next line.  Declaring it
  ignored while using it is undefined behavior in CL (SBCL warns).

HIGH: hit-test recurses into children (mouse.lisp:18-34)
  Was returning the root component for any click within its bounds,
  ignoring nested widgets entirely.  Now checks component-children
  first, returning the deepest match.

MEDIUM: Select/TabBar position hardcoded to (0,0)
  Both rendered at terminal origin regardless of layout position.
  Now read layout-node-x/y for absolute positioning.

MEDIUM: Text-input truncation missing
  Render drew full value string even when exceeding widget width.
  Now truncates to (min (length display) w).

MEDIUM: X10 mouse release detection added (input.lisp:219-226)
  X10 encoding uses button=3 for release.  Was detecting all events
  as press/drag.  Now checks button=3 → :release.

MEDIUM: parse-csi-params handles private markers (input.lisp:128-131)
  < = > ? characters (0x3c-0x3f) treated as parameter start markers
  instead of accumulating bogus digit values.  Latent trap removed.

Deferred (pre-existing design):
- Scrollbox visibility cy vs orig-y: match for column layout (common case)
- Nested scrollbox coordinates: assumes sequential layout positions
- text-input cursor drawing: feature, not bugfix

392 tests pass.
org/text-input.org: remove (declare (ignore w)) from textarea render;
  add truncation to text-input render (subseq display 0 w)
org/mouse.org: hit-test now uses component-layout-node and recurses
  into children for deepest-match hit testing
org/select.org: render reads layout-node-x/y instead of hardcoded (0,0)
org/scrollbox-tabbar.org: tabbar render reads layout-node-x/y
  instead of hardcoded (0,0); x-pos starts at x offset

All 4 org files tangled clean. 392 tests pass.
Demo (demo.lisp):
  - Full interactive demo with 3 tabs: Home, Widgets, Console
  - Uses read-event/SGR mouse paths (exercises real terminal input)
  - Demonstrates text-input, textarea, backend drawing, tab navigation
  - Event log console shows keyboard and mouse events in real time
  - Proper terminal cleanup via shutdown-backend + unwind-protect

README.org:
  - Complete rewrite with getting-started guide, architecture overview
  - API reference for all components with signatures and examples
  - Event loop pattern, layout system, rendering pipeline docs
  - Backend features table, development guide, project structure

Bug fixes:
  - read-raw-byte (input.lisp:89-109): use sb-sys:with-pinned-objects +
    vector-sap for proper sb-posix:read buffer handling (SBCL type error
    with plain (unsigned-byte 8) arrays)
  - input-package.lisp: export textarea-lines (was missing from package)

Version bump: v0.14.0 → v0.15.0

392 tests pass.
demo.lisp:
  - Removed ignore-errors wrapper: run-demo now returns normally,
    followed by (uiop:quit 0) at top level — fixes exit code always 1 bug
  - Manual set-raw-mode/unwind-protect/restore-terminal-state instead of
    with-raw-terminal macro (safer in edge cases)
  - Graceful fallback when raw mode fails: continues in pipe-safe mode
    so the demo renders frames even without terminal control
  - Simplified tab rendering, fixed textarea-lines display

The demo runs correctly in both interactive and pipe-safe modes.
In a real terminal: raw mode, keyboard/mouse event loop.
In pipe-safe mode: spins rendering frames (read-event returns nil).

Verified running: frames render correctly with borders, tabs, content,
status bar, and event counter.
demo.lisp now registers the current directory as a quicklisp project
source and falls back to direct asdf:load-system if quicklisp can't
find cl-tty.  Lets the demo run from a fresh git clone without
symlinking into ~/quicklisp/local-projects/.
The demo now guards the quickload with a (find-package :cl-tty.backend)
check first, tries ql:quickload inside ignore-errors, and falls through
to direct (load cl-tty.asd) + (asdf:load-system :cl-tty) if the
package still isn't loaded.  Works in --disable-debugger mode where
Quicklisp's SYSTEM-NOT-FOUND continuable error kills the process.
No Quicklisp needed at all.  Works from a fresh git clone with
just SBCL installed.  Registering the current directory in ASDF's
central-registry is enough to find cl-tty.asd.
make-raw-termios (input.lisp:66-67): termios-cc accessor in SBCL 2.5.x
takes one arg (the struct) and returns the cc array.  Use (aref ...)
to set individual control characters.  Old code used 3-arg setf form
that no longer works and produced style warnings.

demo.lisp: Now exits with a clear error message when raw mode can't
be established, rather than running in broken pipe-safe mode where
escape sequences are echoed and input is line-buffered.
set-raw-mode now uses (stty raw -echo ...) via sb-ext:run-program
instead of sb-posix:tcgetattr/tcsetattr + termios flag manipulation.
The sb-posix termios API changed between SBCL versions (termios-cc
accessor went from 2-arg to 1-arg), and tcgetattr fails in some
container/PTY environments.

Stty is available on every Unix and is independent of SBCL's
sb-posix version.  set-raw-mode errors if stty -g returns empty
(no real terminal attached).  restore-terminal-state is a no-op
when called with nil.
stty now operates on /dev/tty explicitly (-F flag) instead of
relying on stdin inheritance.  This is more reliable in SBCL's
--script mode where stdin may be handled differently by run-program.
Also ensures stty always targets the controlling terminal regardless
of how the subprocess is spawned.
The -F flag isn't available on all stty implementations.  Using
shell stdin redirect (stty ... < /dev/tty) via /bin/sh is more
portable and doesn't depend on run-program preserving the
controlling terminal across subprocess boundaries.
Added ./demo shell script that sets raw mode via stty before running
the Lisp demo and restores it on exit (including SIGINT/SIGTERM).

demo.lisp no longer attempts to set raw mode from inside SBCL —
terminal raw mode is the shell's responsibility.  This avoids the
recurring problem of sb-ext:run-program + stty not being able to
access the controlling terminal from inside sbcl --script.
Raw terminal mode must be set by the parent process (the shell),
not from inside SBCL.  sb-ext:run-program subprocesses cannot
reliably access the controlling terminal for stty operations.
./demo.sh sets raw mode via stty, runs sbcl --script demo.lisp,
and restores terminal state on exit (EXIT, INT, TERM).

demo.lisp no longer calls with-raw-terminal — it assumes the
calling shell has already set raw mode.
amr merged commit 9e5b1ee8c6 into main 2026-05-11 22:03:18 -04:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: amr/cl-tty#7