passepartout

amr/passepartout

Fork 0

Commit Graph

Author	SHA1	Message	Date
Amr Gharbeia	d3b74f5c88	v0.4.1: native embedding CFFI — full pipeline working, daemon-wired, HITL bug fixed - Native backend returns 768-dim vectors via llama.cpp / C wrapper (/usr/local/lib/libllama_wrap.so) - Wired :native into embed-object dispatch and exported from passepartout package - Model preloads at daemon startup with EMBEDDING_PROVIDER=native (~30s) - Lazy loading via embedding-backend :native also works (first call ~45s) - C wrapper bridges CFFI pointer params to llama.cpp struct-by-value API - Correct struct layouts: llama_model_params(72B), llama_context_params(136B), llama_batch(56B) - BERT pooling: llama_get_embeddings_seq, llama_tokenize takes vocab* not model* - FiveAM tests pass: dimensions, self-similarity, semantic ranking - Fixed pre-existing HITL crash: boundp guard for hitl-pending in core-loop-act - Lazy load guard prevents double-load of native file in embedding-native-ensure-loaded - ROADMAP: v0.4.0 items marked DONE, v0.4.1 native embedding updated with actual implementation	2026-05-07 09:55:33 -04:00
Amr Gharbeia	cd752bb4ad	v0.4.1: native embedding — CFFI binding for llama.cpp (REPL prototype) Some checks failed Deploy (Gitea) / deploy (push) Failing after 2s Details RED: embedding-backend-native does not exist. No CFFI llama binding. GREEN (REPL progress): - cffi:define-foreign-library libllama → loaded - defcstruct with correct sizes (verified via C sizeof program): llama-mparams (72 bytes), llama-cparams (136 bytes), llama-batch (56) - Field offsets verified via C offsetof program - llama_backend_init discovered as required prerequisite - llama-model-default-params correctly fills 72-byte struct (verified) - llama-embedding CLI verified: 768-dim vectors, 22ms/4tokens BLOCKED: llama_model_load_from_file segfaults via CFFI. Suspect struct-by-value vs pointer ABI mismatch on x86-64. Needs interactive SBCL REPL to debug the calling convention (structs >16 bytes passed by hidden reference on SysV). CFFI bindings preserved in org/system-model-embedding-native.org for continued REPL work. Includes: model load, context create, tokenize, encode, embeddings-ith, batch init/free. Model: nomic-embed-text-v1.5.Q4_K_M.gguf (80MB, 768-dim, nomic-bert) at ~/.local/share/passepartout/models/	2026-05-06 21:34:03 -04:00

Author

SHA1

Message

Date

Amr Gharbeia

d3b74f5c88

v0.4.1: native embedding CFFI — full pipeline working, daemon-wired, HITL bug fixed

- Native backend returns 768-dim vectors via llama.cpp / C wrapper (/usr/local/lib/libllama_wrap.so)
- Wired :native into embed-object dispatch and exported from passepartout package
- Model preloads at daemon startup with EMBEDDING_PROVIDER=native (~30s)
- Lazy loading via *embedding-backend* :native also works (first call ~45s)
- C wrapper bridges CFFI pointer params to llama.cpp struct-by-value API
- Correct struct layouts: llama_model_params(72B), llama_context_params(136B), llama_batch(56B)
- BERT pooling: llama_get_embeddings_seq, llama_tokenize takes vocab* not model*
- FiveAM tests pass: dimensions, self-similarity, semantic ranking
- Fixed pre-existing HITL crash: boundp guard for *hitl-pending* in core-loop-act
- Lazy load guard prevents double-load of native file in embedding-native-ensure-loaded
- ROADMAP: v0.4.0 items marked DONE, v0.4.1 native embedding updated with actual implementation

2026-05-07 09:55:33 -04:00

Amr Gharbeia

cd752bb4ad

v0.4.1: native embedding — CFFI binding for llama.cpp (REPL prototype)

Deploy (Gitea) / deploy (push) Failing after 2s

Details

RED: embedding-backend-native does not exist. No CFFI llama binding.

GREEN (REPL progress):
- cffi:define-foreign-library libllama → loaded
- defcstruct with correct sizes (verified via C sizeof program):
  llama-mparams (72 bytes), llama-cparams (136 bytes), llama-batch (56)
- Field offsets verified via C offsetof program
- llama_backend_init discovered as required prerequisite
- llama-model-default-params correctly fills 72-byte struct (verified)
- llama-embedding CLI verified: 768-dim vectors, 22ms/4tokens

BLOCKED: llama_model_load_from_file segfaults via CFFI. Suspect struct-by-value
vs pointer ABI mismatch on x86-64. Needs interactive SBCL REPL to debug the
calling convention (structs >16 bytes passed by hidden reference on SysV).

CFFI bindings preserved in org/system-model-embedding-native.org for
continued REPL work. Includes: model load, context create, tokenize,
encode, embeddings-ith, batch init/free.

Model: nomic-embed-text-v1.5.Q4_K_M.gguf (80MB, 768-dim, nomic-bert)
at ~/.local/share/passepartout/models/

2026-05-06 21:34:03 -04:00

2 Commits