v0.4.1: native embedding CFFI — working backend init, model metadata loads
Some checks failed
Deploy (Gitea) / deploy (push) Failing after 2s

Key discoveries:
- llamba_backend_init works (after sb-int:set-floating-point-modes :traps nil)
- llama_model_default_params fills 72-byte struct correctly
- Bad path test: returns NULL pointer, SBCL handles gracefully
- Real model: loads metadata (768-dim, 12-layer nomic-bert), then
  NULL pointer in weight init (likely tensor_split/devices field)

Standalone test file: test/test_native_embedding_standalone.lisp
Reproduced: sbcl --load quicklisp/setup.lisp --eval '(ql:quickload :cffi)'
         --load test/test_native_embedding_standalone.lisp

Next: GDB debugging session needed to pinpoint which struct field
causes the NULL dereference during Model weight loading.
This commit is contained in:
2026-05-06 22:09:36 -04:00
parent f28363dc45
commit 52a8386282

View File

@@ -0,0 +1,45 @@
(cffi:define-foreign-library libllama (:unix "/usr/local/lib/libllama.so"))
(cffi:use-foreign-library libllama)
(cffi:defcfun ("llama_backend_init" bl) :void)
(cffi:defcfun ("llama_model_default_params" mdp) :void (p :pointer))
(cffi:defcfun ("llama_load_model_from_file" lml) :pointer (path :string) (params :pointer))
(cffi:defcfun ("llama_n_embd" ne) :int32 (m :pointer))
(cffi:defcfun ("llama_free_model" fm) :void (m :pointer))
;; Aggressively clear all FP trap modes
(sb-int:set-floating-point-modes :traps '())
(format t "FP modes set~%") (finish-output)
;; Init
(handler-case (bl) (error (c) (format t "bl err: ~a~%" c)))
(format t "Backend init done~%") (finish-output)
;; Small model test: try with a path that doesn't exist to get a quick error
(let ((mp (cffi:foreign-alloc :uint8 :count 72)))
(dotimes (i 72) (setf (cffi:mem-aref mp :uint8 i) 0))
(mdp mp)
(handler-case
(let ((m (lml "/nonexistent/model.gguf" mp)))
(format t "Model from bad path: ~a~%" m))
(error (c) (format t "Bad path ERR: ~a~%" c)))
(cffi:foreign-free mp))
(format t "Bad path test done~%") (finish-output)
;; Now try real model
(let ((mp (cffi:foreign-alloc :uint8 :count 72)))
(dotimes (i 72) (setf (cffi:mem-aref mp :uint8 i) 0))
(mdp mp)
(setf (cffi:mem-aref mp :int32 4) 0) ;; n_gpu_layers
(format t "Loading real model...~%") (finish-output)
(handler-case
(let ((m (lml "/home/user/.local/share/passepartout/models/nomic-embed-text-v1.5.Q4_K_M.gguf" mp)))
(format t "Model = ~a (null=~a)~%" m (cffi:null-pointer-p m))
(finish-output)
(when (not (cffi:null-pointer-p m))
(format t "n_embd = ~a~%" (ne m))
(fm m)))
(error (c) (format t "Real model ERR: ~a~%" c)))
(cffi:foreign-free mp))
(format t "Done~%")