From d2387f10743880a0138573f9e72b3e9c24233823 Mon Sep 17 00:00:00 2001 From: Hermes Date: Sun, 24 May 2026 22:54:02 +0000 Subject: [PATCH] add note: viability of open-source Wolfram/Mathematica in Common Lisp --- ideas/open-source-wolfram-lisp.org | 74 ++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 ideas/open-source-wolfram-lisp.org diff --git a/ideas/open-source-wolfram-lisp.org b/ideas/open-source-wolfram-lisp.org new file mode 100644 index 0000000..800470a --- /dev/null +++ b/ideas/open-source-wolfram-lisp.org @@ -0,0 +1,74 @@ +:PROPERTIES: +:CREATED: [2026-05-24 Sun] +:ID: 7a8b9c0d-1e2f-3a4b-5c6d-7e8f9a0b1c2d +:END: +#+title: Viability of an Open-Source Wolfram Language / Mathematica in Common Lisp +#+filetags: :ideas:lisp:mathematics:open-source: + +An assessment of what it would take to build a viable open-source equivalent of Wolfram Language and Mathematica in Common Lisp, based on the existing ecosystem and the fundamental architectural alignment between Lisp and symbolic computation. + +**The alignment is natural, not forced.** + +Wolfram Language is, at its core, a term-rewriting system with pattern matching, rule-based transformation, and a uniform symbolic representation for everything (expressions are trees of the form head[arg1, arg2, ...] — the Wolfram equivalent of cons cells). This is very close to what Lisp is natively. A Lisp implementation of the core evaluator — pattern matching, rule application, substitution, term rewriting — is not a foreign port. It is an exercise in expressing Wolfram's semantics in a language whose semantics were designed for the same problem domain. + +Maxima proves this historically. It is a direct descendant of Macsyma, the MIT computer algebra system that inspired Mathematica. Macsyma was written in Lisp. Mathematica's core evaluation model inherits heavily from Macsyma. An open-source Common Lisp computer algebra system already exists, has existed for decades, and works. The question is not whether it can be done, but how much of the modern Mathematica ecosystem can be replicated and at what cost. + +**What already exists.** + +| Layer | Existing CL work | Status | +|---|---|---| +| Symbolic engine / term rewriting | Lisp readers, pattern matching libs (trivia, optima, fare-matcher), rule systems | Foundational primitives exist, no unified Wolfram-equivalent evaluator | +| Computer algebra system | Maxima, FriCAS (Axiom), reduce-algebra | Mature CASes — Maxima alone has differentiation, integration, ODEs, linear algebra, tensors, series, limits, Laplace transforms | +| Numerical computing | magicl, lla (Lisp Linear Algebra), CL-NUM, GSLL (GNU Scientific Library bindings) | Solid — covers BLAS, LAPACK, random numbers, special functions, optimization | +| Visualization | cl-cairo2, Vecto, CLG, CommonQt, cl-zxing | Exists but scattered — no unified plotting framework like Mathematica's | +| Notebook interface | cl-jupyter, common-lisp-jupyter, Lem | Jupyter kernel works. Lem is a native editor approaching notebook capability. No Mathematica-level notebook yet. | +| Rule-based programming | fare-matcher, optima, prolog implementations | Pattern matching is good. Full term-rewriting system needs assembly. | +| Knowledge graph | gbrain, various triplestore libs | Possible but would need Wolfram Alpha-level investment | +| Deployment | ASDF, Quicklisp, SBCL standalone executables | Better than Mathematica's deployment story — Lisp produces real executables | + +**The hard parts.** + +1. **The standard library is the product, not the engine.** Mathematica ships thousands of built-in functions — every mathematical special function, every statistical distribution, every graph algorithm, every image processing filter, every string operation, every data format parser. This is not a technical challenge; it is a sheer volume problem. The open-source answer is to wrap existing C/C++ libraries (GSL for special functions, OpenCV for image processing, igraph for graph algorithms, etc.) and expose them through a unified symbolic interface. This is the Clasp approach: interop with mature C++ libraries rather than rebuilding everything from scratch in Lisp. The Wolfram equivalent would be a CLOS-based symbolic dispatch layer that wraps these libraries and makes them accessible through a consistent term-rewriting evaluator. + +2. **The notebook interface is a product in itself.** Mathematica's notebook is not a terminal with nice formatting. It is a computational notebook with inline typeset math, dynamic graphics, collapsible sections, live evaluation, and a rich document model. The Jupyter ecosystem solves half of this. A Lisp-native notebook would need a rendering engine for mathematical notation (LaTeX or MathJax integration), inline interactive graphics, and a document model compatible with literate computation. Lem is the most promising starting point — it already has Emacs-like extensibility, a GTK frontend, and a Lisp-native codebase. Extending Lem to support computational notebooks with inline graphics and typeset output is the shortest path. + +3. **Performance for specialized domains.** Mathematica's kernel is highly optimized for symbolic operations — pattern matching over large expressions, automatic algorithm selection, memoization, and incremental compilation. A naive Lisp implementation would match Mathematica for small-to-medium expressions but would need significant optimization work for the heavy cases (symbolic integration of large expressions, graph operations on million-node graphs, image processing pipelines). The advantage is that Lisp's compiler infrastructure (SBCL's type inference, VOPs, inlining) gives a much better baseline than most languages. SBCL can generate code that approaches C speed for numerical kernels. + +4. **The knowledge graph (Wolfram Alpha).** Mathematica's integration with Wolfram Alpha — querying computable data about chemistry, geography, finance, linguistics, etc. — is a separate product with a massive engineering investment in data curation. An open-source equivalent would not replicate this. It would either provide a local, user-curatable knowledge base (gbrain fits here) or integrate with existing open knowledge graphs (Wikidata, DBpedia). The gbrain connection is interesting: if Passepartout's knowledge store can answer factual queries with provenance, that becomes the Wolfram Alpha equivalent for the Lisp Mathematica. + +5. **Package ecosystem and community.** Mathematica's advantage is not just its engine but its ecosystem — thousands of paclets, the Wolfram Function Repository, the community that shares notebooks. An open-source equivalent needs a package manager (Quicklisp solves this for Lisp libraries), a repository for symbolic packages (a Wolfram Function Repository equivalent), and a critical mass of users who both use and contribute. Maxima has users but not contributors. The gap is community formation, not technical capability. + +**The viability assessment.** + +| Domain | Viability | Timeline estimate | Risk | +|---|---|---|---| +| Core symbolic evaluator | High — Lisp was designed for this | 6-12 months for working prototype | Low — well-understood problem | +| Computer algebra | High — Maxima already exists | Integrate now; polish 1-2 years | Low — needs UI/UX investment | +| Numerical computing | High — wrappers exist | 3-6 months for unified interface | Low — wrapping problem | +| Visualization | Medium — scattered pieces | 1-2 years for unified framework | Medium — needs new work | +| Notebook interface | Medium — Lem as foundation | 1-2 years to Mathematica parity | Medium — significant UX engineering | +| Standard library breadth | Low — volume problem | 3-5 years with community | High — needs sustained contribution | +| Knowledge graph | Low — curation cost | 2-3 years for basic integration | High — different product category | +| Deployment | High — Lisp executables | Works now | None | + +**The strategic question.** + +The real question is not "can we replicate Mathematica in Lisp" but "should we?" — and if so, for whom. + +Mathematica serves two distinct use cases: + +- **Interactive exploration** — a researcher types an integral, gets a result, visualizes it, iterates. Lisp + Maxima + a good notebook interface already does this, and the experience is competitive for anyone comfortable with Lisp syntax. +- **Deployed computation** — a company builds a production pipeline around Mathematica kernels, deploying computation as a service. Lisp executables are dramatically better here — they are real compiled binaries, not managed by a proprietary kernel, deployable without license fees, embeddable anywhere. + +For the second use case, the open-source Lisp alternative is already superior. The gap is the first use case: the interactive exploration experience, the breadth of built-in functions, and the cultural acceptance of Lisp syntax in communities that currently write Wolfram Language. + +The most viable path is not to clone Mathematica but to integrate Maxima + numerical Lisp libraries under a unified symbolic interface, expose all of it through a Lem-based notebook, and make the Jupyter bridge the primary entry point for users who prefer Python notebooks. This gives you 80% of Mathematica's capability with a fraction of the development cost, and it connects to the existing scientific Python ecosystem rather than competing with it. + +**The deeper point.** + +Mathematica's architecture is Lisp-like because it was inspired by a Lisp system (Macsyma). An open-source Mathematica in Lisp is not a port. It is a return to the original architectural vision, implemented in the language that vision was originally expressed in. The question is whether the community investment materializes — and that depends on whether there is a use case that justifies it. Passepartout's verification infrastructure may be that use case: a verified symbolic computation engine that reasons about its own results is a Mathematica-like system by necessity, and building it in Lisp is the natural path. + +--- + +- [[id:f4e5d6c7-b8a9-0c1d-2e3f-4a5b6c7d8e9f][Schafmeister and Clasp]] — Lisp in computational nanotechnology, existence proof for Lisp viability in scientific computing +- [[id:1c3ec48b-446c-50d2-b53e-126a81f5143f][Passepartout Architecture]] — why Lisp and where the symbolic engine fits