========================
semlix 3.2 release notes
========================

semlix 3.2.0
============

A modernization, performance and tooling release. Backward compatible for
Python 3 users; **Python 2 support was removed** (it had been unmaintained for
years). The canonical change list is in ``CHANGELOG.md``.

Highlights
----------

* **Engine guide** (:doc:`/engines`): a measured comparison of the two lexical
  engines. On 20k documents the bm25s engine indexes ~40x faster and queries
  ~57x faster than the pure-Python core — use it for raw lexical speed and the
  lexical leg of hybrid search; use the core for its full feature set.

* **Optional native acceleration** (:doc:`/native`): ``SEMLIX_COMPILE=1``
  compiles a curated set of hot modules with mypyc; pure-Python stays the
  default and the automatic fallback.

* **Faster hybrid search**: lexical and semantic searches now run concurrently
  and the unused side is skipped when ``alpha`` is 0/1; a bounded query-embedding
  cache avoids re-embedding repeated queries.

* **Incremental BM25s indexing**: only new documents are tokenized on add
  (instead of re-tokenizing the whole corpus each commit).

* **StandardAnalyzer indexing fast path**: a fused tokenize+lowercase+stop+count
  loop, asserted bit-for-bit identical to the generic pipeline.

Breaking / compatibility
------------------------

* **Requires Python >= 3.9.** Python 2 support and the ``cached-property``
  dependency were removed; the core install now has no third-party runtime
  dependencies.

Security
--------

* ``NumpyVectorStore`` persistence is now pickle-free (numpy ``.npz`` + JSON,
  loaded with ``allow_pickle=False``), removing the arbitrary-code-execution
  risk of unpickling a shared/untrusted index. Legacy pickle stores still load
  with a deprecation warning and migrate on the next save.

Fixes
-----

* ``varint_to_int`` crashed on Python 3 (``ord()`` on a bytes element).
* ``BM25sStore.search``: dropped the ``update_vocab`` kwarg removed in bm25s
  0.3.9, corrected query tokenization, and clamped ``k`` to the corpus size.
* ``codec/whoosh3.py``: ``elif fixedsize is 0`` → ``== 0``.

See Also
--------

* :doc:`/engines` -- choosing an engine
* :doc:`/native` -- optional native build
* ``CHANGELOG.md`` -- full change list