Laya-MLX runs Laya typed decisions natively on Apple Silicon in 13 milliseconds, and the repository picked up 5,394 stars in three days

A typed-decision model that used to need a Python-and-PyTorch stack now runs natively on Apple Silicon, in a native MLX loop. The unofficial repository mizorewww/laya-mlx went up on GitHub on 19 September 2026 and had 5,394 stars three days later, on 22 September. It ports the Laya open-source typed-decision model, from Convai Innovations, to Apple's MLX framework. The README calls this an independent MLX port and points at the upstream project as the source of the weights, prompt format, calibration and output schema.

The reported M3 Max latency numbers

The README reports median end-to-end latency of 13.42 ms for one short English typed decision on an M3 Max with 40 GPU cores and 128 GiB of memory, in FP16. The multilingual checkpoint clocks 7.39 ms median. P95 latency is 13.92 ms and 7.79 ms for the two checkpoints. On a 50-question batch at batch_size 64, throughput reaches 146.8 questions per second for the 421M model and 395.0 for the 322M multilingual model. Peak MLX allocation for one short question is 943.6 MiB and 687.6 MiB respectively. The measured environment was macOS 27.2, Python 3.12.13 and MLX 0.32.2.

Port fidelity

The README states that all three checkpoints matched the upstream selected answer on 63 of 63 validation questions in both FP32 and FP16, so 378 comparisons in total. Each configuration also passed 100 repeated finite deterministic calls with zero measured active-memory growth. The three supported checkpoints are convaiinnovations/laya (421M, ModernBERT-large encoder, 512 token context, English), convaiinnovations/laya-multilingual (322M, mmBERT-base encoder, 1,024 tokens), and convaiinnovations/laya-typed-decisions (421M, ModernBERT-large, 1,024 tokens). The maintainer publishes pre-converted FP16 checkpoints on Hugging Face under the account aac6fef, with 36 files that all passed strict remote checksum verification, and every published card carries the model card, validation results, provenance, licence and file checksums.

The scope of the port

The encoder, decision Transformer, scoring head and action head all run in MLX. Tokenisation uses Hugging Face's Rust tokeniser. The runtime supports the three Laya question types: choice returns probabilities over named options, score returns probabilities over ordered rubric levels and their expected score, and noul returns P(true) for a proposition. Question rows are batched independently, and the maintainer notes that the encoder representation depends on both state and question, so the runtime does not claim to encode the state once and reuse hidden states across arbitrary questions. Inference runs entirely on the local machine, with no cloud API, and with no PyTorch or Transformers dependency. Training and fine-tuning stay in the upstream project.

Snake as a live demo

The repository ships a terminal Snake demo. In the paired M3 Max test, the optimised path runs 75.40 moves per second across 2,400 moves, with zero deaths and two visible safety interventions, and the optimised run was 6.5 percent faster than the same-run eager control. The demo runs on Apple Silicon with a terminal at least 104 by 35 cells.

For a team building on Apple Silicon, routing calls, JSON-shaped answers, and yes-or-no gates that used to sit in a cloud API can now sit inside a local Python process, at 13 ms per short decision, with the weights on disk. The latency and accuracy claims come from the maintainer alone, and the port has been public for three days, so long-term reliability is still an open question for anyone outside the project.

Source

mizorewww/laya-mlx on GitHub, README as of 22 September 2026. Checkpoints on Hugging Face at aac6fef/laya-mlx.