Hosstia/AliceAI-T5-35B-A0.6B-MLX-6bit main
Documentation references model or LLM outputContains multiple standalone Cyrillic words
SHA-256139edff8b522fc2c46b96bbcf83711150a43fd6f2aed28b9df9e55a6a9b21578
MaleculeH(Db)Md(Pt)
Evidence
3:3… How to turn a 69-gigabyte encoder-decoder model with 512 experts into an 18-gigabyte MLX project running on a MacBook with 48 GB of memory — without `mlx_lm`, without llama.cpp, and without any off-the-shelf solutions. **Updated (v2):** optimized from 18.9 tok/s to 72.7 tok/s using `mx.gather_qmm …
7:42… odels locally on Apple Silicon, the first thing that comes to mind is MLX, the framework from Apple Research with Metal GPU support. The `mlx_lm` library provides ready-made tools for converting and running inference on dozens of popular models: Llama, Qwen, Mistral, Phi — the …
9:193… _remote_code=True`, and 35 billion parameters?
10
11This article is a technical deep-dive into the real-world experience of converting Yandex's **AliceA …
449:28… aks After load_weights
450
451**Symptom**: after loading weights, the model produces random output.
452
453**Cause**: `lm_head.weight` is tied to `encoder.embed_tokens.weight` via `self.lm_head.weight = self.encoder.embed_tokens.wei …
749:66… me paragraph always triggers the same loop.
750- Example output: `"Ошибка: Неправильное написание имени «Шэнь Ань» — должно быть «Шэнь Ан»"` repeated dozens of times.
751- 4-bit and 6-bit variants **terminate cleanly** on the same paragraphs.
752
753### Roo …