Kaloyan Nikolov

Software engineer working on LLM inference and training. M.Sc. Computer Science @ RWTH Aachen.

---

Current focus

• Multi-Token Prediction (MTP) and speculative decoding for local inference
• KV cache quantization and Metal GPU kernel optimization
• Diffusion-based training for hybrid attention+linear architectures research
• Ternary weight quantization research

---

Active projects

omlx — personal fork with MTP decoding and Q4 KV cache with Hadamard rotation

sleepy-llm — Zig-native LLM inference engine with hand-tuned Metal kernels

sleepy-agent — fully local Android AI assistant, on-device Gemma 4 inference

qwen_orthrus — exploring Orthrus diffusion for ternary-weight and hybrid LLM architectures

---

Background

• Full port of a C++ speech recognition toolkit to Android using NDK.
• Work on a custom MHA PyTorch module
• Training of Attention and CTC multilingual ASR models with code-switching finetuning.

---

Stack

Python · Zig · C/C++ · TypeScript · Kotlin