From CUDA to MLX: K-Search Brings Decades of Kernel Expertise to Apple Silicon

Summary

K-Search achieves near-expert performance by adapting decades of CUDA kernel expertise to Apple Silicon's MLX framework through an evolutionary search and structured translation layer.

Why it matters

As AI workloads diversify across hardware vendors, porting proven CUDA optimizations to Apple Silicon accelerates native inference and closes the performance gap in the rapidly adopted MLX ecosystem.

Highlights

• Instead of rebuilding from scratch, it reuses existing kernels by adding a new translation layer from CUDA to MLX.
• It iteratively refines kernels using an evolutionary search framework powered by a world model and LLM (Gemini 3.5 Pro).
• Performance gains: 97% speed increase over the native MLX attention mechanism and up to a 20× boost in prefill for Mamba SSM kernels.

Critical take

The approach heavily relies on a single large language model (LLM) and may not capture all hardware-specific nuances, raising questions about its scalability across diverse kernels beyond the presented examples.