arXiv 2603.06728
Orion: Characterizing and Programming Apple's Neural Engine for LLM Training and Inference
By Ramchand Kumaresan
Published 2026-03-06
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
Over two billion Apple devices ship with a Neural Processing Unit (NPU) - the Apple Neural Engine (ANE) - yet this accelerator remains largely unused for large language model workloads. CoreML, Apple's public ML framework, imposes opaque abstractions that prevent direct ANE programming and do not support on-device training. We present Orion, to our knowledge the first open end-to-end system that combines direct ANE…