Meet Apple’s Minions
This exciting approach is described in a newly published white paper, ‘Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices.” In it, the researchers explain how constrained GPU resources delay access to inference tasks and increase costs — even while powerful Macs running Apple silicon chips may sit idle.
They’ve developed an approach to use that power. It’s based on three components: a Minion Server, an Executor, and a Watchdog. The server splits tasks up and puts them into a queue; the executor then acts as kind of a nightclub concierge, directing tasks across all the available Macs. The watchdog monitors active tasks and checks for failures — a Mac may be shut down or go to sleep during a task, for example. If a node goes offline, the watchdog returns that sub-task to the queue, making sure the work gets done.
Minions currently supports two kinds of GPU inference on Macs, embedding generation and LLM completions using open-source transformer models from Hugging Face and Ollama, respectively. Ollama is installed and executed on participating devices, enabling local inference without remote API calls. The Apple team plans to extend Minions so it also supports distributed model training in the future, which should mean enterprise AI developers will be able to call on the power of all your available Macs to build new models as required.