exo

exo connects all of your devices into an AI cluster, so you can run models that are larger than would fit on any single machine. Adding more devices can also make models run faster. It is written in Python, released under the Apache-2.0 license and maintained by exo labs.

Its feature list includes automatic discovery of other devices running exo with no manual configuration, support for RDMA over Thunderbolt 5 to cut latency between devices, and topology-aware automatic parallelism that splits a model using a real-time view of device resources and network links. Tensor parallelism shards models across devices, and MLX serves as the inference backend with MLX distributed for communication.

exo speaks four API dialects, OpenAI Chat Completions, Claude Messages, OpenAI Responses and Ollama, so existing clients and tools can be reused. Custom models can be loaded from the Hugging Face hub, and a built-in dashboard manages the cluster and chats with models. It suits enthusiasts and teams who want to run large models locally using several Apple devices or other machines.