About Unsloth
Unsloth is an open-source framework with a desktop app for running and training AI models on your own hardware. The same tool covers inference of local language, image, audio and embedding models as well as fine-tuning them, and it runs on Windows, Linux (including WSL) and macOS, with support for NVIDIA, AMD and Intel GPUs, CPUs and a Vulkan backend.
On the running side it loads GGUF and MLX models, serves them through an OpenAI-compatible API, and lets coding agents such as Claude Code and Codex use local models with tool calling and code execution. It also offers private web search, deep research and retrieval, plus access from other devices over a LAN or a secure remote link. For training it supports LoRA, QLoRA, full fine-tuning, pretraining and reinforcement learning methods such as GRPO and DPO, with export to formats including GGUF.
The project claims fine-tuning that is roughly twice as fast with about 70 percent less VRAM, which was its original draw. Unsloth is Apache-2.0 licensed, can be installed from native packages or a Docker image, and has notebooks and documentation for getting started.
Key features
- Run GGUF and MLX models locally
- Fine-tune LLMs with LoRA and QLoRA
- Reinforcement learning with GRPO and DPO
- OpenAI-compatible API for local models
- Native desktop app and Docker image
- Export trained models to GGUF
Good fit for
- Fine-tuning open models on a single GPU
- Running private local AI with tool calling
- Serving local models to coding agents
Unsloth: questions and answers
- What is Unsloth used for?
- Unsloth is an open-source app and framework for running and fine-tuning LLMs and diffusion models locally, with GGUF and MLX support and an OpenAI-compatible API. It is a good fit for fine-tuning open models on a single GPU, running private local AI with tool calling and serving local models to coding agents.
- Is Unsloth open source?
- Yes. Unsloth is open source under the Apache-2.0 licence. Its source code is on GitHub at unslothai/unsloth and is written mainly in Python.
- Is Unsloth free?
- Yes. Unsloth is open source, so the software itself is free to use.
- Can I self-host Unsloth?
- Yes. Unsloth can be self-hosted on your own server or infrastructure; there is no official hosted version.
- What is Unsloth an alternative to?
- Unsloth is an open-source alternative to Together AI. Other open-source alternatives to Together AI include LocalAI, vLLM and SGLang.
- Is Unsloth actively maintained?
- Yes. The most recent commit to Unsloth was on 14 November 2026, and the latest release is v0.1.917-beta, published on 11 November 2026. The project has 80k stars on GitHub, with 1.5k added in the last 30 days.
Open-source alternatives to Unsloth
See all
LocalAI
AI Infrastructure
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video -
MITvs ChatGPT★ 50k
vLLM
AI Infrastructure
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.0vs Amazon Bedrock★ 97k
SGLang
AI Infrastructure
SGLang is a high-performance serving framework for large language models and multimodal mo
Apache-2.0vs Amazon Bedrock★ 39k
MLflow
AI Infrastructure
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables te
Apache-2.0vs Weights & Biases★ 29k
Ray
Hosting & PaaS
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Li
Apache-2.0vs OpenRouter★ 44k
Kubeflow
AI Tools
The Cloud Native AI Platform
Apache-2.0vs Together AI★ 16k
SaaS alternatives to Unsloth
See all
Together AI
AI Infrastructure
Cloud platform for running, fine-tuning and training open and custom AI models
SaaS
TinyTune
AI Infrastructure
No-code platform to fine-tune open-source LLMs on your own data in minutes
SaaS
Fireworks AI
AI Infrastructure
Inference platform for running and fine-tuning generative AI models at low latency
SaaS
Mistral AI
AI Infrastructure
AI company offering open-weight and commercial language models via API and Le Chat
SaaS
api-hub.ai
AI Infrastructure
Single API for accessing many AI models behind one endpoint
SaaS
Baseten
AI Infrastructure
Inference platform for deploying and scaling AI models in production
SaaS

