Open-source AssemblyAI alternatives

whisper.cpp is a C/C++ implementation of OpenAI's Whisper automatic speech recognition model, built for efficient inference without external dependencies. It is released under the MIT license and part of the ggml project, and the whole high-level model implementation lives in two files, which makes it easy to embed in different platforms and applications.

It is tuned for many kinds of hardware: Apple Silicon through ARM NEON, Accelerate, Metal and Core ML, AVX on x86, VSX on POWER, plus Vulkan, NVIDIA GPUs, AMD ROCm, OpenVINO and several NPUs. Mixed F16 and F32 precision, integer quantization and zero memory allocations at runtime keep resource use low, and a C-style API and voice activity detection are included.

Supported platforms include macOS, iOS, Android, Java, Linux and FreeBSD, WebAssembly, Windows, Raspberry Pi and Docker. The README shows the model running fully offline on an iPhone and suggests building offline voice assistants. It suits developers adding private speech-to-text to apps, and hobbyists transcribing audio on their own hardware.

LocalAI is an open-source AI engine for running models of many kinds, including language, vision, voice, image and video models, on your own hardware. A GPU is not required. It is written in Go and released under the MIT license, was started by Ettore Di Giacinto and is looked after by the LocalAI team.

Its design is a small core with separate backends that are pulled on demand, so nothing unused gets installed. Backends wrap engines such as llama.cpp, vLLM, whisper.cpp, stable-diffusion and MLX, and you can write your own in any language against an open interface. It can run on CPU alone, through Vulkan, or on NVIDIA, AMD, Intel and Apple Silicon hardware.

APIs are drop-in compatible with OpenAI, Anthropic and ElevenLabs across every backend. Multi-user features include API key authentication, user quotas and role-based access, and built-in agents support tool use, RAG, MCP and skills. The project stresses that data stays on your infrastructure. It suits developers and teams who want a private, self-hosted replacement for cloud AI APIs.

Whisper is a speech recognition model from OpenAI, general in purpose, released as open source under the MIT license. It was trained on a large and diverse audio dataset and handles several tasks in one model: transcription in multiple languages, translation of speech, and identifying the spoken language. The repository contains the Python code needed to run it.

It uses a Transformer sequence-to-sequence model trained on several speech processing tasks, including voice activity detection, which are represented together as a sequence of tokens predicted by the decoder. As a result, one model can stand in for what used to take several separate components in a speech pipeline. Special tokens act as task specifiers or classification targets.

The authors trained and tested with Python 3.9 and PyTorch 1.10, and expect the code to work with Python 3.8 to 3.11 and recent PyTorch versions. It can be installed with pip as openai-whisper or directly from the repository, and it relies on a few packages such as OpenAI's tiktoken tokenizer. Links to a blog post, paper, model card and Colab example are provided. It suits developers and researchers who need speech-to-text they can run themselves.