8 open-source alternatives ranked by real activity

Open-source Cloudflare Workers AI alternatives

A curated, ranked list of the 8 best open-source alternatives to Cloudflare Workers AI.

The best open-source alternative to Cloudflare Workers AI is Ollama. If that doesn't suit you, other good options are vLLM, LocalAI, Ray and Nvidia.

Commercial alternatives to Cloudflare Workers AI include Fireworks AI, Baseten, Together AI and DeepInfra. See all 14.

Cloudflare Workers AI alternatives are mainly AI infrastructure tools, but some are also hosting & PaaS tools and AI tools. 8 of them shipped code in the last 30 days, 8 can be self-hosted, and 7 use a permissive licence.

Last updated November 13, 2026 · ranked by GitHub stars, growth and recent commits

Ollama

A tool for downloading and running open large language models locally through a command line and REST API, with Docker support and many integrations.

GitHub stars
187k
Last commit
yesterday
Latest release
v0.40.16
Licence
MIT
Self-hosted
Yes
ollama.comOllama homepage screenshot

Ollama makes it straightforward to run open large language models on your own machine. You install it on macOS, Windows or Linux, or use the official Docker image, then pull a model and chat with it from the command line. It is written in Go and released under the MIT license, and it builds on the llama.cpp project for inference.

Besides the CLI, Ollama provides a REST API for managing and running models, with official Python and JavaScript client libraries. Models can be imported and customized through a Modelfile, and an online library lists what is available. It can be connected to coding agents and assistants such as Claude Code, OpenCode, Codex and OpenClaw, and many community chat interfaces, including Open WebUI and LibreChat, work with it.

Because models run on your own hardware, Ollama can serve as a self-hosted alternative to cloud assistants such as ChatGPT and Perplexity when privacy, offline use or cost control matter. It is aimed at developers and hobbyists experimenting with open models, and at teams that want to wire local models into their own applications.

Key features

  • Run open language models locally from the CLI
  • REST API for managing and running models
  • Official Python and JavaScript libraries
  • Modelfile for customizing and importing models
  • Official Docker image
  • Integrations with coding agents and chat interfaces

Pricing: Running models locally is free, and the Free plan includes starter cloud credits. Pro costs $20, Max $100 and Team $500 per month with included usage credits; Enterprise is custom.

Read more about OllamaWebsiteGitHub#1 Cloudflare Workers AI alternative

vLLM

A Python library and server for fast, memory-efficient LLM inference and serving, with PagedAttention, continuous batching and broad quantization support.

GitHub stars
97k
Last commit
yesterday
Latest release
v0.33.0rc3
Licence
Apache-2.0
Self-hosted
Yes
vllm.aivLLM homepage screenshot

vLLM is an open-source library for running and serving large language models efficiently. It originated at UC Berkeley's Sky Computing Lab and is now developed by a large community of contributors from academia and industry. It is written in Python and released under the Apache-2.0 license.

Speed comes from techniques such as PagedAttention for attention key-value memory, batching incoming requests continuously, chunked prefill, prefix caching, CUDA and HIP graphs, and optimized attention and mixture-of-experts kernels. It supports many quantization formats, including FP8, INT8, INT4, GPTQ and AWQ, as well as speculative decoding and disaggregated prefill and decode.

On the usability side, vLLM integrates with Hugging Face models and supports parallel sampling, beam search, streaming outputs, structured outputs and tool calling, plus several parallelism modes (tensor, pipeline, data, expert and context) for distributed inference. It targets NVIDIA, AMD and TPU hardware and suits teams that serve open models in production and need high throughput.

Key features

  • PagedAttention and continuous batching
  • Quantization including FP8, INT8, GPTQ and AWQ
  • Speculative decoding support
  • Tensor, pipeline and expert parallelism
  • OpenAI-compatible API server
  • Hugging Face model integration
  • Streaming and structured outputs

Pricing: Free and open source under the Apache-2.0 license.

Read more about vLLMWebsiteGitHub#2 Cloudflare Workers AI alternative

LocalAI

An open-source AI engine that runs LLMs, vision, voice, image and video models on your own hardware behind OpenAI-compatible APIs, with no GPU required.

GitHub stars
50k
Last commit
yesterday
Latest release
v4.12.1
Licence
MIT
Self-hosted
Yes
localai.ioLocalAI homepage screenshot

LocalAI is an open-source AI engine for running models of many kinds, including language, vision, voice, image and video models, on your own hardware. A GPU is not required. It is written in Go and released under the MIT license, was started by Ettore Di Giacinto and is looked after by the LocalAI team.

Its design is a small core with separate backends that are pulled on demand, so nothing unused gets installed. Backends wrap engines such as llama.cpp, vLLM, whisper.cpp, stable-diffusion and MLX, and you can write your own in any language against an open interface. It can run on CPU alone, through Vulkan, or on NVIDIA, AMD, Intel and Apple Silicon hardware.

APIs are drop-in compatible with OpenAI, Anthropic and ElevenLabs across every backend. Multi-user features include API key authentication, user quotas and role-based access, and built-in agents support tool use, RAG, MCP and skills. The project stresses that data stays on your infrastructure. It suits developers and teams who want a private, self-hosted replacement for cloud AI APIs.

Key features

  • Runs LLM, vision, voice, image and video models
  • Composable backends pulled on demand
  • OpenAI, Anthropic and ElevenLabs API compatibility
  • Runs on CPU, NVIDIA, AMD, Intel and Apple Silicon
  • API key authentication, quotas and role-based access
  • Built-in agents with tools, RAG and MCP

Pricing: Free and open source under the MIT license.

Read more about LocalAIWebsiteGitHub#3 Cloudflare Workers AI alternative

Ray

Ray is an open source framework for managing, executing, and optimizing compute needs, designed to support any AI or ML workload.

GitHub stars
44k
Last commit
yesterday
Latest release
ray-2.60.0
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
ray.ioRay homepage screenshot

Ray is an AI compute engine that consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. It is Python-native and built by developers for developers, designed to orchestrate infrastructure for distributed workloads on any accelerator at any scale.

The platform aims to address the AI Complexity Wall by supporting any AI or ML workload, any data types and model architectures, and heterogeneous GPUs and CPUs with fine-grained scaling. It can scale from a laptop to thousands of GPUs.

Key features

  • Supports any AI or ML workload
  • Handles multi-modal data processing including images, videos, audio
  • Runs distributed training for Gen AI foundation models, time series models, and traditional AI/ML models
  • Offers model serving with independent scaling and fractional resources via Ray Serve
  • Supports batch inference with heterogeneous compute using CPUs and GPUs
Read more about RayWebsiteGitHub#4 Cloudflare Workers AI alternative

Nvidia

NVIDIA Triton Inference Server is an optimized cloud and edge inferencing solution that supports multiple backends and provides APIs for model serving.

GitHub stars
11k
Last commit
2 days ago
Latest release
v2.74.0
Licence
BSD-3-Clause
docs.nvidia.comNvidia homepage screenshot

NVIDIA Triton Inference Server is a software platform designed for optimized inference of AI models in cloud and edge environments. It supports a variety of model backends including TensorRT-LLM, vLLM, PyTorch, ONNX, and OpenVINO, among others.

The server provides features for concurrent model execution, scheduling, batching, model pipelines, and state management. It includes client libraries for Python and C/C++, and supports HTTP/REST and gRPC protocols, as well as an OpenAI-compatible API.

Key features

  • Supports multiple model backends including TensorRT-LLM, vLLM, PyTorch, ONNX, and OpenVINO
  • Provides concurrent model execution and scheduling with batching
  • Supports model pipelines via ensembles and business logic scripting
  • Includes state management and request cancellation
  • Offers metrics, tracing, and caching capabilities
  • Provides client libraries for Python and C/C++
Read more about NvidiaWebsiteGitHub#5 Cloudflare Workers AI alternative

Hugging Face Chat UI

SvelteKit chat interface for large language models that connects to any OpenAI-compatible API and powers HuggingChat.

GitHub stars
11k
Last commit
yesterday
Latest release
v0.10.0
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
huggingface.coHugging Face Chat UI homepage screenshot

Chat UI is a chat interface for LLMs built as a SvelteKit application. It is the open source codebase behind HuggingChat on Hugging Face and lets you run a similar conversational front end against models you choose, whether hosted or running locally.

The current version speaks only to OpenAI-compatible APIs, configured through a base URL and a models endpoint. The README lists the Hugging Face Inference Providers router, a llama.cpp server, Ollama through its OpenAI-compatible bridge, OpenRouter and Poe as example backends. Older provider-specific integrations were removed from the main branch but remain available on a legacy branch.

Setup involves creating an .env.local file with the endpoint and key, choosing a database option, and launching the app, with an optional Docker image. The project is written in TypeScript with Tailwind CSS and is licensed under Apache-2.0, making it a base for self-hosted assistants.

Key features

  • Chat front end for any OpenAI-compatible API
  • Works with llama.cpp, Ollama and OpenRouter
  • Hugging Face Inference Providers support
  • Configured through a simple .env.local file
  • Optional Docker image
  • SvelteKit and Tailwind CSS codebase

Pricing: Free and open source under the Apache-2.0 license.

Read more about Hugging Face Chat UIWebsiteGitHub#6 Cloudflare Workers AI alternative

BentoML

BentoML is an open-source framework for serving AI and ML models and custom inference pipelines, with a hosted platform for scaling them.

GitHub stars
8.9k
Last commit
11 days ago
Latest release
v1.4.40
Licence
Apache-2.0
Self-hosted
Yes
Hosted version
Available
bentoml.comBentoML homepage screenshot

BentoML Open-Source is presented as a flexible way to serve AI and ML models and custom inference pipelines in production. A model is packaged as a Python service using decorators, where the code declares the image, the GPU type and an API function, as in the vLLM example on the homepage.

The same company sells the Bento Inference Platform for managing, monitoring and optimizing inference, and Bento Cloud for GPU compute. Deployments can run in your own cloud, on-premises or on Kubernetes. The homepage names vLLM, TRT-LLM, JAX, SGLang, PyTorch and Transformers among the frameworks it can serve.

Key features

  • Unified packaging for models of any architecture
  • Open model catalog with quick deploys
  • Platform for managing and monitoring inference
  • Cross-region and elastic scaling, including scale to zero
  • Bring your own cloud, on-prem or Kubernetes
  • GPU hardware through Bento Cloud
  • Pre-optimized open models available on release
Read more about BentoMLWebsiteGitHub#7 Cloudflare Workers AI alternative

beta9

Open-source runtime for serverless AI workloads that deploys and scales GPU inference, sandboxes, and background jobs from a Python interface.

GitHub stars
1.9k
Last commit
yesterday
Latest release
worker-0.1.834
Licence
AGPL-3.0
Self-hosted
Yes
Hosted version
Available
beam.cloudbeta9 homepage screenshot

Beam, whose open-source engine lives in the beta9 repository, is a fast runtime for serverless AI workloads. It gives developers a Pythonic interface to deploy and scale AI applications without managing infrastructure. The code is written in Go and released under the AGPL-3.0 license, and topics include GPU, LLM inference, autoscaling, and functions as a service.

Features include sub-second container cold starts using a custom runtime, scheduler, and embedded caching, parallel fan-out to hundreds of containers, hot reloading, webhooks and scheduled jobs, scale-to-zero by default, and mounted distributed storage volumes. GPU support runs on the Beam cloud, with cards such as the RTX 4090 and H100 mentioned, or on your own GPUs.

The project can be self-hosted or used through the hosted platform, which requires creating an account. Beam suits machine learning engineers and startups that want to serve models, run batch jobs, or sandbox untrusted code without operating a Kubernetes GPU cluster themselves.

Key features

  • Serverless GPU inference
  • Sub-second container cold starts
  • Scale-to-zero workloads
  • Parallel fan-out to many containers
  • Webhooks and scheduled jobs
  • Volume storage and bring-your-own GPUs

Pricing: The Developer plan is $0 per month plus usage and Team is $89 per month plus usage; Growth is custom. Usage is billed by the millisecond, with H100 machines from $1.83 per hour.

Read more about beta9WebsiteGitHub#8 Cloudflare Workers AI alternative

Commercial Cloudflare Workers AI alternatives

Hosted, closed-source products that compete with Cloudflare Workers AI. Showing 12 of 14.

See every Cloudflare Workers AI alternative on its tool page

Cloudflare Workers AI alternatives: questions

What is the best open-source alternative to Cloudflare Workers AI?
Ollama is the top-ranked open-source alternative to Cloudflare Workers AI on Enlisted: A tool for downloading and running open large language models locally through a command line and REST API, with Docker support and many integrations. Other strong options are vLLM, LocalAI, Ray and Nvidia.
What are the best commercial alternatives to Cloudflare Workers AI?
Fireworks AI, Baseten, Together AI, DeepInfra and Amazon Bedrock are among the hosted, closed-source alternatives to Cloudflare Workers AI listed on Enlisted, which lists 14 in total. Compare their pricing and features on each tool page.
Are these Cloudflare Workers AI alternatives free?
All 8 are open source, so the code is free to use under its licence, and all of them can be self-hosted on your own server or computer. 4 also offer a paid or managed cloud version if you'd rather not host it yourself.
How is this list of Cloudflare Workers AI alternatives ranked?
By a score built from GitHub stars, star growth over the last 30 days and how recently the code changed. 8 of these projects shipped code in the last 30 days. Data is refreshed daily, and nobody can pay to move up.

People also look for alternatives to…

View all