222 tools · 88 open-source · 134 SaaS
The best AI infrastructure tools. Ranked by real activity.
The top-ranked open-source AI infrastructure tool on Enlisted is WeKnora (37k GitHub stars), followed by Ollama and vLLM. Among SaaS products, other tools on Enlisted are most often compared with Polyrouter, IBM watsonx and OpenRouter.
Enlisted lists 222 AI infrastructure tools: 88 open source and 134 SaaS. 79 of the open-source ones can be self-hosted on your own server or computer. The most common open-source licences here are Apache-2.0 (44 tools), MIT (20) and AGPL-3.0 (7). The open-source ones are most often listed as alternatives to Amazon SageMaker, Vellum and OpenRouter.
Last updated November 13, 2026 · ranked by GitHub stars, growth and recent commits
The top 48 of 222 AI infrastructure tools

WeKnora
AI Infrastructure
Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous
OSS★ 37k
Ollama
AI Infrastructure
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other model
MITvs ChatGPT★ 187k
vLLM
AI Infrastructure
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.0vs Amazon Bedrock★ 97k
llama.cpp
AI Infrastructure
LLM inference in C/C++
MITvs OpenAI API Platform★ 134k
Dify
AI Infrastructure
Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collabo
OSSvs Gumloop★ 161k
New API
AI Infrastructure
A unified AI model hub for aggregation & distribution. It supports cross-converting variou
AGPL-3.0vs OpenRouter★ 52k
SGLang
AI Infrastructure
SGLang is a high-performance serving framework for large language models and multimodal mo
Apache-2.0vs Amazon Bedrock★ 39k
LiteLLM
AI Infrastructure
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (o
OSSvs Amazon Bedrock★ 63k
Langfuse
AI Infrastructure
Trace, evaluate, and improve AI agents with one open platform. Use production data to unde
OSSvs Weights & Biases★ 37k
RAGFlow
AI Infrastructure
RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cu
Apache-2.0vs Vectara★ 94k
Langflow
AI Infrastructure
Langflow is a powerful tool for building and deploying AI-powered agents and workflows.
MITvs Gumloop★ 157k
Promptfoo
AI Infrastructure
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI.
MITvs Braintrust★ 27k
Unsloth
AI Infrastructure
Unsloth is an open-source framework for running and training LLMs.
Apache-2.0vs Together AI★ 79k
TraceRoot
AI Infrastructure
TraceRoot - open source self improving layer for ai agents YC S25
OSSvs Weights & Biases★ 1.4k
Opik
AI Infrastructure
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows wit
Apache-2.0vs Weights & Biases★ 23k
LocalAI
AI Infrastructure
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video -
MITvs ChatGPT★ 50k
Rome
AI Infrastructure
A compounding agent OS for recursive agents. Also an open source alternative to Grok Bot a
MITvs Grok★ 898
KiwiFS
AI Infrastructure
Markdown filesystem for agents and teams.
OSS★ 877
Arize Phoenix
AI Infrastructure
AI Observability & Evaluation
OSSvs Weights & Biases★ 12k
Qdrant
Databases
Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the
Apache-2.0vs Pinecone★ 36k
Microsandbox
AI Infrastructure
Easy, fast microVMs for untrusted workloads
Apache-2.0★ 9k
PrivateGPT
AI Infrastructure
Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, t
Apache-2.0vs ChatGPT★ 58k
RD-Agent
AI Infrastructure
Research and development (R&D) is crucial for the enhancement of industrial productivity,
MIT★ 15k
MLflow
AI Infrastructure
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables te
Apache-2.0vs Weights & Biases★ 29k
llamafile
AI Infrastructure
Distribute and run LLMs with a single file.
OSSvs ChatGPT★ 27k
Sim
AI Infrastructure
Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows.
Apache-2.0vs Gumloop★ 30k
Label Studio
AI Infrastructure
Label Studio is a multi-type data labeling and annotation tool with standardized output fo
Apache-2.0vs Roboflow★ 29k
Yuxi
AI Infrastructure
可私有部署的多租户知识智能体平台,统一知识库、知识图谱、多智能体执行、MCP/Skills、沙盒与权限管理。
MIT★ 7.7k
Chroma
Databases
Search infrastructure for AI
Apache-2.0vs Pinecone★ 30k
OpenConnector
AI Infrastructure
Connect Gmail, GitHub, Notion, and 1500+ apps through OOMOL, then use those connections in
Apache-2.0★ 6.2k
MLC LLM
AI Infrastructure
Universal LLM Deployment Engine with ML Compilation
Apache-2.0★ 23k
Trigger.dev
Workflow Automation
Trigger.dev – build and deploy durable AI agents and workflows
Apache-2.0vs IFTTT★ 17k
CVAT
AI Infrastructure
Computer Vision Annotation Tool (CVAT) is a leading platform for building high-quality vis
MITvs Roboflow★ 17k
LanceDB
Databases
Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage L
Apache-2.0vs Pinecone★ 12k
MCP Toolbox
AI Infrastructure
MCP Toolbox for Databases is an open source MCP server for databases.
Apache-2.0vs Connect★ 17k
Obot
AI Infrastructure
Complete AI Governance Platform from Obot AI
MIT★ 1.2k
Weaviate
Databases
Weaviate is an open-source vector database that stores both objects and vectors, allowing
OSSvs Pinecone★ 17k
Flyte
AI Tools
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on y
Apache-2.0vs Amazon SageMaker★ 7.8kGPT-Load
AI Infrastructure
Self-hosted AI gateway for scheduling API keys and subscription accounts across 35 channel
MIT★ 7.2k
Laminar
AI Infrastructure
Laminar - open-source observability platform purpose-built for AI agents. YC S24.
Apache-2.0vs Weights & Biases★ 3.5k
Nvidia
AI Infrastructure
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
BSD-3-Clausevs Fireworks AI★ 11k
AnythingMCP
AI Infrastructure
Open-source, self-hosted MCP gateway: turn any REST/OpenAPI, SOAP, GraphQL, OData or SQL A
AGPL-3.0★ 776
HelixDB
Databases
HelixDB is an OLTP graph database with native vector and full-text search built in Rust on
Apache-2.0vs Amazon Neptune★ 6.3k
Docs MCP Server
Developer Tools
Grounded Docs MCP Server: Open-Source Alternative to Context7, Nia, and Ref.Tools
MIT★ 1.9k
BentoML
AI Infrastructure
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM
Apache-2.0vs Amazon SageMaker★ 8.9k
LangWatch
AI Infrastructure
The platform for LLM evaluations and AI agent testing
Apache-2.0vs LangSmith★ 5k
Astron Agent
AI Infrastructure
Enterprise-grade, commercial-friendly agentic workflow platform for building next-generati
Apache-2.0★ 8.9k
Vespa
Search
The AI search platform
Apache-2.0vs Elastic★ 7.2k
AI infrastructure tools: questions
- What is the best open-source AI infrastructure tool?
- WeKnora is the top-ranked open-source AI infrastructure tool on Enlisted, with 37k GitHub stars. It is followed by Ollama, vLLM and llama.cpp.
- What are the best free AI infrastructure tools?
- The top-ranked open-source AI infrastructure tools with a standard licence are Ollama (MIT), vLLM (Apache-2.0) and llama.cpp (MIT), so their code is free to use. 48 of the 134 SaaS products here have a free plan, including IBM watsonx, OpenRouter and LangSmith.
- Which AI infrastructure tools can be self-hosted?
- 79 of the 88 open-source AI infrastructure tools on Enlisted can be self-hosted on your own server or computer; the top-ranked are WeKnora, Ollama and vLLM.
- How are these AI infrastructure tools ranked?
- Open-source tools are ordered by a score built from GitHub stars, star growth over the last 30 days and how recently the code changed; 69 of the 88 had new commits or a release in the last 30 days. SaaS products have no public repository, so they get a fixed baseline score. GitHub data is refreshed daily, and nobody can pay to move up.