#1
An LLM application platform for building AI workflows, RAG pipelines and agents on one canvas, deployable in the cloud, a private VPC or self-hosted.
- GitHub stars
- 161k
- Last commit
- today
- Latest release
- 1.19.0
- Self-hosted
- Yes
- Hosted version
- Available
dify.ai
Dify is a platform for building applications on top of large language models. Instead of wiring together prompts, retrieval code and model APIs by hand, teams assemble AI workflows, knowledge pipelines and agents in a shared visual workspace, then move them from prototype to production without changing stacks.
The product covers a workflow studio for agentic flows, a knowledge pipeline that prepares searchable knowledge bases for retrieval-augmented generation, model management across many providers, and a marketplace of tools, models and integrations. Observability integrations such as Opik, Langfuse and Arize Phoenix help trace and debug runs once an app is live.
Dify comes in a few editions: a community edition you deploy yourself with Docker, a hosted cloud service, and an enterprise edition for private deployments. The repository carries a custom license rather than a standard one, so check its terms before offering Dify as a service to others.
Key features
- Visual workflow studio for agentic apps
- Knowledge pipelines for retrieval-augmented generation
- Model management across many LLM providers
- Marketplace of tools, models and integrations
- Observability integrations such as Langfuse
- Deploy on cloud, VPC or Docker
Pricing: Free Sandbox on Dify Cloud and a free self-hosted Community edition. Cloud Professional costs $590 and Team $1590 per workspace per year; Enterprise is custom.
#2
An open-source LLM engineering platform for tracing, evaluating and improving AI applications, with prompt management, datasets and a playground, self-hosted or cloud.
- GitHub stars
- 37k
- Last commit
- today
- Latest release
- v4.79.0
- Self-hosted
- Yes
- Hosted version
- Available
langfuse.com
Langfuse is an open-source LLM engineering platform that helps teams develop, monitor, evaluate and debug AI applications together. You instrument your app, and Langfuse records traces of LLM calls and surrounding logic such as retrieval, embeddings and agent actions, so complex runs and user sessions can be inspected and debugged.
Prompt management lets you centrally version and iterate on prompts, with caching on server and client so that changes do not add latency to your app. Evaluation features cover LLM-as-a-judge, code evaluators, user feedback, manual labeling and custom pipelines through the API and SDKs. Datasets provide test sets and benchmarks, there is an LLM playground for trying prompts, and integrations exist for OpenAI, LangChain and LlamaIndex. It is built on the ClickHouse database, and the Langfuse team has been part of ClickHouse since January 2026.
You can use Langfuse Cloud or self-host it, which the maintainers say takes minutes. The repository license is listed as 'Other' because it combines open-source and enterprise components, so check the terms. It suits teams shipping production LLM features who need visibility into quality, latency and cost.
Key features
- Tracing of LLM calls, retrieval and agent actions
- Prompt versioning with caching
- LLM-as-a-judge and custom evaluations
- Datasets for tests and benchmarks
- Interactive LLM playground
- Integrations with OpenAI, LangChain and LlamaIndex
Pricing: Core has a $29 monthly base that includes 100k units, then $8 per 100k units with volume discounts. Hobby and Pro tiers exist but their prices are not shown in the text; Enterprise is by sales.
#3
TraceRoot is an open-source observability layer for AI agents that turns production traces into detector findings, datasets and evaluations.
- GitHub stars
- 1.4k
- Last commit
- yesterday
- Latest release
- v1.3.8
- Self-hosted
- Yes
- Hosted version
- Available
traceroot.ai
TraceRoot describes itself as an open-source self-improving layer for AI agents. It captures traces from production agent systems and turns them into actionable feedback and evaluations, closing a loop with your coding agent so that problems found in production lead to improvements. The project is a Y Combinator S25 company.
Its capabilities include tracing of LLM calls, tool use and agent steps with OpenTelemetry-compatible Python and TypeScript SDKs, covering inputs, outputs, latency, tokens and cost. Detectors screen incoming traces for behaviors such as hallucinations, tool failures and logic errors, with configurable sampling and judge models. Datasets and evaluations let you version test cases, score runs and compare candidate versions, while dashboards and threshold alerts track quality, latency and cost.
A CLI exports traces and findings into your coding workflow, and an in-app AI assistant can explore traces with access to your source code and GitHub context, using a hosted model or your own key. TraceRoot offers a hosted cloud and a self-hosting guide, is written in TypeScript, and its license is listed as Other.
Key features
- OpenTelemetry-compatible agent tracing
- Detectors that flag failures in traces
- Versioned datasets and evaluations
- Dashboards and threshold alerts
- CLI for exporting traces and findings
- In-app AI assistant with code context
#4
Opik is an open-source platform from Comet for tracing, evaluating and monitoring LLM applications, RAG systems and AI agents.
- GitHub stars
- 23k
- Last commit
- yesterday
- Latest release
- 2.2.143
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
comet.com
Opik is an open-source LLM observability and evaluation platform built by Comet. It covers the application lifecycle starting with early traces during development and ending with production monitoring, for teams that build LLM apps and AI agents. The code is Python, licensed under Apache-2.0, and the README says the full platform is free to self-host. The project documentation lives on the Comet site.
Capabilities include deep tracing of LLM calls and agent activity including complete trace trees for agents with several steps and tool calls, and evaluation with datasets, experiments and LLM-as-a-judge metrics for tasks like hallucination detection, moderation and RAG assessment. An Agent Optimizer SDK improves prompts and agents, dashboards and online evaluation rules support production monitoring, guardrails help with safe AI practices, and a PyTest integration tests LLM pipelines on each commit. Integrations include frameworks such as LangChain, LlamaIndex and OpenAI clients.
Key features
- Tracing for LLM calls and agent steps
- Datasets and experiments for evaluation
- Evaluation metrics using LLM-as-a-judge
- Prompt and agent optimization SDK
- Production dashboards and online evaluation
- PyTest integration for CI checks
Pricing: Free to self-host, with a free hosted tier of 25k spans. Pro includes 100k spans with extra spans at $5 per 100k; Enterprise is custom.
#5
Arize Phoenix is an open-source AI observability platform for tracing, evaluating and experimenting with LLM applications.
- GitHub stars
- 12k
- Last commit
- yesterday
- Latest release
- @arizeai/phoenix-cli@1.22.0
- Self-hosted
- Yes
- Hosted version
- Available
Phoenix is built for experimentation, evaluation and troubleshooting of LLM apps. It traces an application's runtime using OpenTelemetry-based instrumentation, runs LLM-based evaluations of responses and retrieval, and keeps versioned datasets and experiments for comparing changes to prompts, models and retrieval.
Beyond tracing, it has a playground for comparing models and replaying traced calls, prompt management with version control and tagging, PXI, an agent built in for debugging traces, and a remote MCP server so tools such as Claude Code and Cursor can query traces, datasets and experiments. It is vendor and language agnostic, with support for frameworks like LangGraph, CrewAI, LlamaIndex and DSPy and providers such as OpenAI, Anthropic and AWS Bedrock.
Phoenix is open source and self-hosted. Install with pip install arize-phoenix and phoenix serve, run it with uvx, or deploy container images on Docker or Kubernetes through the Helm chart. For a managed platform, Arize sells Arize AX, and the same OpenTelemetry and OpenInference instrumentation works with both.
Key features
- OpenTelemetry-based tracing of LLM applications
- LLM-powered evaluation of responses and retrieval
- Versioned datasets for experiments and fine-tuning
- Prompt playground with model comparison and trace replay
- Prompt management with version control and tagging
- Remote MCP server for coding agents
- Setup command detects framework and wires up trace export
#6
MLflow is an open-source platform for debugging, evaluating and monitoring AI agents, LLM applications and machine learning models.
- GitHub stars
- 29k
- Last commit
- yesterday
- Latest release
- v3.18.0
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
mlflow.org
MLflow is an open-source AI engineering platform for agents, LLM applications and ML models. It helps teams debug, evaluate, monitor and optimize AI applications in production while keeping control of costs and access to models and data. It is written in Python, released under the Apache-2.0 license, and has been developed since 2018, with topics covering MLOps, model management and LLMOps.
For LLM and agent work it offers production observability with traces, plus evaluation, prompt management and prompt optimization, and an AI Gateway that governs costs and model access. It works with Python, TypeScript and JavaScript, Java and other languages, and integrates with OpenTelemetry and MCP. Getting started means starting an MLflow server, enabling logging in your code and running it, then exploring traces and metrics in the web UI on port 5000. A setup wizard can connect to an MLflow server or a Databricks workspace and add tracing with a coding agent.
Key features
- Tracing and observability for LLM apps and agents
- Evaluation tools for models and agents
- Prompt management and prompt optimization
- AI Gateway for cost and model access control
- OpenTelemetry and MCP integration
- Web UI for exploring traces and metrics
Pricing: Free and open source under the Apache-2.0 license; the README also mentions connecting to a Databricks workspace.
#7
Open-source observability platform for AI agents with OpenTelemetry-based tracing, evaluations, dashboards, and plain-English alerts on agent behavior.
- GitHub stars
- 3.5k
- Last commit
- yesterday
- Latest release
- v0.3.4
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
laminar.sh
Laminar is an open-source observability platform built for AI agents. It records what agents do during each run so that teams can debug failures, track behavior, and measure quality. The project is released under the Apache-2.0 license and the backend is written largely in Rust, with a TypeScript front end.
Tracing is based on OpenTelemetry, and a single line of SDK setup can automatically trace libraries such as the Vercel AI SDK, Browser Use, Stagehand, LangChain, OpenAI, Anthropic, and Gemini. Signals let you describe a behavior, such as an agent getting stuck in a loop, in plain English and be notified in Slack when it occurs. Evals run through an SDK and CLI locally or in CI, with a UI for comparing results, and SQL queries and dashboards cover traces, metrics, and events.
Coding agents can reach the data through MCP and CLI access to investigate issues themselves. Laminar can be self-hosted locally with Docker Compose, or used as the managed platform at laminar.sh. It suits teams building production agents who need tracing, evaluation datasets, and annotation in one place.
Key features
- OpenTelemetry-native tracing SDK
- Plain-English signals with Slack alerts
- Evals via SDK and CLI with comparison UI
- SQL queries over traces, spans, and metrics
- Custom dashboards for traces and events
- MCP and CLI access for coding agents
- Datasets and data annotation tools
Pricing: A free plan includes 1 GB of data. Starter costs $30 and Pro $150 per month with unlimited seats and per-GB overage; Enterprise is custom with an on-premise option.
#8
LangWatch is an open-source platform for tracing, evaluating and testing LLM applications and AI agents, with prompt management and an AI gateway.
- GitHub stars
- 5k
- Last commit
- today
- Latest release
- langwatch-3.23.0
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
langwatch.ai
LangWatch is an Apache-2.0 platform for running AI in production. It lets teams trace, test, route and govern LLM calls across the company, covering both the agents they build themselves and the coding assistants their engineers use. The project is aimed at AI engineers and platform teams that need visibility and control over model usage.
Its feature areas include LLM operations such as observability, agent testing, evaluations and prompt management, plus tracking of coding agent sessions with cost per pull request and per team and privacy controls. An AI gateway offers a single endpoint compatible with the OpenAI and Anthropic APIs, with virtual keys, budgets and routing, and an AI governance layer sits on top. Repository topics mention DSPy, datasets, simulation testing and low-code evaluation.
LangWatch is written in TypeScript and can be used through LangWatch Cloud or self-hosted; the README says only Node.js is required to try it locally, with separate guidance for production deployments. It suits teams moving LLM prototypes into monitored, tested production systems.
Key features
- LLM observability and tracing
- Agent testing and simulation
- Evaluations over datasets
- Managing and testing prompts
- AI gateway with virtual keys and budgets
- Coding agent cost tracking per team
Pricing: Free Developer plan with 50k events a month. Growth costs €29 per core-seat per month plus €5 per 100k events beyond 200k; Enterprise is custom. Self-hosting is supported.
#9
Helicone is an open-source LLM observability platform and AI gateway that logs requests, tracks cost and latency, and routes traffic across models.
- GitHub stars
- 6.3k
- Last commit
- yesterday
- Latest release
- v2025.08.21-1
- Licence
- Apache-2.0
- Self-hosted
- Yes
- Hosted version
- Available
helicone.ai
Helicone is an open-source platform for AI engineers that combines LLM observability with an AI gateway. It records the requests your application sends to model providers so you can trace agents, chatbots and document-processing pipelines, debug sessions, and see how much each call costs and how long it takes.
Integration is meant to be quick: a single line of code can log requests made through OpenAI, Anthropic, LangChain, Gemini, the Vercel AI SDK and similar tools. The gateway side offers access to many models through the OpenAI API format, with intelligent routing and automatic fallbacks. Other features listed in the README and repository topics include cost and latency tracking, datasets and fine-tuning support, prompt management, evaluation and a playground.
Helicone is licensed under Apache-2.0 and written in TypeScript. It can be self-hosted, and the vendor also runs a hosted cloud version. It suits teams that want visibility into LLM spend and quality without building their own logging pipeline.
Key features
- One-line request logging for LLM providers
- Agent and session tracing
- Cost and latency tracking
- AI gateway with routing and fallbacks
- Prompt management and evaluation
- Datasets for fine-tuning workflows
Pricing: Free Hobby plan with 10,000 requests. Pro costs $79 and Team $799 per month plus usage, each with a 7-day trial; Enterprise is custom.
LangSmith alternatives: questions
- What is the best open-source alternative to LangSmith?
- Dify is the top-ranked open-source alternative to LangSmith on Enlisted: An LLM application platform for building AI workflows, RAG pipelines and agents on one canvas, deployable in the cloud, a private VPC or self-hosted. Other strong options are Langfuse, TraceRoot, Opik and Arize Phoenix.
- What are the best commercial alternatives to LangSmith?
- Braintrust, Galileo, Portkey, Arize AI and Comet are among the hosted, closed-source alternatives to LangSmith listed on Enlisted, which lists 33 in total. Compare their pricing and features on each tool page.
- Are these LangSmith alternatives free?
- All 9 are open source, so the code is free to use under its licence, and all of them can be self-hosted on your own server or computer. 9 also offer a paid or managed cloud version if you'd rather not host it yourself.
- How is this list of LangSmith alternatives ranked?
- By a score built from GitHub stars, star growth over the last 30 days and how recently the code changed. 9 of these projects shipped code in the last 30 days. Data is refreshed daily, and nobody can pay to move up.