Open-source Azure AI Foundry alternatives

Dify is a platform for building applications on top of large language models. Instead of wiring together prompts, retrieval code and model APIs by hand, teams assemble AI workflows, knowledge pipelines and agents in a shared visual workspace, then move them from prototype to production without changing stacks.

The product covers a workflow studio for agentic flows, a knowledge pipeline that prepares searchable knowledge bases for retrieval-augmented generation, model management across many providers, and a marketplace of tools, models and integrations. Observability integrations such as Opik, Langfuse and Arize Phoenix help trace and debug runs once an app is live.

Dify comes in a few editions: a community edition you deploy yourself with Docker, a hosted cloud service, and an enterprise edition for private deployments. The repository carries a custom license rather than a standard one, so check its terms before offering Dify as a service to others.

Langflow is an open-source platform for building AI-powered agents and workflows. Developers assemble flows in a visual authoring interface and can then deploy each flow as an API or export it as JSON for use in Python applications. It is written in Python and released under the MIT license.

The visual builder is paired with source-code access, so any component can be customized in Python, and an interactive playground lets you test and refine flows step by step. Langflow can orchestrate several agents with conversation handling and retrieval, works with the major LLMs and vector databases, and can expose flows as an MCP server so they become tools for MCP clients. Observability integrations include LangSmith and LangFuse.

You can install the Python package locally with uv, after which it serves the app at a local address, or use Langflow Desktop for Windows and macOS, which bundles its dependencies. It suits developers and AI teams prototyping retrieval and agent workflows who want both a drag-and-drop canvas and the freedom to write code.

vLLM is an open-source library for running and serving large language models efficiently. It originated at UC Berkeley's Sky Computing Lab and is now developed by a large community of contributors from academia and industry. It is written in Python and released under the Apache-2.0 license.

Speed comes from techniques such as PagedAttention for attention key-value memory, batching incoming requests continuously, chunked prefill, prefix caching, CUDA and HIP graphs, and optimized attention and mixture-of-experts kernels. It supports many quantization formats, including FP8, INT8, INT4, GPTQ and AWQ, as well as speculative decoding and disaggregated prefill and decode.

On the usability side, vLLM integrates with Hugging Face models and supports parallel sampling, beam search, streaming outputs, structured outputs and tool calling, plus several parallelism modes (tensor, pipeline, data, expert and context) for distributed inference. It targets NVIDIA, AMD and TPU hardware and suits teams that serve open models in production and need high throughput.

LiteLLM is an AI gateway, open source, offering one unified interface for calling more than 100 LLM providers, including OpenAI, Anthropic, Gemini, Bedrock and Azure, using the OpenAI request format. It can be used as a Python SDK inside an application or deployed as a proxy server that serves a whole team or organization. The core is Python with a Rust component, and GitHub lists the license as 'Other'.

The project aims to remove the hassle of handling a separate SDK, login scheme, request format and set of errors for each model. On top of that the gateway provides load balancing, guardrails, spend tracking, virtual keys and an admin dashboard, and it exposes endpoints for chat completions, responses, embeddings, images, audio, batches and reranking.

Because requests keep the OpenAI format, teams can swap providers without rewriting application code. It is self-hosted, with hosted-proxy and enterprise tier options linked in the README, and its topics also reference an MCP gateway and LLMOps. It suits platform and AI engineering teams that need central control over model access, budgets and logging.

Langfuse is an open-source LLM engineering platform that helps teams develop, monitor, evaluate and debug AI applications together. You instrument your app, and Langfuse records traces of LLM calls and surrounding logic such as retrieval, embeddings and agent actions, so complex runs and user sessions can be inspected and debugged.

Prompt management lets you centrally version and iterate on prompts, with caching on server and client so that changes do not add latency to your app. Evaluation features cover LLM-as-a-judge, code evaluators, user feedback, manual labeling and custom pipelines through the API and SDKs. Datasets provide test sets and benchmarks, there is an LLM playground for trying prompts, and integrations exist for OpenAI, LangChain and LlamaIndex. It is built on the ClickHouse database, and the Langfuse team has been part of ClickHouse since January 2026.

You can use Langfuse Cloud or self-host it, which the maintainers say takes minutes. The repository license is listed as 'Other' because it combines open-source and enterprise components, so check the terms. It suits teams shipping production LLM features who need visibility into quality, latency and cost.