Open-source Firebase Test Lab alternatives

Appium is an open-source framework for automating apps on mobile, desktop and IoT platforms. It builds on the W3C WebDriver protocol, which means tests are written against a standard interface and can drive native apps, hybrid apps, mobile web and desktop software on Android, iOS, Windows, macOS and more.

Its architecture is modular and extensible. Drivers add support for automating particular platforms, clients let you write tests in the programming language of your choice, and plugins extend the core behavior. That gives rise to a broad ecosystem of related software around the core server. Appium is installed with npm, and the documentation covers system requirements for each platform.

The team supports only the latest version, so anyone on an older major release is directed to migration guides covering the v1 to v2 and v2 to v3 transitions. Appium is written in TypeScript under the Apache-2.0 license and runs on your own machines, build servers or device farms. It is a standard choice for QA engineers and developers who need cross-platform mobile test automation.

This entry covers Detox, the open-source mobile testing framework from Wix. Detox runs end-to-end tests against an app while it is running on a real device or simulator, driving it the way a person would, with the aim of replacing much of the manual QA that slows down continuous integration.

Its answer to flaky end-to-end tests is gray-box testing: it monitors the app's asynchronous operations and synchronizes with them automatically rather than relying on arbitrary waits. Tests are written in JavaScript with an async-await API that supports breakpoints, and the framework works with any test runner, with Jest integration provided out of the box. It targets React Native projects on Android and iOS and is designed to run in CI systems such as Jenkins, Travis CI and Circle CI.

Detox is MIT licensed and used as a development dependency in your own repository, so there is nothing to host. The README notes that support for running on real iOS devices is not yet available. It suits mobile teams that want reliable, repeatable end-to-end tests inside their delivery pipeline.

This entry covers WebdriverIO, a modern test automation framework that drives browsers and mobile apps from Node.js. It is used for end-to-end tests as well as unit and component tests in the browser, and runs tests based on the WebDriver and WebDriver BiDi protocols and on Appium automation technology.

It works with popular test frameworks for BDD and TDD, such as Mocha, Jasmine and Cucumber, and can run tests locally or in the cloud on services including Sauce Labs, BrowserStack, TestingBot and TestMu AI, formerly LambdaTest. The project provides a developer guide, API reference, changelog and roadmap, and contributors can start in a prepared dev container using GitHub Codespaces or Gitpod.

WebdriverIO is written in TypeScript and licensed under MIT. It is installed as a development dependency in a JavaScript or TypeScript project, so there is nothing to host. Commercial support is available through a Tidelift subscription. It suits QA engineers and front-end teams who want one framework for web and mobile test automation.