Optimus

Optimus is a workflow orchestrator for data transformation, data modeling, pipelines and data quality management. Analysts and engineers write simple SQL queries and YAML configuration, and Optimus takes care of dependency management, scheduling and running transformation jobs at scale. The listing name Raystack is the organization behind it.

Key capabilities include managing warehouse tables and views through YAML, scheduling SQL transformations, automatically parsing queries to build dependency graphs rather than hand-writing DAGs, dry-running queries during deployment, and compile-time templating with variables, loops, conditionals and macros. It also supports cross-tenant dependencies, post-transformation hooks such as sinking BigQuery tables to Kafka, Python transformations and custom plugins.

Specifications are managed in git and the service is driven through REST and gRPC. The topics point to Airflow and BigQuery integrations, and the Go codebase is licensed under Apache-2.0. It suits data teams building ELT or analytics engineering workflows.