Apache Airflow
Programmatically author, schedule and monitor workflows with Python
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring data and ML workflows. Pipelines are defined in Python, enabling dynamic generation and full flexibility. It features a modern web UI, scalable architecture, and integrations with major cloud providers like AWS, GCP, and Azure.
Users define workflows as Python DAGs (Directed Acyclic Graphs), which Airflow schedules, executes via workers, and monitors through a web interface.
data engineers and developers building and managing automated workflows
Background.
- Status
- launched
- Business model
- open-source
- Company
- Apache Software Foundation
Similar projects.
Editorial take on the space this project sits in — momentum signals, adjacent moves, our call on whether the wedge is real. Get pinged when we publish a new read or when the landscape shifts.
Have a take on this space?
Tell us what you’d build differently, where you think the incumbents miss, or what we’ve gotten wrong about this project. Comments + reactions are coming soon.