Apache Spark
Unified engine for large-scale data analytics
Apache Spark is an open-source, multi-language analytics engine designed for large-scale data engineering, data science, and machine learning workloads. It supports Python, SQL, Scala, Java, and R, and can run on single-node machines or distributed clusters. Spark provides high-level APIs for batch processing, streaming, and ML model training.
Users install PySpark or use the official Docker image to run distributed data processing jobs using DataFrame APIs or SQL across single nodes or clusters.
Data engineers, data scientists, and machine learning practitioners
Background.
- Status
- launched
- Business model
- open-source
- Company
- Apache Software Foundation
Similar projects.
Editorial take on the space this project sits in — momentum signals, adjacent moves, our call on whether the wedge is real. Get pinged when we publish a new read or when the landscape shifts.
Have a take on this space?
Tell us what you’d build differently, where you think the incumbents miss, or what we’ve gotten wrong about this project. Comments + reactions are coming soon.