Azure Databricks – Capacity Planning for optimum Spark Cluster
Perficient Digital Transformation
JUNE 7, 2022
Provides an interactive workspace that enables collaboration between data engineers, data scientists and machine learning engineers. In a big data pipeline, the data is ingested into Azure cloud through Azure Data Factory in batches, or streamed near real-time using Apache Kafka, Event Hub or IoT Hub. of Executors = 60GB / 15 = 4GB.
Let's personalize your content