In the realm of data management and analytics, the terms ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) have been commonplace for decades. They describe the processes involved in moving data from one system to another, transforming it as needed along the way. However, with the advent of technologies like Salesforce Data Cloud, a new concept is gaining traction: “noETL / noELT.” But what does this mean for you, especially if you’re not knee-deep in the technical jargon of data integration? Let’s explore.
Understanding ETL and ELT
First, a quick refresher on ETL and ELT:
- ETL (Extract, Transform, Load): This traditional approach involves extracting data from various sources, transforming it into a usable format, and then loading it into a target system, often a data warehouse or analytics platform.
- ELT (Extract, Load, Transform): ELT reverses the transformation step, loading raw data directly into the target system and then transforming it as needed within that system.
Both ETL and ELT have their pros and cons, but they can be complex and time-consuming processes, requiring specialized skills and infrastructure.
Enter noETL / noELT
Now, let’s talk about noETL / noELT, as championed by platforms like Salesforce Data Cloud. The “no” in noETL / noELT signifies a departure from the traditional data integration approaches. Here’s what it means for you:
1. Faster Time to Insights
- With noETL / noELT, data can be accessed and utilized more quickly. Instead of waiting for data to go through multiple transformation stages, you can start analyzing it almost immediately.
2. Real-Time or Near Real-Time Analytics
- By eliminating the upfront transformation step, noETL / noELT enables real-time or near real-time analytics. This means you can make decisions based on the most current data available.
3. Simplified Data Integration
- For end-users, noETL / noELT translates to simplified data integration. You don’t need to worry about intricate data pipelines or complex transformation logic. Data becomes more accessible and usable.
4. Scalability and Cost Efficiency
- Scalability is often improved with noETL / noELT, as it reduces the overhead associated with managing large-scale data integration processes. This can result in cost savings as well.
What it Means for You
If you’re a business user, analyst, or decision-maker leveraging Salesforce Data Cloud or similar technologies, here’s what noETL / noELT means for you:
- Ease of Use: You can focus more on extracting value from data rather than managing its integration.
- Quicker Insights: Rapid access to data means quicker insights, enabling faster and more informed decision-making.
- Adaptability: NoETL / noELT architectures are often more adaptable to changing data sources and analytical needs.
Where are we today?
As of April 2024 there are two platforms that are Generally Available (GA) that can be used like this with Salesforce Data Cloud.
- Snowflake
- Google BigQuery
There are two other platforms that are in Pilot mode as of April 2024. We are excited to see those move from Pilot to GA.
- Databricks
- Amazon RedShift
And looking forward, as mentioned in this article at cio.com, Salesforce Data Cloud is looking towards leveraging these two abilities moving forward.
- Allowing for data lakes that use Apache Iceberg to surface in Data Cloud with direct file access at the storage level.
- Salesforce Data Cloud will also add zero-copy support to the Data Kits that ISVs use to distribute datasets and enrich customers’ data in Salesforce Data Cloud.
What we are so excited about at Perficient is that we can bring expertise to both sides of a project involving these technologies. We have two different business units that focus on each side…
- A Salesforce Business Unit with experts in Salesforce Data Cloud
- A Data Solutions Business Unit to help with the Data Lake solutions like Snowflake, Google BigQuery, Databricks and Amazon Redshift.
- Here is a recent blog post from a colleague of mine in that Data Solutions business unit.
In conclusion, the rise of noETL / noELT represents a significant shift in how we approach data integration and analytics. It promises to democratize data access and streamline processes for users across organizations. As these technologies continue to evolve, staying informed about their implications will be crucial for maximizing their benefits. Embrace the simplicity and agility that noETL / noELT brings, and harness the power of data more effectively in your day-to-day operations.
Woww ..awesome and useful stuff really liked your efforts
Terry Luschen, what a fantastic article! It was incredibly informative and beautifully written.
The new Salesforce Zero Copy Network isn’t just groundbreaking for data efficiency; it’s also a win for the environment. By using fewer resources to extract data and leaving it in place for analysis, it aligns perfectly with companies that are environmentally conscious.
This space is truly exciting, especially when paired with the capabilities of generative AI. Imagine being able to feed some of these data streams into a RAG (Retrieval-augmented generation) – it’s a concept that I find absolutely fascinating.