Zero - ETL ?

Created: 17 Jan 2023, Modified: •

•

alt text

AWS announced ETL-free integration between Aurora and Redshift, and between Redshift and Apache Spark
Google announced the Zero-ETL approach with big Query
Databricks announced direct queries on external systems using JDBC
Microsoft Lake storage, in combination with Snowflake / Snowpipe, is close to the Zero-ETL approach

As Data Engineer, We well-know a question “What is Zero ETL”?

So what is Zero ETL

The Zero ETL is ideated to eliminate these problems by providing a secure way for data to move between different systems without any manual intervention. It ensures that all data is up-to-date through continuous federation between all connected systems.
In Zero ETL, engineers will learn how to process data in the data lakes, lake houses, warehouses, or Spark clusters and Python scripts, so on, and they will rely on low-code platforms for many of their needs.
This approach can be useful in situations where data needs to be transferred quickly and efficiently between systems, to enable near real-time analytics and machine learning (ML) without the need for complex data transformation or manipulation.
With near real-time access to transactional data, you can leverage this for analytics and capabilities such as built-in ML, materialized views, data sharing, and federated access to multiple data stores and data lakes to derive insights from transactional and other data.