Transform
While Pipelines in Hevo address the Extract and Load jobs in our Extract-Load-Transform (ELT) approach of moving your data from the Source to the Destination warehouse, the Transform step is achieved using Models and Workflows.
Some advantages that make the ELT approach preferable to the traditional ETL approach are:
-
You get the Source data for analysis with much shorter load times than in the ETL approach, where each transformation has to be routed to the technical team and involves a much longer wait time for data to be transformed and loaded into the Destination.
-
Near-endless storage capabilities and incredible processing power of cloud-based data warehouses, such as, Amazon Redshift and Google BigQuery, allow data aggregation from numerous Sources to support multiple transformations of the data.
-
You have the Source data with you to transform in any manner and any number of times, on the fly.
-
You can transform only the data you need for a particular business analysis.
The most important fallout of the ELT approach is that now the data transformation can be easily performed by business analysts and other non-technical persons also. To enable this, Hevo offers the intuitive and easy-to-use Models interface, where you can apply transformations on the data till you reach the final data model that is required for your business reporting and intelligence needs. Using the Query Builder in Models, you can design and test standard SQL queries that can aggregate data from one or more systems and transform it for analysis and reporting and work closely with the Destination schema. You can also create SQL Join clauses that can combine data from multiple tables, and write the query results into a new table.
Workflows further extend the functionality offered by Models and provide you a powerful way to derive insights in the here and now of your business. Workflows allow you to join multiple Models, whereby the output from one Model can form the input to one or more Models. This allows you to perform deeper analysis and gather powerful data inputs for your reporting and BI needs.
You can schedule the Models and Workflows to run at specific times and frequency, giving you further independence and control in analyzing your data.