Pipelines
Edge Pipeline is currently available under Early Access. You can request access to evaluate and test its features.
A Pipeline moves data from a Source system to a Destination system. Your Source can be a database, such as PostgreSQL, and the Destination system can be a data warehouse, such as Snowflake.
You can connect only one Source and Destination in a Pipeline. However, multiple Pipelines may load to the same Destination.
Creating a Pipeline in Hevo involves configuring the following:
-
Source: You can configure a new Source by providing the required connection parameters or use an existing one. Read Sources for the various Sources that Hevo integrates with.
-
Destination: You can use an existing Destination or specify the necessary connection parameters to configure a new one. At this time, you can replicate data from a Source database only to a Snowflake Destination. Read Destinations.
-
Pipeline: In the Pipeline configuration, you can define the way in which you want Hevo to load data to your Destination. Here, you can specify how your Destination schema should evolve and also how often the data from the Source should be synchronized with the Destination.
-
Objects: Hevo allows you to select the objects from which you want to replicate data. You can also define a primary key for your object here if you have opted to load data to the Destination with the Merge data loading policy.