Creating a Pipeline

Last updated on May 30, 2023
On This Page

Read Creating your First Pipeline to familiarize yourself with the steps for creating a Pipeline. The configuration settings that you are required to specify are displayed based on multiple factors such as:

  • The type of Source. For example, JDBC, SaaS, or File-based. Read each Source for the settings required for it.

  • The format of the Source data, and the format you require to maintain at the Destination. For example, the format in which JSON Events may be parsed, or the format the data is in (XML, CSV) in an Amazon S3 Source. Read Parsing Nested JSON Fields in Events.

  • The amount and type of data you want to ingest. For example, historical data or incremental data. Read Types of Data Synchronization.

  • The way in which you want Hevo to read your data. Read Ingestion Modes and Query Modes for Ingesting Data.

You can configure separate Pipelines to ingest data from the same Source table and load it into different Destinations.

In the case of database and data warehouse Sources, the Source and the Destination can be in the same instance. However, you cannot move data between the same database via the Pipeline.

If you navigate away from the Pipeline creation page, Hevo saves the Pipeline in Draft status. You, or any other member of your team member, can resume setting up that draft Pipeline at a later time or delete it if you do not wish to continue with it. Read Draft Pipelines.

See Also

Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Dec-07-2022 NA Added information about separate Pipelines ingesting data from the same Source table.
Aug-30-2022 1.96 Updated page to add information about draft Pipelines.
Jun-28-2021 NA Moved section, Modifying Source and Destination Configuration to a new page.
Jun-01-2021 1.64 Added section, Modifying Source and Destination Configuration.

Tell us what went wrong