PostgreSQL

Hevo supports the following variations of PostgreSQL as a Source:

Click each variant above to know the prerequisites and pre-configuration setups specific to it. Once you complete the setups, follow the steps mentioned below to configure it as a Source in your Hevo Pipeline and start ingesting data.


Select the Source Type

To select PostgreSQL as the Source:

  1. Click PIPELINES in the Asset Palette.

  2. Click + CREATE in the Pipelines List View.

  3. In the Select Source Type page, select the PostgreSQL variant.

Select the Pipeline Mode

Select how you want Hevo to read your data from the PostgreSQL Source. This can be Custom SQL, Logical Replication, or Table.

Read Pipeline Modes.

  • For Pipeline mode as Table, refer to Object Settings below to configure the objects to be replicated.

  • For Pipeline mode as Logical Replication, follow the steps provided in each PostgreSQL variant document to set up logical replication.

Specify PostgreSQL Connection Settings

In the Configure your PostgreSQL Source page, specify the following:

PostgreSQL settings

  • Pipeline Name: A unique name for your Pipeline.

  • Database Host: The PostgreSQL host’s IP address or DNS.

  • Database Port: The port on which your PostgreSQL server is listening for connections. The default port is 5432.

  • Database User: The read-only user that can read the tables in your database.

  • Database Password: Password for the read-only user.

  • Database Name: The database that you wish to replicate.

  • Schema Name: The name of the Schema in the database.

  • Connection Settings:

    • Connect through SSH: If you want to connect to Hevo using an SSH tunnel, check Connecting Through SSH. Also, you will have to whitelist Hevo’s IP addresses. Refer to the content for your PostgreSQL variant for steps to do this.

    • Use SSL: Enable it to use SSL encrypted connection. You should also enable this if you are using Heroku PostgreSQL databases. To enable this, specify the following:

      • CA File: The file containing the SSL server certificate authority (CA).

      • Client Certificate: The client public key certificate file.

      • Client Key: The client private key file.

  • Advanced Settings (for Pipelines with Logical Replication mode)

    • Load Historical Data: If this option is enabled, the entire table data is fetched during the first run of the Pipeline. If disabled, Hevo loads only the data that was written in your database after the time of creation of the Pipeline.

    • Merge Tables: If this option is enabled, Hevo merges tables with the same name from different databases while loading the data to the warehouse. Hevo loads the Database Name field with each record. If disabled, the database name is prefixed to each table name. Read How does the Merge Tables feature work?.

Click TEST & CONTINUE. Proceed to configuring the data ingestion and setting up the Destination.


Object Settings

Object settings must be configured if the Pipeline mode is Table. To do this:

  • After specifying the Source settings above, in the SELECT OBJECTS YOU WANT TO REPLICATE page, select the objects to be replicated.

    Note: Each object represents a table in your database.

  • In the CONFIGURE SOURCE OBJECTS page, specify the query mode to be used for each selected object.

Last updated on 11 Feb 2021