Why is the Pipeline not ingesting data from my PostgreSQL Source objects?

Last updated on Jul 04, 2023

Your Pipeline may not ingest data from the selected PostgreSQL Source objects (tables) for the following reasons:

Insufficient permissions

The database user configured in your Pipeline does not have permissions to read data from the selected objects. You need to grant permissions to the database user. Refer to your PostgreSQL variant for the steps.

Include New Tables in the Pipeline option disabled

It is possible that the objects were created or restored after the Pipeline was created, and the Pipeline is not configured to ingest data from new objects.

You should modify your Pipeline to enable the Include New Tables in the Pipeline option for your Source.

In addition, ensure that the database user has the permission to read data from new tables created in your Source database. If permissions are granted, the new objects are included in the Pipeline but are in SKIPPED state. You can include these objects in the Pipeline to ingest data from them.

Refer to the section, Grant Privileges to the User in your PostgreSQL variant for the steps to grant access to the database user.

Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Mar-23-2023 NA Added information for clarity in the section, Include New Tables in the Pipeline option disabled.
Mar-15-2023 NA New document.

Tell us what went wrong