How do I change the query mode for Pipelines created with Table mode?

For an existing Pipeline created with the Table Pipeline mode, you can only change the query mode of individual objects and not of the entire Pipeline.

To change the query mode for an object:

  1. Select the Edit Config option from the Action menu of the object in the Pipeline Detailed View page.

    Select the Edit option

  2. Select the required Query Mode from the drop-down.

    Select the Query Mode

    Depending on the query mode, specify the following parameters:

    Query Mode Parameter
    Full Load None
    Delta-Timestamp Timestamp Column (name) and Timestamp Column Delay
    Change Data Capture Incrementing Column (name), Timestamp Column (name), and Timestamp Column Delay
    Unique Incrementing Append Only Unique Column (name)
    XMIN (available only for PostgreSQL Sources) XMIN Column

    For example, for the Delta-Timestamp query mode, you must specify the parameters as shown below:

    Delta-Timestamp Query Mode Example

  3. Click SAVE CHANGES.

The ingestion for the object starts once you change the query mode. Please note that the entire data is ingested again from the beginning. The re-ingested historical data is not billed.

Read Query Modes for Ingesting Data from Relational Databases for more information.

Note: This method works only for the Pipelines having a relational database (such as Amazon Aurora MySQL, Amazon Redshift, SQL Server, MySQL, or PostgreSQL) as the Source.


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Nov-07-2022 NA Created as a new document.
Last updated on 04 Nov 2022

Tell us what went wrong