Handling Deletes in Log-based Pipelines

Last updated on Feb 03, 2023
On This Page

In Pipelines created using the logical replication mode, Hevo uses the database logs to replicate data. As a result, Hevo can track all operations, such as insert, update, or delete, that take place in the database and replicate them accordingly. Hevo replicates delete actions in the database logs to the Destination table by marking the value of the metadata column, __hevo__marked_deleted as True.

Hevo track deletes in Pipelines configured for the following RDBMS Sources in the respective Ingestion modes as listed in the table:

Source Ingestion Mode
MySQL BinLog
PostgreSQL Write-ahead Log
MongoDB - OpLog
- Change Streams

Note: In either of these modes, Hevo cannot track array elements deleted in the MongoDB document at Source, as the logs contain the updated array. To capture such delete actions, you must create a Full Load Model. Read Types of Models.
SQL Server Change Tracking

Note: Enabling Change Tracking at the database or object-level automatically selects Change Tracking as the query mode for the Source objects that you want to track in new Pipelines.
Oracle Redo Log
AWS DynamoDB Streams


See Also

Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Mar-21-2022 NA Revised the content for clarity and coherence.
Was this page helpful?

Tell us what went wrong