- Introduction
- Getting Started
- Data Ingestion
- Data Loading
- Loading Data in a Database Destination
- Loading Data to a Data Warehouse
- Optimizing Data Loading for a Destination Warehouse
- Manually Triggering the Loading of Events
- Scheduling Data Load for a Destination
- Loading Events in Batches
- Name Sanitization
- Table and Column Name Compression
- Parsing Nested JSON Fields in Events
- Pipelines
- Data Flow in a Pipeline
- Ingestion Modes
- Familiarizing with the Pipelines UI
- Pipeline Objects
- Working with Pipelines
- Transformations
-
Schema Mapper
- Using Schema Mapper
- Mapping Statuses
- Auto Mapping Event Types
- Mapping a Source Event Type with a Destination Table
- Mapping a Source Event Type Field with a Destination Table Column
- Schema Mapper Actions
- Fixing Unmapped Fields
- Resolving Incompatible Schema Mappings
- Resizing String Columns in the Destination
- Schema Mapper Compatibility Table
- Failed Events in a Pipeline
- Pipeline FAQs
- Events Usage
- Sources
- Free Sources
- Analytics
- Collaboration
- CRM
- Data Warehouses
- Databases
- E-Commerce
- File Storage
- Finance & Accounting
-
Marketing
- AdRoll
- Apple Search Ads
- AppsFlyer
- Criteo
- Facebook Ads
- Facebook Page Insights
- Google Ads
- Google Campaign Manager
- Google Play Console
- Google Search Console
- HubSpot
- Instagram Business
- Klaviyo
- LinkedIn Ads
- Mailchimp
- Marketo
- Microsoft Advertising
- Outbrain
- Pardot
- Pinterest Ads
- Segment
- SendGrid
- SendGrid Webhook
- Salesforce Marketing Cloud
- Snapchat Ads
- Taboola
- Twitter Ads
- Streaming
- Source FAQs
- Destinations
- Transform
- Activate
- Alerts
- Account Management
- Troubleshooting
-
Troubleshooting Sources
- Troubleshooting Amazon DynamoDB
- Troubleshooting MongoDB
- Troubleshooting MS SQL
- Troubleshooting MySQL
- Troubleshooting Oracle
-
Troubleshooting PostgreSQL
-
Errors during Pipeline creation
- Error 1003 - Authentication failure
- Error 1006 - Connection settings errors
- Error 1011 - Access role issue for logical replication
- Error 1012 - Access role issue for logical replication
- Error 1014 - Database does not exist
- Error 1017 - Connection settings errors
- Error 1023 - No pg_hba.conf entry
- Error 1024 - Number of requested standby connections
-
Errors during Pipeline creation
- Troubleshooting Salesforce
- Troubleshooting Destinations
-
Troubleshooting Sources
- Glossary
- Release Notes
- Release Version 1.88
- Release Version 1.87
- Release Version 1.86
- Release Version 1.84 & 1.85
- Release Version 1.83
- Release Version 1.82
- Release Version 1.81
- Release Version 1.80 (Jan-24-2022)
- Release Version 1.79 (Jan-03-2022)
- Release Version 1.78 (Dec-20-2021)
- Release Version 1.77 (Dec-06-2021)
- Release Version 1.76 (Nov-22-2021)
- Release Version 1.75 (Nov-09-2021)
- Release Version 1.74 (Oct-25-2021)
- Release Version 1.73 (Oct-04-2021)
- Release Version 1.72 (Sep-20-2021)
- Release Version 1.71 (Sep-09-2021)
- Release Version 1.70 (Aug-23-2021)
- Release Version 1.69 (Aug-09-2021)
- Release Version 1.68 (Jul-26-2021)
- Release Version 1.67 (Jul-12-2021)
- Release Version 1.66 (Jun-28-2021)
- Release Version 1.65 (Jun-14-2021)
- Release Version 1.64 (Jun-01-2021)
- Release Version 1.63 (May-19-2021)
- Release Version 1.62 (May-05-2021)
- Release Version 1.61 (Apr-20-2021)
- Release Version 1.60 (Apr-06-2021)
- Release Version 1.59 (Mar-23-2021)
- Release Version 1.58 (Mar-09-2021)
- Release Version 1.57 (Feb-22-2021)
- Release Version 1.56 (Feb-09-2021)
- Release Version 1.55 (Jan-25-2021)
- Release Version 1.54 (Jan-12-2021)
- Release Version 1.53 (Dec-22-2020)
- Release Version 1.52 (Dec-03-2020)
- Release Version 1.51 (Nov-10-2020)
- Release Version 1.50 (Oct-19-2020)
- Release Version 1.49 (Sep-28-2020)
- Release Version 1.48 (Sep-01-2020)
- Release Version 1.47 (Aug-06-2020)
- Release Version 1.46 (Jul-21-2020)
- Release Version 1.45 (Jul-02-2020)
- Release Version 1.44 (Jun-11-2020)
- Release Version 1.43 (May-15-2020)
- Release Version 1.42 (Apr-30-2020)
- Release Version 1.41 (Apr-2020)
- Release Version 1.40 (Mar-2020)
- Release Version 1.39 (Feb-2020)
- Release Version 1.38 (Jan-2020)
- Upcoming Features
Handling Deletes in Log-based Pipelines
On This Page
In Pipelines created using the logical replication mode, Hevo uses the database logs to replicate data. As a result, Hevo can track all operations, such as insert, update, or delete, that take place in the database and replicate them accordingly. Hevo replicates delete actions in the database logs to the Destination table by marking the value of the metadata column, __hevo__marked_deleted
as True.
Hevo track deletes in Pipelines configured for the following RDBMS Sources in the respective Ingestion modes as listed in the table:
Source | Ingestion Mode |
---|---|
MySQL | BinLog |
PostgreSQL | Write-ahead Log |
MongoDB | - OpLog - Change Streams Note: In either of these modes, Hevo cannot track array elements deleted in the MongoDB document at Source, as the logs contain the updated array. To capture such delete actions, you must create a Full Load Model. Read Types of Models. |
SQL Server | Change Tracking Note: Enabling Change Tracking at the database or object-level automatically selects Change Tracking as the query mode for the Source objects that you want to track in new Pipelines. |
Oracle | Redo Log |
AWS DynamoDB | Streams |
Revision History
Refer to the following table for the list of key updates made to this page:
Date | Release | Description of Change |
---|---|---|
Mar-21-2022 | NA | Revised the content for clarity and coherence. |
Last updated on 21 Mar 2022