If I remove Hevo metadata columns from the Destination table, are these regenerated in the next load?

Yes, even if you manually remove Hevo generated metadata columns from the Destination table, Hevo re-generates all these columns while ingesting new Events and creates the associated metadata. These columns are used along with the primary keys to deduplicate the ingested Events, which is why Hevo adds them back even after they are dropped.

The metadata columns created by Hevo are __hevo_id, __hevo_database_name, __hevo_ingested_at, __hevo_loaded_at , __hevo_source_modified_at and __hevo__marked_deleted. Of these, you can choose not to add the __hevo_loaded_at column to the Destination table. All other columns are required and when you delete them, the metadata information they hold is lost. In the next run of the Pipeline, while Hevo recreates these columns, the absence of similar metadata in the Destination table can result in some duplicates.

Read Loading Data in a Database Destination - Hevo Data.


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Nov-07-2022 NA Created as a new document.
Last updated on 03 Nov 2022

Tell us what went wrong