If I remove Hevo metadata columns from the Destination table, are these regenerated in the next load?

Yes, even if you manually remove Hevo generated metadata columns from the Destination table, Hevo re-generates all these columns while ingesting new Events and creates the associated metadata. These columns are used along with the primary keys to deduplicate the ingested Events, which is why Hevo adds them back even after they are dropped.

The metadata columns created by Hevo are __hevo_id, __hevo_database_name, __hevo_ingested_at, __hevo_loaded_at , __hevo_source_modified_at and __hevo__marked_deleted. Of these, you can choose not to add the __hevo_loaded_at column to the Destination table. All other columns are required and when you delete them, the metadata information they hold is lost. In the next run of the Pipeline, while Hevo recreates these columns, the absence of similar metadata in the Destination table can result in some duplicates.

Read Loading Data in a Database Destination - Hevo Data.

Revision History