Events not Found for future timestamp values in CDC mode

Applies To Database Pipelines created with CustomSQL or Table ingestion modes.
Error Message Text(s) Events not found for future timestamp values in CDC mode

Error Scenario

This error may occur for a Pipeline created with CustomSQL and Table ingestion modes, if a timestamp column configured for the Change Data Capture (CDC) query mode contains values that are in the future, with respect to the Source database or server timezone.

Potential Causes

The Change Data Capture (CDC) query mode uses the now() function to get new records and updates that are newer than the last ingested record. The now() function is a timestamp function that gets the exact timestamp of the moment at which the function is run. Read MySQL NOW() Function for more information.

If the Source timestamp values are in the future, there are no records older than the timestamp fetched by the now() function, hence, Hevo does not have any records to ingest.

Suggested Action(s)

Set the Milliseconds to wait value to a negative value, for example, -25200000 for 7 hours, or a value as per the difference you expect the data to have from the current time of the Source database or server, using the Edit config option for the concerned object.

Edit object config

The Pipeline adds this value to the timestamp of the now() function to identify the records to ingest. The Milliseconds to wait setting enables the Pipeline to check the configured timestamp field’s value in the past for a positive value, and in the future for a negative value.
For example, if your Source has timestamp values 7 hours ahead of the current time, the value, -25200000 tells the Pipeline to check for timestamp values 7 hours in the future.


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Nov-08-2022 NA New document.
Last updated on 09 Nov 2022

Tell us what went wrong