Pausing a Log-based Pipeline

Each log has a retention period associated with it which specifies the duration a log is maintained before it is automatically deleted. Logs are purged periodically to prevent them from taking up too much disk space.

Consequences of Pausing a Log-based Pipeline

Pausing a Pipeline pauses the process of ingestion. Pausing a log-based Pipeline for more than 24 hours may lead to data loss as a result of the log being deleted. The log can get deleted due to the expiry of its retention period or limited disk storage space in the case of large log files.

In case there is a loss of data after resuming a paused Pipeline, restart the historical load for all the objects to ingest the lost data. To do so, in the Pipeline Overview page:

Note: You must be assigned the Team Administrator, Team Collaborator, Pipeline Administrator, or Pipeline Collaborator role in Hevo, to pause a log-based Pipeline.

1. Select the Objects check box to select all the objects in the Pipeline. You can also select specific objects by selecting the check box next to their names.

   

2. Select the Restart option from the MORE drop-down to start the historical data ingestion.

The historical load starts immediately. The re-ingested data does not count towards your quota consumption and is not billed.

Note: You can similarly recover lost data by restarting individual jobs configured with change tracking mode in the case of an SQL Server Source.

See Also

• Data Loss Prevention

Revision History