Data Loss Prevention and Recovery

There can be situations where your Pipeline fails to run as per the defined schedule due to a functional bug or technical issue such as a power outage. While for SaaS-based Sources you can re-ingest the data by adjusting the offset and re-running the Pipeline, for webhook and log-based Sources, this can potentially lead to data getting lost. Specifically:

• Webhook Sources: Lost data cannot be recovered as the ingestion is push-based and the offset cannot be adjusted to a historical date and time.

• Log-based Sources: Data may be recovered if the issue is resolved within the log retention period. If the log has expired, the data cannot be re-ingested.

Automatic Data Backup for Recovery

To overcome the risk of data loss, Hevo keeps an encrypted backup of your last three days’ data for webhook and log-based Sources. This helps avoid absolute data loss as offset can be reset to three days prior, and the data can be re-ingested. The primary keys, if any, are retained during the data recovery process.