Data Loss Prevention
ON THIS PAGE
There can be situations where your Pipeline fails to run as per the defined schedule due to a functional bug or technical issue, such as, a power outage. While for SaaS-based Sources you can re-ingest the data by adjusting the offset and re-running the Pipeline, for webhook and log-based Sources, this can potentially lead to data getting lost. Specifically:
Webhook Sources: Lost data cannot be recovered as the ingestion is push-based and the offset cannot be adjusted to a historical date and time.
Log-based Sources: Data may be recovered if the issue is resolved within the log retention period. If the log has expired, the data cannot be re-ingested.
Automatic Data Backup for Recovery
To overcome the risk of data loss, Hevo keeps an encrypted backup of your last three days’ data for webhook and log-based Sources. This helps avoid absolute data loss as offset can be reset to three days prior, and the data can be re-ingested. The primary keys, if any, are retained during the data recovery process.