Replay Queue is one of the most critical components of the Pipeline. Replay Queue ensures that under no circumstances data is lost or dropped from the Pipeline. It is the place where Hevo parks the data when something is not right.
Whenever Hevo finds that in an event cannot proceed ahead in the Pipeline it is parked in the Replay Queue and made visible on the Hevo UI for inspection.
There can be several reasons for us to park data into the Replay Queue:
- Network failures. We might encounter network or connectivity failures with your destination at times. We might also encounter network problems in Hevo's infrastructure. Though we make the best efforts to eliminate all such errors they are still bound to appear once in a while.
- Schema incompatibility and conflicts. Schema changes in your Sources and Destinations may change without us knowing about those changes. Though we make the best efforts to keep ourselves in sync with all those changes, there can always be a certain lag in doing so.
- Bad transformation code. Transformation code provided by you may have bugs.
- Incorrect mappings. There can be manual errors while mapping source and destination tables.
Replay Queue makes all such failed events visible on the UI as shown above.
You can Replay all events in the Replay Queue at once by clicking REPLAY ALL button in the Replay Queue section. You can also select an event type and Replay events for a particular reason using REPLAY link.
After being replayed events are fed back into the Transformations stage of the pipeline.
Note: In case the events were created through the Transformations Code they are fed back to the Schema Mapper stage rather than Transformations stage.
It is possible to skip events when they are useful anymore. You can Skip all events in the Replay Queue at once by clicking SKIP ALL button in the Replay Queue section. You can also select an event type and Skip events for a particular reason using SKIP link.
One skipped, events are purged from the Pipeline and cannot be recovered later.
View Sample events
You may want to view a few sample events which were parked in the Replay Queue to get to the root cause especially in cases where the event was parked due to a schema compatibility issue. You can view Sample events by clicking the View Sample link.
For the rows in the Replay Queue with a lightning icon, Hevo attempts to automatically replay the events when it finds that the root cause has been resolved. Please note that the attempts are made periodically every few minutes and Hevo uses complex algorithms to determine whether to auto-replay events or not. The lightning icon showing up does not guarantee that the events will be auto-replayed.