Each entry in the File Log provides the following information:
- Name of the File
- The time at which Hevo started ingesting the file
- The progress statistics for file ingestion. More about this later.
- Current ingestion status of the file.
- The time at which the ingestion status changed.
A file can have one of the following statuses:
- Ingesting - Hevo is downloading and ingesting the file into the pipeline
- Ingested - Hevo has completely ingested the file into the pipeline. At this stage, all the records in the file haven't synced to the destination
- Synced - All the records in the files have been synced to the destination. The status of the file is synced even if some of the records are filtered or skipped in the pipeline.
- Failed - Hevo has failed to read or ingest the file.
Note: If some of the events of the file are parked in the replay queue, the status of the file remains as Ingested until and unless all the records are synced (or filtered or skipped) to the destination.
How to read progress statistics of a file
The progress bar shown along with each entry in the file log displays the statistics of the records in the following stages:
- Synced - The records which have been synced successfully to the destination
- Failed - The records which didn't reach the destination due to a failure. All these records are parked in the Replay Queue.
- Filtered - The records that have been filtered out through the transformations code
- Skipped - The records skipped through the Skip Event option in the Schema Mapper
Hovering on any of the statuses displays the number of records in the said status against the total number of records in the file.