Scheduling a Pipeline

On This Page

The Pipeline schedule determines how frequently Hevo fetches the data from a Source. (It may take slightly longer for the data to be visible in your Destination). The Pipeline frequency also impacts the rate at which your Events quota gets consumed. Read Pipeline Frequency and Events Quota Consumption for more information.

A Pipeline may run multiple ingestion jobs to fetch data from your Source. All these jobs run as per a schedule defined for them. The schedule is displayed below the Source name in the Pipeline Summary.

Pipeline Schedule

You can change the data ingestion schedule from the Pipeline’s Action menu. Every Source can have a different minimum and maximum ingestion frequency.

The changed schedule is applicable immediately after change. Let us suppose you change the schedule at 1.00 p.m. (UTC) and set a 4-hour ingestion frequency. The Events are now ingested at 1.00 p.m. (UTC), 5.00 p.m. (UTC), 9.00 p.m. (UTC), and so on.

The Change Schedule feature is not available for the following:

  • Log-based Pipelines: A high ingestion frequency causes accumulation of logs leading to their expiry.

  • Webhooks: As Events are pushed in real-time from the Source, the concept of schedule does not apply to Webhooks.

  • Sources that do not allow modifications to the ingestion frequency.

Other than the scheduled ingestion run, ingestion may be triggered explicitly via the Run Now option. To run the ingestion for the entire Pipeline, click Run Now in the Actions menu. To run the ingestion for a particular object, click Run Now in the Actions menu for the object.

Run ingestion for an object

Similar to ingestion, you can also define the schedule for loading data into your Destination. The schedule applies to all the Pipelines that may be configured with that Destination. Due to this, the schedule cannot be controlled at the Pipeline level. Read Scheduling Data Load for a Destination.

See Also

Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Nov-10-2021 NA - Added information in the Overview section about how Pipeline frequency impacts the Events Quota Consumption.
- Added a See Also link to the Pipeline Frequency and Events Quota Consumption page.
Oct-04-2021 1.73 Added information about how the Change Schedule feature can be used for scheduling a Pipeline.
Sep-20-2021 NA Added the last paragraph about scheduling data loads at the Destination level.
Last updated on 11 Nov 2021