- Introduction
- Getting Started
- Creating an Account in Hevo
- Subscribing to Hevo via AWS Marketplace
-
Connection Options
- Connecting Through SSH
- Connecting Through Reverse SSH Tunnel
- Connecting Through VPN
- Connecting Through Mongo PrivateLink
- Connecting Through AWS Transit Gateway
- Connecting Through AWS VPC Endpoint
- Connecting Through AWS VPC Peering
- Using Google Account Authentication
- How Hevo Authenticates Sources and Destinations using OAuth
- Reauthorizing an OAuth Account
- Familiarizing with the UI
- Creating your First Pipeline
- Data Loss Prevention and Recovery
- Data Ingestion
- Data Loading
- Loading Data in a Database Destination
- Loading Data to a Data Warehouse
- Optimizing Data Loading for a Destination Warehouse
- Deduplicating Data in a Data Warehouse Destination
- Manually Triggering the Loading of Events
- Scheduling Data Load for a Destination
- Loading Events in Batches
- Data Loading Statuses
- Data Spike Alerts
- Name Sanitization
- Table and Column Name Compression
- Parsing Nested JSON Fields in Events
- Pipelines
- Data Flow in a Pipeline
- Familiarizing with the Pipelines UI
- Working with Pipelines
- Managing Objects in Pipelines
- Pipeline Jobs
-
Transformations
-
Python Code-Based Transformations
- Supported Python Modules and Functions
-
Transformation Methods in the Event Class
- Create an Event
- Retrieve the Event Name
- Rename an Event
- Retrieve the Properties of an Event
- Modify the Properties for an Event
- Fetch the Primary Keys of an Event
- Modify the Primary Keys of an Event
- Fetch the Data Type of a Field
- Check if the Field is a String
- Check if the Field is a Number
- Check if the Field is Boolean
- Check if the Field is a Date
- Check if the Field is a Time Value
- Check if the Field is a Timestamp
-
TimeUtils
- Convert Date String to Required Format
- Convert Date to Required Format
- Convert Datetime String to Required Format
- Convert Epoch Time to a Date
- Convert Epoch Time to a Datetime
- Convert Epoch to Required Format
- Convert Epoch to a Time
- Get Time Difference
- Parse Date String to Date
- Parse Date String to Datetime Format
- Parse Date String to Time
- Utils
- Examples of Python Code-based Transformations
-
Drag and Drop Transformations
- Special Keywords
-
Transformation Blocks and Properties
- Add a Field
- Change Datetime Field Values
- Change Field Values
- Drop Events
- Drop Fields
- Find & Replace
- Flatten JSON
- Format Date to String
- Format Number to String
- Hash Fields
- If-Else
- Mask Fields
- Modify Text Casing
- Parse Date from String
- Parse JSON from String
- Parse Number from String
- Rename Events
- Rename Fields
- Round-off Decimal Fields
- Split Fields
- Examples of Drag and Drop Transformations
- Effect of Transformations on the Destination Table Structure
- Transformation Reference
- Transformation FAQs
-
Python Code-Based Transformations
-
Schema Mapper
- Using Schema Mapper
- Mapping Statuses
- Auto Mapping Event Types
- Manually Mapping Event Types
- Modifying Schema Mapping for Event Types
- Schema Mapper Actions
- Fixing Unmapped Fields
- Resolving Incompatible Schema Mappings
- Resizing String Columns in the Destination
- Schema Mapper Compatibility Table
- Limits on the Number of Destination Columns
- File Log
- Troubleshooting Failed Events in a Pipeline
- Mismatch in Events Count in Source and Destination
- Audit Tables
- Activity Log
-
Pipeline FAQs
- Can multiple Sources connect to one Destination?
- What happens if I re-create a deleted Pipeline?
- Why is there a delay in my Pipeline?
- Can I change the Destination post-Pipeline creation?
- Why is my billable Events high with Delta Timestamp mode?
- Can I drop multiple Destination tables in a Pipeline at once?
- How does Run Now affect scheduled ingestion frequency?
- Will pausing some objects increase the ingestion speed?
- Can I see the historical load progress?
- Why is my Historical Load Progress still at 0%?
- Why is historical data not getting ingested?
- How do I set a field as a primary key?
- How do I ensure that records are loaded only once?
- Events Usage
- Sources
- Free Sources
-
Databases and File Systems
- Data Warehouses
-
Databases
- Connecting to a Local Database
- Amazon DocumentDB
- Amazon DynamoDB
- Elasticsearch
-
MongoDB
- Generic MongoDB
- MongoDB Atlas
- Support for Multiple Data Types for the _id Field
- Example - Merge Collections Feature
-
Troubleshooting MongoDB
-
Errors During Pipeline Creation
- Error 1001 - Incorrect credentials
- Error 1005 - Connection timeout
- Error 1006 - Invalid database hostname
- Error 1007 - SSH connection failed
- Error 1008 - Database unreachable
- Error 1011 - Insufficient access
- Error 1028 - Primary/Master host needed for OpLog
- Error 1029 - Version not supported for Change Streams
- SSL 1009 - SSL Connection Failure
- Troubleshooting MongoDB Change Streams Connection
- Troubleshooting MongoDB OpLog Connection
-
Errors During Pipeline Creation
- SQL Server
-
MySQL
- Amazon Aurora MySQL
- Amazon RDS MySQL
- Azure MySQL
- Generic MySQL
- Google Cloud MySQL
- MariaDB MySQL
-
Troubleshooting MySQL
-
Errors During Pipeline Creation
- Error 1003 - Connection to host failed
- Error 1006 - Connection to host failed
- Error 1007 - SSH connection failed
- Error 1011 - Access denied
- Error 1012 - Replication access denied
- Error 1017 - Connection to host failed
- Error 1026 - Failed to connect to database
- Error 1027 - Unsupported BinLog format
- Failed to determine binlog filename/position
- Schema 'xyz' is not tracked via bin logs
- Errors Post-Pipeline Creation
-
Errors During Pipeline Creation
- MySQL FAQs
- Oracle
-
PostgreSQL
- Amazon Aurora PostgreSQL
- Amazon RDS PostgreSQL
- Azure PostgreSQL
- Generic PostgreSQL
- Google Cloud PostgreSQL
- Heroku PostgreSQL
-
Troubleshooting PostgreSQL
-
Errors during Pipeline creation
- Error 1003 - Authentication failure
- Error 1006 - Connection settings errors
- Error 1011 - Access role issue for logical replication
- Error 1012 - Access role issue for logical replication
- Error 1014 - Database does not exist
- Error 1017 - Connection settings errors
- Error 1023 - No pg_hba.conf entry
- Error 1024 - Number of requested standby connections
- Errors Post-Pipeline Creation
-
Errors during Pipeline creation
- PostgreSQL FAQs
- Troubleshooting Database Sources
- File Storage
- Engineering Analytics
- Finance & Accounting Analytics
-
Marketing Analytics
- ActiveCampaign
- AdRoll
- Amazon Ads
- Apple Search Ads
- AppsFlyer
- CleverTap
- Criteo
- Drip
- Facebook Ads
- Facebook Page Insights
- Firebase Analytics
- Freshsales
- Google Ads
- Google Analytics
- Google Analytics 4
- Google Analytics 360
- Google Play Console
- Google Search Console
- HubSpot
- Instagram Business
- Klaviyo v2
- Lemlist
- LinkedIn Ads
- Mailchimp
- Mailshake
- Marketo
- Microsoft Ads
- Onfleet
- Outbrain
- Pardot
- Pinterest Ads
- Pipedrive
- Recharge
- Segment
- SendGrid Webhook
- SendGrid
- Salesforce Marketing Cloud
- Snapchat Ads
- SurveyMonkey
- Taboola
- TikTok Ads
- Twitter Ads
- Typeform
- YouTube Analytics
- Product Analytics
- Sales & Support Analytics
- Source FAQs
- Destinations
- Familiarizing with the Destinations UI
- Cloud Storage-Based
- Databases
-
Data Warehouses
- Amazon Redshift
- Amazon Redshift Serverless
- Azure Synapse Analytics
- Databricks
- Firebolt
- Google BigQuery
- Hevo Managed Google BigQuery
- Snowflake
-
Destination FAQs
- Can I change the primary key in my Destination table?
- How do I change the data type of table columns?
- Can I change the Destination table name after creating the Pipeline?
- How can I change or delete the Destination table prefix?
- Why does my Destination have deleted Source records?
- How do I filter deleted Events from the Destination?
- Does a data load regenerate deleted Hevo metadata columns?
- How do I filter out specific fields before loading data?
- Transform
- Alerts
- Account Management
- Activate
- Glossary
Releases- Release 2.31 (Nov 18-Dec 16, 2024)
- Release 2.30 (Oct 21-Nov 18, 2024)
-
2024 Releases
- Release 2.29 (Sep 30-Oct 22, 2024)
- Release 2.28 (Sep 02-30, 2024)
- Release 2.27 (Aug 05-Sep 02, 2024)
- Release 2.26 (Jul 08-Aug 05, 2024)
- Release 2.25 (Jun 10-Jul 08, 2024)
- Release 2.24 (May 06-Jun 10, 2024)
- Release 2.23 (Apr 08-May 06, 2024)
- Release 2.22 (Mar 11-Apr 08, 2024)
- Release 2.21 (Feb 12-Mar 11, 2024)
- Release 2.20 (Jan 15-Feb 12, 2024)
-
2023 Releases
- Release 2.19 (Dec 04, 2023-Jan 15, 2024)
- Release Version 2.18
- Release Version 2.17
- Release Version 2.16 (with breaking changes)
- Release Version 2.15 (with breaking changes)
- Release Version 2.14
- Release Version 2.13
- Release Version 2.12
- Release Version 2.11
- Release Version 2.10
- Release Version 2.09
- Release Version 2.08
- Release Version 2.07
- Release Version 2.06
-
2022 Releases
- Release Version 2.05
- Release Version 2.04
- Release Version 2.03
- Release Version 2.02
- Release Version 2.01
- Release Version 2.00
- Release Version 1.99
- Release Version 1.98
- Release Version 1.97
- Release Version 1.96
- Release Version 1.95
- Release Version 1.93 & 1.94
- Release Version 1.92
- Release Version 1.91
- Release Version 1.90
- Release Version 1.89
- Release Version 1.88
- Release Version 1.87
- Release Version 1.86
- Release Version 1.84 & 1.85
- Release Version 1.83
- Release Version 1.82
- Release Version 1.81
- Release Version 1.80 (Jan-24-2022)
- Release Version 1.79 (Jan-03-2022)
-
2021 Releases
- Release Version 1.78 (Dec-20-2021)
- Release Version 1.77 (Dec-06-2021)
- Release Version 1.76 (Nov-22-2021)
- Release Version 1.75 (Nov-09-2021)
- Release Version 1.74 (Oct-25-2021)
- Release Version 1.73 (Oct-04-2021)
- Release Version 1.72 (Sep-20-2021)
- Release Version 1.71 (Sep-09-2021)
- Release Version 1.70 (Aug-23-2021)
- Release Version 1.69 (Aug-09-2021)
- Release Version 1.68 (Jul-26-2021)
- Release Version 1.67 (Jul-12-2021)
- Release Version 1.66 (Jun-28-2021)
- Release Version 1.65 (Jun-14-2021)
- Release Version 1.64 (Jun-01-2021)
- Release Version 1.63 (May-19-2021)
- Release Version 1.62 (May-05-2021)
- Release Version 1.61 (Apr-20-2021)
- Release Version 1.60 (Apr-06-2021)
- Release Version 1.59 (Mar-23-2021)
- Release Version 1.58 (Mar-09-2021)
- Release Version 1.57 (Feb-22-2021)
- Release Version 1.56 (Feb-09-2021)
- Release Version 1.55 (Jan-25-2021)
- Release Version 1.54 (Jan-12-2021)
-
2020 Releases
- Release Version 1.53 (Dec-22-2020)
- Release Version 1.52 (Dec-03-2020)
- Release Version 1.51 (Nov-10-2020)
- Release Version 1.50 (Oct-19-2020)
- Release Version 1.49 (Sep-28-2020)
- Release Version 1.48 (Sep-01-2020)
- Release Version 1.47 (Aug-06-2020)
- Release Version 1.46 (Jul-21-2020)
- Release Version 1.45 (Jul-02-2020)
- Release Version 1.44 (Jun-11-2020)
- Release Version 1.43 (May-15-2020)
- Release Version 1.42 (Apr-30-2020)
- Release Version 1.41 (Apr-2020)
- Release Version 1.40 (Mar-2020)
- Release Version 1.39 (Feb-2020)
- Release Version 1.38 (Jan-2020)
- Early Access New
- Upcoming Features
Facebook Ads
Facebook Ads allows marketers to retrieve statistics about their ads, ad sets, and campaigns running on Facebook.
Hevo uses the Facebook Marketing API to replicate your Facebook Ads data into the desired Destination database or data warehouses for scalable analysis. For this, you must authorize Hevo to access data from your Facebook Ads account.
Note: From Release 2.25, Hevo has upgraded to v19.0 of the Marketing API. You may need to reauthorize Hevo for existing Pipelines. Refer to section, Configuring Facebook Ads as a Source for the authorization steps.
You can also ingest data from your Instagram Ads account using the Facebook Ads connector. Instagram Ads follows the same schema as the Facebook Ads connector and offers the same data replication settings.
Prerequisites
-
An active Facebook (FB) account from which data is to be ingested exists.
-
You have access to the Facebook Ads console and related statistics.
Note: Facebook’s authentication system uses pop-ups that may encounter issues if ad blockers are not disabled during the setup. Read Authorization.
-
You are assigned the Team Administrator, Team Collaborator, or Pipeline Administrator role in Hevo to create the Pipeline.
Configuring Facebook Ads as a Source
Perform the following steps to configure Facebook Ads as the Source in your Pipeline:
-
Click PIPELINES in the Navigation Bar.
-
Click + CREATE PIPELINE in the Pipelines List View.
-
On the Select Source Type page, select Facebook Ads.
-
On the Configure your Facebook Ads account page, do one of the following:
-
Select a previously configured account and click CONTINUE.
-
Click Add Facebook Ads Account and perform the following steps to configure an account:
-
Log in to your Facebook account, and click Continue as <Company Name> in the pop-up dialog.
-
Click Save to authorize Hevo to access your Facebook Ads and related statistics.
-
Click Got it in the confirmation dialog.
-
Note: If you enable Advanced Protection, such as two-factor authentication or password changes, you will need to reauthorize Hevo; otherwise, the Pipeline will be marked as failed.
-
-
On the Configure your Facebook Ads Source page, specify the following:
-
Pipeline Name: A unique name for your Pipeline, not exceeding 255 characters.
-
Select Ad accounts: The Facebook Ads account(s) from where you want to replicate the data. One Facebook account can contain multiple Facebook Ads accounts.
-
Report Type: Select one of the following report types to ingest data from your Facebook Ads:
-
Predefined Reports: Hevo automatically selects all the reports and their respective fields and columns for ingestion.
-
Custom Reports: Hevo allows you to manually select the aggregation level, the aggregation time, and the fields for the Facebook Ads report that you want to replicate. Refer to the section, Custom Reports to know how to configure them.
-
-
Ad Action Report Time: The reported time of an action by a Facebook user. The available options are:
-
Impression: The time at which your ad was watched.
-
Conversion: The time when an action was taken after watching your ad.
-
Mixed (Default): The time when an impression or link click occurred on your ad and the time when a conversion action, such as a purchase on your website, occurred for it.
-
-
Historical Sync Duration: The duration for which you want to ingest the existing data from the Source. Default duration: 1 Year.
-
Attribution Setting
-
Custom Attribution Window Setting: You can specify either one or both of the following:
-
Click Attribution Window: The number of days between a person clicking your ad and taking an action such as install or subscribe. Default value: 7d_click (7 days).
-
View Attribution Window: The number of days between a person viewing your ad and taking an action such as install or subscribe. Default value: 1d_view (1 day).
-
-
Use Account Attribution Setting: The attribution setting that you have defined in your Facebook Ads account.
-
Use Unified Attribution Setting: The attribution settings that you have defined for the individual ad sets.
-
-
Advanced Settings
-
Thumbnail Height: The rendered height of the thumbnails provided in the thumbnail URL. Default value: 64.
-
Thumbnail Width: The rendered width of the thumbnails provided in the thumbnail URL: Default value: 64.
Note: The specified height and width values must be within the range 1-65000.
-
-
-
Click TEST & CONTINUE.
-
Proceed to configuring the data ingestion and setting up the Destination.
Custom Reports
With custom reports, you can manually select the parameters such as the aggregation settings, breakdown settings, and fields for the Facebook Ads reports that you want to replicate.
Note: The data included in the custom report is based on the set of parameters you select.
To configure custom reports, specify the following Custom Report Settings:
-
Select Fields: The fields that you want to include in the report such as Ad name or Campaign ID.
-
Select Breakdowns: The filters to further narrow down the results of your retrieved reports. For example, you can break down the results by Date or Age Group.
-
Aggregation Window: The time interval for which the report data is aggregated. This category shows you the trend and performance based on 1 Day, 1 Week, 1 Month, or 90 days period. Default value: 1 Day. For example, if the aggregation time is 1 Month, the report contains data for the past one month from the time the Pipeline runs.
-
Aggregation Level: The primary filter on which the report data is aggregated, such as the Ad ID or the Account ID. Default value: Ad ID.
Data Replication
For Teams Created | Default Ingestion Frequency | Minimum Ingestion Frequency | Maximum Ingestion Frequency | Custom Frequency Range (in Hrs) |
---|---|---|---|---|
Before Release 2.21 | 12 Hrs | 15 Mins | 24 Hrs | 1-24 |
After Release 2.21 | 12 Hrs | 30 Mins | 24 Hrs | 1-24 |
For predefined reports
-
Historical Data: The first run of the Pipeline ingests historical data for the selected reports on the basis of the historical sync duration specified at the time of creating the Pipeline and loads it to the Destination. Default duration: 1 Year.
-
Incremental Data: Once the historical data ingestion is complete, every subsequent run of the Pipeline fetches new and updated data for the reports as per the ingestion frequency.
-
Data Refresh: Hevo creates a replication task for Facebook Ads reports. It also creates a data refresher task per report type to fetch the attributing results of an older date. By default, 30 days old data is fetched once per day.
For custom reports
-
Historical Data: In the first run of the Pipeline, Hevo ingests historical data for the reports on the basis of the historical sync duration selected at the time of creating the Pipeline and loads it to the Destination. Default duration: 1 Year.
-
Incremental Data: Once the historical data ingestion is complete, every subsequent run of the Pipeline fetches new and updated data for the reports as per the ingestion frequency.
-
Data Refresh: Hevo resyncs the data of the past 30 days.
Schema and Primary Keys
Hevo uses the following schema to upload the predefined reports’ data in the Destination:
Note:
-
The schema for custom reports is derived dynamically based on the settings you provide.
-
From Release 1.87 onwards, with JDBC-based Destinations, Hevo creates the Destination schema for predefined reports based on the data retrieved from Facebook Ads. This ensures that the limit on the number of Destination columns is not exceeded, and only fields that contain data are created in the Destination tables. However, this could lead to some Source fields being mapped to incompatible Destination columns. Read Resolving Incompatible Schema Mappings.
Data Model
The following is the list of tables (objects) that are created at the Destination when you run the Pipeline for predefined reports:
Note: For custom reports, the data model is derived dynamically based on the custom report settings you provide.
Object | Primary Key | Description |
---|---|---|
Ads | - ad_id |
Contains information about the ads in your Facebook Ads account. |
Ads Insights | - ad_id - date_start - date_stop |
Contains detailed statistics for each campaign, set, ad combination per day. |
Ads Insights Age and Gender | - account_id - ad_id - age - date_start - date_stop - gender |
Contains detailed statistics for each campaign, set, ad combination per day, sorted by age and gender. |
Ads Insights Country | - account_id - ad_id - date_start - date_stop - country |
Contains detailed statistics for each campaign, set, ad combination per day, sorted by country. |
Ads Insights Region | - account_id - ad_id - date_start - date_stop - region |
Contains detailed statistics for each campaign, set, ad combination per day, sorted by region. |
Ads Insights DMA | - ad_id - date_start - date_stop - dma |
Contains detailed statistics for each campaign, set, ad combination per day, sorted by DMA. |
Ads Insights Platform and Device | - ad_id - date_start - date_stop - impression_device - publisher_platform |
Contains daily statistics for each advertisement grouped by platform, and device details. |
Ad Creative | - id |
Defines the creative fields of one or more ads. |
Ad Sets | - id |
Is a group of ads that share the same daily or lifetime budget, schedule, bid type, bid info, and targeting data. Ad sets enable you to group ads according to criteria, and you can retrieve the ad-related statistics that apply to a set. |
Campaigns | - id |
Represent the objective for an advertiser. For example, to drive page post engagement. |
Additional Information
Read the detailed Hevo documentation for the following related topics:
Source Considerations
- Facebook Ads allows fetching data up to the past three years. Hence, you can change the offset using the Change Position action only within the past three years; beyond this, the API fails.
Limitations
- This integration does not support replicating data for reviews and pages. Hevo has a separate integration for Facebook Page Insights that can be used to fetch page and post insights.
See Also
Revision History
Refer to the following table for the list of key updates made to this page:
Date | Release | Description of Change |
---|---|---|
Nov-05-2024 | NA | Added a note in the Configuring Facebook Ads as a Source section to indicate that enabling Advanced Protection requires reauthorizing Hevo for existing Pipelines. |
Sep-30-2024 | 2.28.1 | Updated section, Data Replication to change the default ingestion frequency to 12 Hrs. |
Jul-01-2024 | 2.25 | Updated the note in the page overview to Marketing API v19.0. |
Jun-17-2024 | 2.24.2 | Updated the Configuring Facebook Ads as a Source to add Advanced Settings for custom thumbnail size. |
Mar-05-2024 | 2.21 | Updated the ingestion frequency table in the Data Replication section. |
Jan-29-2024 | 2.19.3 | Added a note in the page overview about the Marketing API v18.0 update. |
Oct-03-2023 | NA | Updated the section, Configuring Facebook Ads as a Source as per the latest Facebook UI. |
Mar-23-2023 | 2.10 | Updated section, Configuring Facebook Ads as a Source and Custom Reports to reflect the latest UI. |
Sep-05-2022 | NA | Updated section, Data Replication to reorganize the content for better understanding and coherence. |
Aug-24-2022 | NA | Updated the Configuring Facebook Ads as a Source section to reflect the latest UI changes. |
Jun-21-2022 | 1.91 | Updated the Configuring Facebook Ads as a Source and Custom Reports sections to reflect the UI changes. |
May-10-2022 | NA | Updated the historical data sync information for predefined reports in the Data Replication section. |
Apr-25-2022 | 1.87 | Updated the note in the Schema and Primary Keys section to state that the Destination schema is derived from the Source data for Pipelines with JDBC Destinations. |
Apr-21-2022 | NA | Added a note in the page overview about the option to replicate Instagram Ads data using the Facebook Ads connector. |
Mar-21-2022 | NA | Reorganized the section, Custom Reports for better understanding and coherence. |
Oct-25-2021 | NA | Added the Pipeline frequency information in the Data Replication section. |
Sep-20-2021 | 1.72 | Added a Source Consideration whereby users can change the offset to ingest data only up to the past three years. |
Sep-09-2021 | 1.71 | Updated the section, Data Model to reflect the correct field names for the Ads Insights Platform and Device object. |
Jul-22-2021 | NA | Updated the Data Model to include the primary key information. |
May-19-2021 | 1.63 | - Updated the content to reflect the latest UI and functionality. - Added the section, Custom Reports. |