Auto Mapping Event Types
On This Page
Hevo provides you an option to automatically map Event Types and fields from your Source to the corresponding tables and columns in the Destination, thereby eliminating the need for any human intervention. You can enable Auto Mapping at the Pipeline or the Event Type level. For example, you can disable Auto Mapping for the entire Pipeline but enable it for just one Event Type or vice versa.
When you disable Auto Mapping for an Event Type, its current mapping is retained but its status changes from Auto Mapped to Mapped. Any subsequent changes made to it in the Source are not automatically made in the Destination table moving forward. For information on manually editing the mapping, read Modifying Schema Mapping for Auto Mapped Event Types.
Once you configure the Source and Destination of a Pipeline and select Auto Mapping, Hevo replicates the Source schemas as is to the Destination table. It also handles any future schema changes that may occur in the Source data.
As part of Auto Mapping, Hevo:
Creates the respective tables and fields in the Destination with the appropriate schema if these are not present already.
Maps all the Source Event Types automatically to the tables in the Destination.
The following table describes how Auto Mapping handles different data scenarios:
|Sr. No.||Data Scenario||Auto-Mapping On||Auto-Mapping Off|
|1||New Event Types are added in the Source.||No user intervention is required. Hevo creates and maps the respective tables in the Destination and initiates the data replication process.||You need to create and map the respective tables manually before being able to sync the data with the Destination.|
|2||New fields are added to the existing Event Types in the Source.||No user intervention is required. Hevo creates the respective columns in the Destination.||You need to map the respective columns manually before being able to sync the data with the Destination.|
|3||Fields are deleted from the existing Event Types in the Source.||No user intervention is required. Hevo doesn’t load any data further in the respective columns.||No user intervention is required. Hevo doesn’t load any data further in the respective columns.|
|4||Fields data types are changed to compatible data types in the Source.||No user intervention is required if data type promotion is supported for the Destination, else Hevo sidelines the affected Events. In such a case, you can use Transformations, or you can create and map the table again.||Hevo sidelines the affected Events until mappings are resolved. You can use Transformations, or you can create and map the table again.|
|5||Fields data types are changed to incompatible data types in the Source.||Hevo sidelines the affected Events until mappings are resolved. You can use Transformations if possible, else you need to create and map the table again.||Hevo sidelines the affected Events until mappings are resolved. You can use Transformations if possible, else you need to create and map the table again.|
|6||Tables are deleted in the Destination.||No user intervention is required. Hevo re-creates the deleted tables and loads the Events in the Destination.||You need to re-create and map the deleted tables manuallybefore being able to sync the data with the Destination.|
|7||Columns are deleted in the Destination.||No user intervention is required. Hevo re-creates the deleted columns and loads the Events in the Destination.||You need to re-map the deleted columns manually before being able to sync the data with the Destination.|
Changes to the Source data or its mapping are reflected in the Destination only when the changed Events are next ingested. For example, suppose a Field is created in an Event Type in the Source while Auto-Mapping is on. The corresponding column is created in the Destination automatically only when Hevo ingests an Event that carries the value for the new field.
Click the VIEW SCHEMA button below the Pipeline Summary Bar to see the Source schema details such as the list of objects synced and data types of the fields pertaining to each object.
When tables are created in the Destination for the first time based on the Source schema, any primary keys defined in the Source are retained in the Destination tables during Auto Mapping. In addition, for the Amazon Redshift data warehouse, sort keys are automatically created on the primary keys defined for the respective tables. This is similar to the functionality offered in manual schema mapping.
As of Release 1.69, for existing tables in the Destination, the primary keys from the Source schema are mapped if a primary key is not already defined for the table. If the Destination table already contains duplicate Events, those are not deleted as part of this mapping; only the new Events are de-duplicated.
Note: A sort key defines the column by which data is sorted in a table. Availability of sorted data improves the query efficiency during loading of Events as number of scans required to identify the location of update are substantially reduced. Consequently, job performance is improved.
Refer to the following table for the list of key updates made to this page:
|Date||Release||Description of Change|
|Aug-09-2021||1.69||- Created a table to illustrate how Auto Mapping handles different data scenarios.
- Revised the content to highlight how Hevo handles deduplication if primary keys are not present in the Destination tables.