Mapping a Source Event Type with a Destination Table

Manually mapping a Source Event Type to a Destination table allows you to:

  • Modify the mapping between Source and Destination fields.

  • De-select fields of an Event Type so that these are not loaded to the Destination.

  • Change the Destination table the Event Type is mapped to.

  • Delete an existing Destination table.

  • Create a new Destination table and mappings.

Refer to the sections below for steps to perform these actions.

If you had previously enabled Auto-Mapping and now disable it, the existing mapping is not affected and the Event Type still carries the status MAPPED. You must specifically edit the mapping to apply any changes.

Event remains mapped even after Auto-Mapping is disabled

Prerequisites

  • Auto-mapping is disabled for the Event Type to be mapped.

Accessing the Mapping Summary Page

You can create mappings or edit existing mappings in the Mapping Summary page.

To access it:

  1. In the Pipelines toolbar, click the Schema Mapper icon.

  2. In the Schema Mapper overview page, do one of the following to access the Mapping Summary:

    • Click on any Event Type row.

    • Click on the Event Type you want to map.

  3. In the Mapping Summary page, make sure the Event Type you want to map is selected. Else, click on it and select it.

    For example, in the following image, the departments Event Type is selected and you can click EDIT MAPPING to edit its mapping:

    Select an Event Type for editing its mapping

Modifying Existing Field Mappings

  1. Click EDIT MAPPING in the Mapping Summary page.

  2. Click the drop-down next to the Destination table field name.

    1. Select the new field to map to.

      Change field mapping

      If you select a field with an incompatible data type, Schema Mapper generates a warning icon.

      Warning icon for incompatible fields

    2. Deselect the checkbox next to a field name if it should not be loaded to the Destination.

      Deselect fields to not load them into Destination

    3. Click APPLY CHANGES.
      If incompatible fields have been mapped, a warning message is displayed. Click CONFIRM to proceed.

      Confirm incompatible field mappings

      Any fields you had deselected are marked as Ignored.

      Fields not selected are ignored

Changing the Destination table

You can select a different Destination table to map the Source Event to and edit the mapping of the fields.

To change the Destination table:

  1. Do one of the following:

    • In the MAPPING SUMMARY pane, click the Kebab menu icon next to the Destination Table name. Click Change Destination Table.

      Change Destination table from mapping summary page

    • Click EDIT MAPPING in the Mapping Summary page. Then, click the Destination Table drop-down.

      Change Destination table from edit mapping summary page

  2. Select the new table from the Destination Table drop-down.

  3. Map the Source Fields to the Destination Fields. Any items needing your attention are highlighted on screen.

  4. Take the appropriate actions, as required, and click APPLY CHANGES.

    Apply changes to map a different table

    Note: You can only map fields with compatible data types. Else, a warning icon is displayed. Further, only data types that can best accommodate the Source and Destination field types are available for selection.

Mapping an Event Type to a New Destination Table

If existing Destination tables do not meet your requirements. or you want to define cluster keys in case of Google BigQuery Destination, you can create a fresh table in the Mapping Summary page.

To do this:

  1. Click Create Table & Map in the Destination table drop-down.

    Create a table

    Hevo creates a table with fields that match the Source fields. It then creates the default mapping and assigns names to the Destination fields based on any previous mappings that existed or the Source field names, in that order.

    Create table and mappings

    You can choose to retain the default settings or change these using the following steps:

  2. Specify a unique Destination Table Name.

    Note: Hevo validates the length of the table and column names that you create in the Destination to ensure these are within the character limits set by the Destination.

  3. Select the Partition Style. This is applicable only for the Google BigQuery data warehouse. The partition style can be:

    • No Partition: Tables are not partitioned.

    • Ingestion Time Based: Tables are partitioned based on the data ingestion (load) date or when these are written to Google BigQuery. Read more here.

    • Field Based: Tables that are partitioned based on a TIMESTAMP or DATE column.

  4. For each Source field, specify the corresponding Destination Field Name.

  5. In the Data Type drop-down, select the data type for the Destination field. Only the data types compatible with the Source field are available for selection.

    Note: You cannot define fields with Array and Struct data types as cluster keys in BigQuery Destination tables.

  6. Optionally, use the Up/Down arrow next to a Source Field name to move it up or down in the list.

  7. Optionally, select the check box next to a field to include it in the mapping and load data for it to the Destination, if it was previously ignored.

  8. Optionally, in the Primary Key column, select the check box for the field you want to mark as a primary key.

  9. Optionally, in the Partition Key column, select the field you want to mark as a partition key. This partitions the data of the selected field. Partitioned tables help you run more efficient queries by reducing the amount of data read in each query. Read Partitioning in BigQuery.

  10. Optionally, in the Cluster Key column, select the fields you want to mark as cluster keys. This is applicable only for the Google BigQuery data warehouse Destinations. Read Clustering in BigQuery.

    Note: You can select a maximum of four fields as cluster keys.

  11. Optionally, click the DELETE icon for a row if you do not want to add that field to the Destination table.

  12. Click CREATE TABLE & MAP.

  13. Click CONFIRM in the dialog box that appears. The new table is created and mapped to the Source Event Type.

    New destination table is created

Read about Mapping a Source Event Type Field with a Destination Table Column.

Deleting a Destination Table

Deleting a Destination table is not specifically required for modifying the schema mapping of an Event Type. However, you can use Schema Mapper to view the Destination tables and perform any cleanup, such as deleting a table. When you delete a table, the associated mappings are also discarded.

To delete a table:

  1. Do one of the following in the Mapping Summary page:

    • Click the Kebab menu next to the Destination Table name. Then, click Drop Table.

      Drop Destination table

    • Click EDIT MAPPING. Search for the table you want to delete and click the Delete icon next to it.

      Delete Destination table

  2. Click YES, I AM SURE in the confirmation dialog that is displayed, to delete the table.
    Note: Deleting a table is a permanent action and cannot be undone. All the data loaded to the table from any Source is also deleted and cannot be restored. Therefore, you must carefully consider your decision to delete a table.

    Table delete warning


See Also


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Apr-06-2021 1.60 Updated the section Mapping an Event Type to a New Destination Table to include steps to enable clustering and partitioning for Google BigQuery Destinations.
Mar-23-2021 1.59 - Added the section Deleting a Destination Table.
- Added the section Mapping an Event Type with a New Destination Table.
- Updated the sections to reflect the detailed steps and latest UI.
Last updated on 01 Jun 2021