Mapping a Source Event Type Field with a Destination Table Column

Once you have a Destination table selected in your Mapping Summary page while creating or editing a mapping, Hevo selects the column in the Destination table that can be mapped to the Source Event Type Field if they have matching names. If you wish to map the field with a different column you can do one of the following:

  • Map with a different existing column: Click on the Destination column name dropdown and select the existing column with which you want to map the Source field.
  • Create a new column:
    1. Click in the Destination Fields field.
    2. Enter the name for the new column.

If a Source Event Type field has a null value, it is not displayed in the Schema Mapper. To display it, at least one of the following must occur:

  • There should be an Event containing a non-null value for it.
  • You must write a transformation to insert some default value in it.

In scenarios where no column with a matching name is present in the Destination table, Hevo suggests the name and the data type of the column. These columns are marked with the iconnew column. If you do not want to go with the suggestion, you can do one of the following:

  • Map with an existing column with a non-matching name:
    1. Click on the input field where the suggested name is written.
    2. Click the Cancel icon.
    3. Choose the existing field from the Select Field drop-down.
  • Choose a different column name for the new column: Type a unique name for the new column.

Limits on the Number of Columns

Some Destinations impose a hard limit on the number of columns that can be replicated to a table. For some other Destinations, Hevo may restrict the number of columns in order to optimize the data replication process. Further, Hevo reserves six columns for metadata information it may append to a table during replication. Therefore, the final number of columns that you can map and replicate is derived as:

Number of Columns = Destination Limit - 6 Hevo Metadata Columns

Destination Name Number of Columns Limit Imposed By
Amazon Aurora MySQL 4096-6 = 4090 Hevo
Amazon Redshift 1600-6 = 1594 Destination
MS SQL (all variants except Amazon Aurora MS SQL) 1024-6 = 1018 Destination
MySQL (all variants except Amazon Aurora MySQL) 4096-6 = 4090 Destination
PostgreSQL (all variants except Amazon Aurora PostgreSQL) 1600-6 = 1594 Destination
Snowflake 4096-6 = 4090 Hevo

If the Source Even Type (object) has X number of fields and the Destination table only allows Y, where Y < X number of columns, then the Events of excess columns are sidelined with the error:

Source Event Type has more than *<X-6>* fields. Please remove the fields that are not required via Transformations and reset the mappings for the Event Type from Schema Mapper.

If you have selected the Merge Tables or Merge Collections option in the Source configuration to merge tables/collections with the same name, then:

  • The columns of the table that is picked first are mapped first. Subsequently, columns from other tables are mapped. If the aggregate of the columns of all tables being merged is within the Destination’s limits, no error is displayed.

  • If the tables have columns with same names, the columns of the table that is picked first are mapped first. In case the data types of columns of subsequent tables do not match, their Events are sidelined.

  • If all tables individually have the maximum allowed columns or more, then the same error as above is displayed.

  • If the first table’s columns exceed the limit, while the next table has lesser column, still the table read first is considered first.



See Also

Schema Mapper Compatibility Table

Last updated on 25 Feb 2021