Transformations
ON THIS PAGE
The data received from the Source system might not be suitable to be replicated to the Destination database or data warehouse, as is. Using Transformations in Hevo, you can prepare the data in multiple ways before loading it to the Destination.
These are the common scenarios where transformation may be required or helpful:
- Cleansing: Your data may contain unclean values or values in inconsistent formats. You may want to convert all booleans to 0s and 1s or trim special characters or spaces from values. All these operations can be done in transformations.
- Data Enrichment: Data can be enriched from static sources. For example, IP addresses can be converted to cities or zip codes, coordinates can be converted to geo-hashes etc. You may also enrich your data from your business specific static data-sets.
- Re-expression: You can re-express your data into different units or formats, for example, converting all currency values to USD or changing all dates to a particular time zone.
- Filtering: You can filter events that you don’t want to replicate to your warehouse based on complex logic.
- Normalization: Have nested event structure coming from a NoSQL Source like MongoDB? You can create normalized schemas from a single event through transformations.
Transformation Methods
You can design and apply transformations on Events and Event Types in two ways:
To change the transformation method, click the 
icon in the Transformation canvas.
Note: You cannot use both methods in the same transformation. Also, transformation of one type cannot be converted to the other automatically. If you switch from one method to the other, you need to rewrite the transformation completely.
Testing the Transformation
Once you write/modify your transformations, you can test these against real sample Events received by Hevo from your Source. You can also test them against Events from a specific Pipeline or a specific Event Type. You can even modify the sample Events to test the transformation for any edge case scenario.
Deploying the Transformation
Once you have tested the transformation, click DEPLOY. The transformation is applied to all incoming Events.
You can click RESET to return to the last deployed stage.
We would advise you to monitor the Failed Events for any failures due to the newly deployed code.
Transformation Version History
Click the icon to view the history of all deployed transformations.
You can restore any past deployed transformation by selecting it and clicking RESTORE THIS VERSION. The version is restored in Draft state. You must click DEPLOY to finally deploy it.
It is advised that you test the restored transformation before you deploy it. The system warns you if you try to directly deploy it.
- Articles in this section