Hevo can load data from any of your sources into the Data Lake. In this document, we will walk through the steps to add Hevo Data Lake as a Destination.
- Add Destination
A destination can either be added while creating a pipeline or by directly heading to Destinations option under the Admin tab on the left app bar and clicking Add Destination button.
- Select Destination type
Click on Select Destination Type drop-down and select Datalake.
Fill in the Connection Settings
- Destination Name: A unique name for the destination.
- Database Name: The database where all the tables will be, if it doesn’t exist, it will be created for you.
- Bucket Name: Since we're using S3 as the data store, this bucket denotes S3 Bucket name you want to dump the data to.
- Prefix: A location prefix where you want your data to be in.
- File Format: Select one of the file formats appropriate to your use case.
Save DestinationYou can test the connection. Hevo tries to create and delete a dummy table with name ‘dummy_table’. You’ll know if it encounters a failure. Click on Continue to save the destination.
Note: You’ll get a thrift URI which you can use to plug any external query engine like Presto or Apache Impala.