MySQL

Hevo can load data from any of your Pipelines into a MySQL database. In this document, we will walk through the steps to add MySQL as a Destination.

We do not recommend using MySQL as a Destination for building your production Pipelines. It can perform poorly even for low volumes of data. If you run into performance issues, these may be unresolvable, and you will have to migrate to a different Destination.


Step 1. Add Destination

Do one of the following:

  • After you configure the Source during Pipeline creation, click ADD DESTINATION.

  • Click DESTINATIONS in the Asset Palette, and then, in the Destinations List View, click + CREATE.


Step 2. Select Destination Type

In the Add Destination page, select MySQL.

Alternatively, use the Search Destination Type search box to search for the Destination.


Step 3. Configure MySQL Connection Settings

Specify the following settings in the Configure your MySQL Destination page:

MySQL Destination settings

  • Destination Name: A unique name for this Destination.
  • Database Host: MySQL host’s IP address or DNS. To connect to a local database, follow the steps given below.
  • Database Port: The port on which your MySQL server is listening for connections (default is 3306).
  • Database User: A user with a non-administrative role of the MySQL database.
  • Database Password: Password of the user.
  • Database Name: Name of the Destination database where data will be loaded.
  • Additional Settings:

    • Connect through SSH: Enable this option to connect to Hevo using an SSH tunnel. Read Connecting Through SSH. Else, whitelist Hevo’s IP addresses for your region.

    • Sanitize Table/Column Names?: Enable this option to remove all non-alphanumeric characters, spaces in between the table and column names and replace them with an underscore (_). Read Name Sanitization.


Step 4. Test Connection

After filling the details, click on TEST CONNECTION button to test connectivity to the Destination MySQL server.


Step 5. Save Destination

Once the test is successful, save the Destination by clicking on SAVE DESTINATION.


Connect to a Local Database

To connect to a local database, Hevo requires an internet-facing IP address. This means that the database running on your local machine must be publicly accessible for Hevo to be able to connect to it.

We recommend using the ngrok utility for enabling this. Read about ngrok at https://ngrok.com/product.

Prerequisites

  • MY-SQL/MS-SQL service is running on your local machine.

  • You have an account on ngrok and an installed ngrok utility on your local machine. To run ngrok on your local machine, follow these one-time steps:

    1. Extract the ngrok utility:

      • On Linux or MacOS, unzip ngrok from a terminal:

        unzip /path/to/ngrok.zip
        
      • On Windows, double-click ngrok.zip to extract it.

    2. Authenticate ngrok in your local machine:

      ./ngrok authtoken <your_auth_token>
      

      You can get the auth token from your ngrok dashboard. For example, in the image below, the auth_token starts with 1t7uGaJshUvnh2Fg1MApl.

      Auth token

Connecting to the Local Database

Perform the following steps to connect to the local database:

  1. Log in to your database server.

  2. Start a TCP tunnel forwarding to your database port.

    ./ngrok tcp <your_database_port>
    

    For example, the port address for MySQL is 3306. Therefore, the command would be:

    ./ngrok tcp 3306
    
  3. Copy the public IP address (hostname and port number) for your local database and port. For example, in the image below, 8.tcp.ngrok.io is the database hostname and 19789 is the port number.

    Getting hostname and port

  4. Paste the hostname and port number into the Database Host and Database Port fields respectively.

    MySQL settings

  5. Specify all other settings and click TEST & CONTINUE.


Destination Considerations

  • You must disable any foreign keys defined in the target tables. Foreign keys do not allow data to be loaded until the reference table has a corresponding key defined.
  • You can replicate data for only 4090 columns in a given MySQL table. Read Limits on the Number of Columns.

Limitations

None.


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Jul-26-2021 1.68 Added section, Connect to a Local Database.
Jul-12-2021 NA Updated the section, Destination Considerations.
Last updated on 22 Jul 2021