Amazon Aurora MySQL (Edge)

Prerequisites

• The Amazon Aurora MySQL instance is running.

• The MySQL version is 8.0 or higher. You can choose the MySQL version while creating the instance.

• Binary Log (BinLog) replication is enabled on the primary instance of your Amazon Aurora MySQL database server.

  Note: Amazon Aurora MySQL does not support BinLog replication on read replicas.

• Hevo’s IP address(es) for your region is added to the Amazon Aurora MySQL database IP Allowlist.

• The SELECT, RELOAD, SHOW DATABASES, and REPLICATION privileges are granted to the database user.

  Note: We recommend that you create a database user for configuring your Amazon Aurora MySQL Source in Hevo. However, if you already have one, refer to section Grant privileges to the user.

• The database hostname and port number of the Source instance are available.

Perform the following steps to configure your Amazon Aurora MySQL Source:

Set up MySQL Binary Logs for Replication

A binary log is a collection of log files that records information about data modifications and data object modifications made on a MySQL server instance. Typically binary logs are used for data replication and data recovery.

Hevo supports data ingestion for replication from MySQL servers via binary logs (BinLog). For this, binary logging must be enabled on your MySQL instance.

Perform the following steps to enable binary logging for your Amazon Aurora MySQL server:

1. Configure the parameter group

1. Log in to the Amazon RDS console.

2. In the left navigation pane, click Databases.

3. In the Databases section on the right, click the DB identifier of your Amazon Aurora database instance.

   

4. Click the Configuration tab, and then click the link text under DB cluster parameter group.

   Note: If you are using the default Aurora DB cluster parameter group, you need to create one with the Type as DB cluster parameter group, as the default one is not editable.

   

5. Click Edit.

   

6. On the page that appears, search and update the value of the following parameters:

   

   Parameter Name	Value
   binlog_format	ROW
   binlog_row_image	full
   gtid-mode	ON
   enforce_gtid_consistency	ON

   Note:

   • Enabling GTIDs (Global Transaction Identifiers) is recommended because it simplifies replication by uniquely identifying transactions, making it easier to track them.

   • Ensure that the value of binlog_row_value_options parameter is not set to PARTIAL_JSON.

7. Click Save Changes.

8. Reboot the database instance that you are using to connect to Hevo, to apply the above changes.

   To do this:

   1. In the left navigation pane, under Dashboard, click Databases.

   2. In the Databases section on the right, select the DB identifier of your Amazon Aurora MySQL database instance.

   3. In the Actions drop-down, click Reboot.

      Note: If you reboot a database instance with the Writer instance role in the DB cluster, all the remaining reader instances of that database in the cluster are rebooted as well.

      

   4. On the Reboot page, click Confirm to reboot your DB instance.

      

The replication reference guide on MySQL’s documentation portal provides a complete reference of the options available for replication and binary logging.

2. Configure the BinLog retention period

1. Log in to your Amazon Aurora MySQL database instance with ADMIN privileges.

2. Run the following command to view the current BinLog retention period (in hours):

   call mysql.rds_show_configuration;
   

3. If the BinLog retention period is less than 72 hours, run the following command to set it to at least 72 hours (three days).

   call mysql.rds_set_configuration('binlog retention hours', 72);
   

Allowlist Hevo IP addresses for your region

You need to allowlist the Hevo IP address for your region to enable Hevo to connect to your Amazon Aurora MySQL database. To do this:

1. Log in to the Amazon RDS console.

2. In the left navigation pane, click Databases.

3. In the Databases section on the right, click the DB identifier of your Amazon Aurora database instance.

   

4. In the Connectivity & security tab, click the link text under Security, VPC security groups.

   

5. On the Security Groups page, select the check box for your Security group ID, and from the Actions drop-down, click Edit inbound rules.

   

6. On the Edit inbound rules page:

   

   1. Click Add rule.

   2. In the Type column, select MySQL/Aurora from the drop-down.

   3. In the Source column, select Custom from the drop-down and enter Hevo’s IP addresses for your region. Repeat steps 1-3 to allowlist all the IP addresses.

   4. Click Save rules.

Create a Database User and Grant Privileges

1. Create a database user (Optional)

2. Grant privileges to the user

The following table lists the privileges that the database user for Hevo requires to connect to and ingest data from your MySQL database:

Connect to your Amazon Aurora MySQL database as a root user with an SQL client tool, such as MySQL Workbench, and run the following script. These commands grant only the necessary privileges required by Hevo to ingest data from your Amazon Aurora MySQL database.

# Grant Privileges to the Database User
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO <database_username>@'%';
GRANT EXECUTE ON PROCEDURE mysql.rds_show_configuration TO '<database_username>'@'<hostname>';

# Grant Hevo access to the Database
GRANT SELECT ON <database_name>.* TO <database_username>;

# Finalize the User’s Permissions
FLUSH PRIVILEGES;

Note:

• The SELECT, RELOAD, and SHOW DATABASES privileges are required only for the historical load.

• Replace the placeholder values in the commands above with your own. For example, <database_username> with hevo.

Retrieve the Database Hostname and Port Number (Optional)