Throughput Capacity Exceeded for Table

Error Summary

Hevo is unable to ingest data from the Source table because the read throughput capacity configured for the table is lower than the rate at which Hevo is requesting data. In Amazon DynamoDB, each table has a defined limit on the number of read requests it can handle per second. When incoming read requests exceed this limit, DynamoDB temporarily blocks additional requests, preventing Hevo from reading data.

Potential Causes

During historical data ingestion, if a Source table has more than 1 Million records, Hevo splits the table into 10 segments and creates separate ingestion tasks for each segment. These tasks run in parallel to speed up ingestion. As a result, the number of read requests sent to DynamoDB increases. If the table’s configured read throughput capacity is insufficient to handle this increased load, DynamoDB is unable to serve the requests, causing ingestion to fail.

Suggested Action(s)

• Increase the read throughput capacity of the Source table. To do this, perform the following steps:

  1. Sign in to the AWS DynamoDB Console.

  2. In the left navigation pane, click Tables, and then select the table for which you want to increase the read throughput capacity.

     

  3. Click the Actions drop down, and then click Edit capacity.

     

  4. On the Edit read/write capacity page, in the Capacity mode section, do one of the following:

     • Select On-demand if the table experiences a high number of read requests during historical data ingestion. In this mode, DynamoDB automatically adjusts read capacity based on incoming requests.

       

     • Select Provisioned, and then under the Read capacity section, specify the required Provisioned capacity units relevant to your requirements. Read DynamoDB provisioned capacity mode to understand how to calculate the required Read Capacity Unit (RCU).

       

  5. Scroll down to the bottom and click Save changes.

  Note: You do not need to resync the object after increasing the table’s read throughput capacity. If you need to re-ingest data for the object in the future, it is recommended that you first set the table to on-demand capacity mode and then resync the object. Once the resync is complete, you can change the table’s capacity mode based on your requirements.

• Contact Hevo Support to adjust historical data ingestion behavior based on your table’s read throughput capacity. This includes modifying the number of records required to trigger parallel ingestion and changing the number of segments used to divide the table during historical data ingestion.

Revision History