Configuration Changes in Elasticsearch

Last updated on Mar 03, 2025
On This Page

AWS Elasticsearch uses circuit breakers to prevent its nodes from running out of JAva Virtual Machine (JVM) memory. If it estimates that an ongoing operation will exceed the set circuit breaker limits, it stops the operation and returns an error. Hevo catches some of these errors and recommends corrective actions. Refer to this document for a list of these actions and the recommended changes to your Elasticsearch settings.

Based on the type of change, use one of the following ways to update your AWS Elasticsearch configuration settings:

  • Modify the settings on a running cluster in your Generic Elasticsearch instance using the cluster update settings API.

  • Modify the settings on a not-started or shut down node in the elasticsearch.yml configuration file.

  • Modify the configuration settings of your AWS Elasticsearch instance by contacting Hevo Support.

Handling Elasticsearch Exceptions

Exception while sorting or aggregating by the _id field.

Probable cause Aggregation or sorting of documents is being done using the _id field.
Error Message Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled.

Recommended action: Set the value of indices.id_field_data.enabled to true by editing the elasticsearch.yml file or using the cluster update settings API.

Note: This action applies to Elasticsearch versions below 8.0. For versions 8.0 and above, you must specify the unique identifier field for all objects on the object configuration page during Pipeline creation.

Circuit breaker exception when loading a [<field>] into the field data cache.

Probable cause The heap memory required to load the data for the field has caused the field data cache to exceed the predefined memory limit.
Error Message [fielddata] Data too large, data for [_id] would be [6194394308/5.7gb], which is larger than the limit of [6190792704/5.7gb].

Note: This is a sample error message.

Recommended action: Increase the value of indices.breaker.fielddata.limit in the elasticsearch.yml file.

Content too long exception from the Elasticsearch query.

Probable cause The length of the document being returned by the Elasticsearch query exceeds 100 MB.
Error Message Entity content is too long [224161559] for the configured buffer limit [104857600].

Note: This is a sample error message.

Recommended action: Increase the value of http.max_content_length in the elasticsearch.yml file.


See Also

Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Jan-16-2024 NA Updated section, Handling Elasticsearch Exceptions to add information about _id field being used for sorting only in specific Elasticsearch versions.
Jun-09-2022 NA New document.
Was this page helpful?

Tell us what went wrong