Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

PutElasticsearchHttpRecord - Requested array size exceeds VM size limit

Explorer

I have a processor tree that I'm testing out with large json objects (1GB each). The tree is basically

GetFile -> UpdateAttribute (add avro schema name) -> PutElasticsearchHttpRecord

 

But PutElasticsearchHttpRecord gives me the following error (see screenshot)

 

HarshR_0-1595963353932.png

 

Can anybody tell me what's wrong? I have already assigned enough RAM in bootstrap.conf in NiFi:

# JVM memory settings
java.arg.2=-Xms16g
java.arg.3=-Xmx32g

 

 

 

1 REPLY 1

Explorer

@mburgessany suggestions? at the moment I am using SplitRecord -> PutElasticsearchHttpRecord and splitting it into 100,000 but it's pretty slow in comparison

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.