Support Questions
Find answers, ask questions, and share your expertise

PutElasticsearchHttpRecord - Requested array size exceeds VM size limit

PutElasticsearchHttpRecord - Requested array size exceeds VM size limit

Explorer

I have a processor tree that I'm testing out with large json objects (1GB each). The tree is basically

GetFile -> UpdateAttribute (add avro schema name) -> PutElasticsearchHttpRecord

 

But PutElasticsearchHttpRecord gives me the following error (see screenshot)

 

HarshR_0-1595963353932.png

 

Can anybody tell me what's wrong? I have already assigned enough RAM in bootstrap.conf in NiFi:

# JVM memory settings
java.arg.2=-Xms16g
java.arg.3=-Xmx32g

 

 

 

1 REPLY 1

Re: PutElasticsearchHttpRecord - Requested array size exceeds VM size limit

Explorer

@mburgessany suggestions? at the moment I am using SplitRecord -> PutElasticsearchHttpRecord and splitting it into 100,000 but it's pretty slow in comparison