- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
PutElasticsearchHttpRecord - Requested array size exceeds VM size limit
- Labels:
-
Apache NiFi
Created ‎07-28-2020 12:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a processor tree that I'm testing out with large json objects (1GB each). The tree is basically
GetFile -> UpdateAttribute (add avro schema name) -> PutElasticsearchHttpRecord
But PutElasticsearchHttpRecord gives me the following error (see screenshot)
Can anybody tell me what's wrong? I have already assigned enough RAM in bootstrap.conf in NiFi:
# JVM memory settings
java.arg.2=-Xms16g
java.arg.3=-Xmx32g
Created ‎08-04-2020 07:35 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@mburgessany suggestions? at the moment I am using SplitRecord -> PutElasticsearchHttpRecord and splitting it into 100,000 but it's pretty slow in comparison
