Member since
09-11-2024
2
Posts
2
Kudos Received
0
Solutions
10-09-2024
09:02 PM
1 Kudo
Hi Team, Currently i am using 5 node cluster of NIFI - 1.27 version. node have 47 Gb RAM My NIFI heap memory level keep on increasing and crashing by reaching 150 hours. Frequency of data is around 100 Gb in a day. I am using variety of processors (Kafka, split and merge , distributed cache processors,Attribute level processors, hive3streaming processor,(select and put)Execute SQL (postgre,SSIS),Kudu processor etc). Please find nifi properties below: # JVM memory settings java.arg.2=-Xms16g java.arg.3=-Xmx20g java.arg.7=-XX:ReservedCodeCacheSize=512m java.arg.9=-XX:+UseCodeCacheFlushing java.arg.23=-XX:ParallelGCThreads=8 java.arg.24=-XX:ConcGCThreads=4 java.arg.25=-XX:G1ReservePercent=10 java.arg.26=-XX:+UseStringDeduplication java.arg.27=-XX:InitiatingHeapOccupancyPercent=25 java.arg.28=-XX:MaxGCPauseMillis=200 java.arg.40=-XX:SurvivorRatio=8 java.arg.41=-XX:NewRatio=3 java.arg.42=-Xmn6g java.arg.13=-XX:+UseG1GC nifi.content.repository.archive.max.retention.period=7 hours nifi.content.repository.archive.max.usage.percentage=50% nifi.provenance.repository.max.storage.time=30 hours nifi.provenance.repository.max.storage.size=10 GB nifi.provenance.repository.rollover.time=10 mins nifi.provenance.repository.rollover.size=100 MB nifi.provenance.repository.query.threads=2 nifi.provenance.repository.index.threads=2 nifi.queue.swap.threshold=10000 Tagging @SAMSAL @MattWho for quick response Thanks, Krish
... View more
Labels:
- Labels:
-
Apache NiFi
09-11-2024
11:58 PM
1 Kudo
Hi Team, i am using 1.27 NIFI, For QuerySalesforceObject processor i am unable to find proxy configuration service for instance url, We have Proxy configuration service for OAuth2 Access token but not for salesforce url, Please suggest for proxy service enablement for salesforce instance URL as well. Sample screenshot of existing configuration below. Thanks, Krish
... View more
Labels:
- Labels:
-
Apache NiFi