Member since
07-30-2019
891
Posts
195
Kudos Received
91
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
668 | 10-05-2021 01:53 PM | |
12775 | 09-23-2019 06:03 AM | |
4723 | 05-04-2019 08:42 PM | |
890 | 06-11-2018 12:45 PM | |
8797 | 06-04-2018 01:11 PM |
11-14-2019
06:52 AM
1 Kudo
With that big of a file, you are going to need to adjust settings (memory) and evaluate performance of the file repositories. I would recommend to start with small files, and work your way up. Complete a benchmark at each stage, then begin testing config changes. This should be a lengthy evaluation where in the end, you end up with a configuration capable of consuming 3 gb files.
... View more
09-23-2019
06:03 AM
@budati If you cannot get the blank space in, then just go back to the default of ; and add it between the parameters in the Command Arguments property.
... View more
10-01-2018
06:44 PM
@leah wang The NiFi java process is actually two processes. The actual NiFi java process and a bootstrap java process. The bootstrap process has a very small memory footprint, 12MB to 24MB, that looks for the NiFi java process. If it doesn't find that process it will try to start it. The NiFi java process will reserve the memory allocated, in this case 8GB, and hold it. It doesn't actually use the memory, but reserves it so it is available as NiFi needs it. There won't be any garbage clean up until the memory usage is around 80% utilization.
... View more
06-21-2019
12:11 PM
Kafka Consumer Demarcator could be one solution.Use Shift+enter and try persisting into HDFS.
... View more
05-15-2018
05:06 PM
@David Sandoval That is great to hear. Would you do me a favor and accept my answer?
... View more
03-17-2018
10:08 AM
1 Kudo
@Pavan M As you are not transferring any of the flowfiles to REL_FAILURE, Transfer the else flowfiles to failure relation and auto terminate the failure relation, else:
session.transfer(flowFile, REL_FAILURE) Auto terminate failure relation (or) You can use session.remove to remove the flowfile else: session.remove(flowFile) by using any of the above ways you can achieve same result as you are expecting.
... View more
12-21-2017
08:20 PM
1 Kudo
@sally sally Check the back pressure object threshold value in the connection feeding the MergeContent processor to make sure it has been changed to a value that will allow enough files to queue up.
... View more
04-24-2018
08:57 PM
1 Kudo
@Jose
Gonzalez You can specify more then one host but it is not required. Once the RPG establish a connection to the target host it will retrieve the S2S details of the target cluster and store that locally. If the host you provided become unavailable at anytime after that initial connection , it will try anyone of the other nodes it learned about previously to get S2S details. Having multiple nodes configured helps when NiFi by giving the source Nifi more then one target node to establish initial connection with. - Your load-balancing issue is completely unrelated to how many nodes URLS you configured in your RPG. Here is an article that covers how load-balancing works with an RPG: https://community.hortonworks.com/content/kbentry/109629/how-to-achieve-better-load-balancing-using-nifis-s.html - Thanks, Matt
... View more