@Sami Ahmad Users who are not authorized for specific NiFi components (process groups, processors, input/output ports, etc...) will only see "ghost" processors. Those ghost processors will not contain any information on the face of them the reveals their type or configuration. - - Unless authorized, user also will not be able to list queues, perform provenance queries, etc on FlowFiles that traverse these components. - There are good reasons why NiFi does not hide these ghost processor completely: 1. All components operate within the same resource constraints of a single JVM. It is important that all user can identify areas of any flow that may be backlogging or have large queues, as these things may impact their own dataflows. While the can not see the actual data or processor details, they can observe FlowFile stat info as you can see in above screenshot. 2. If User1 built a dataflows only user 1 could see, User2 may end up building a dataflow directly on top of it. While functionally NiFi does not care about overlapping components, user3 who logs in and has access to both user1 and user2 flows would. It would make it very hard for user 3 to follow the canvas in such a case. - Thanks, Matt ... View more

@Vladislav Shcherbakov - The ListFTP processor generates the following FlowFile attributes on each 0 byte FlowFile it generates: so you would typically use NiFi's Expression Language (EL) to define values for those properties dynamically per each processed FlowFile: For example: The property "Remote File" on the FetchFTP processor set a value of "${filename}". With each FlowFile received it will return the value assigned to this attribute and use it to retrieve the correct FlowFiles content from the target FTP server. - Thank you, Matt ... View more

@rajat puchnanda Here is screenshot showing the menus and icons that go along with the above explanation: Thanks, Matt ... View more

https://community.hortonworks.com/content/kbentry/109629/how-to-achieve-better-load-balancing-using-nifis-s.html ... View more

thanks. works perfectly ... View more

@Matt Clarke Ah ok. That makes sense. I did paste the public IP in both at the time. The current config works with having the setting empty or to private IP. Thank you. Mistery solved. >_< ... View more

@Bobby Harsono - Some processor may be designed to utilize memory outside of the JVM. Some of the scripting processor like ExecuteProcess or ExecuteStreamCommand are a good examples. They are calling a process or script external to NiFi. Those externally executed commands will have a memory footprint of their own. - Listen type processors like ListenTCP or ListenUDP is another example. These have memory footprints both inside and outside the NiFi JVM heap space. These processors can be configured with socket buffer which is created outside of heap space.- - Thanks, Matt ... View more

@dhieru singh The Jira lists the fix as being addressed in Apache NiFi 1.7.0 which has not been released yet. - Thanks, Matt ... View more

@Henrik Olsen The FetchSFTP will make a separate connection for each file being retrieved. Concurrent Tasks will allow you to specify the number of concurrent connections allowing more then one file to retrieved per processor execution schedule (still one file per connection). - Yes, HDF 3.1 will have all these goodies. Suggest skipping directly to HDF 3.1.2 which was just released since it has a loyt of fixes for some annoying bugs in HDF 3.1 and 3.1.1. - You will have the option to use either an external REDIS configured how you like or an internal NiFi DistributedMapCacheServer with the WAIT and NOTIFY processors. - The DistributedMapCacheServer provides the following configurations: There is no TTL for the DistributedMapCacheServer option. - There also isn't a processor that will dump out the current contents of the DistirbutedMapCacheServer, but you should be able to write a script that can do that for you. Here is an example script that is used to remove a cached entry: https://gist.github.com/ijokarumawak/14d560fec5a052b3a157b38a11955772 - I do not know a lot about REDIS, but as an externally managed cache service, it probably will give you a lot more options as well as a cluster capability so you don't have a single point of failure like you would have with the DistributedMapCacheServer. - Thank you, Matt ... View more

@Artem Anokhin If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer. *** Forum tip: Pleasse try to avoid responding to an Answer by starting a new answer. Instead use the "add comment" tp respond to en existing answer. There is no guaranteed order to different answers which can make following a response thread difficult especially when multiple people are trying to assist you. ... View more