@Bern this is your new question without a accepted solution.  So a bit confused by your last response. Matt ... View more

@AlokKumar  NiFi allows very granular authorizations down to the individual component.  A component such as a processor will inherit its authorizations from the Process Group in which it resides, IF there are no explicit policies set directly in the processor itself.  Likewise, a Process group will inherit it's authorization from it's parent Process Group if it does not have explicit policies set directly on that child process group. When you launch NiFi for the very first time, NiFi will create the root Process Group for you and it will have the name "NiFi Flow".  It is the UI canvas you see when you access the UI.  Form the second image you shared we can see that you have access the "policies" for a child process group named "Copy of ProcessGroupAdminTest".   What we can also see from this is that it is inheriting the "view the component" policy from the root process group "NiFi Flow":  This is why you will see the add user and delete options as greyed out.  You need to first click "override" and choose either to start with no users or copy the current authorized users.  After doing this you will be able to add additional user to this policy on this child process group. Keep in mind that once you override inheritance on this components policy(s), inheritance no longer applies to this component.  Any changes to the policies set on the parent Process Group "NiFi Flow" will not get applied to this child Process Group. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@HoangNguyen  The "Deprecation log" has nothing to do with your running dataflows on your NiFi canvas.  The deprecation log contains notifications about NiFi components you may be using that are deprecated.  Deprecated components get removed in future Apache NiFi versions.  This log is to make you aware of this so you can make dataflow design changes to stop using them before you migrate to newer Apache NiFi release.  The NiFi Standard Log Files  include the "bootstrap log, app log, and user log".  The app log is where you will find alll your dataflow component based logging.  In the logback.xml, "logger" will write to the nifi-app.log by default unless a specific "appender-ref is declared for the logger.   NiFi app.log can produce a lot of logging, but to get it all you can adjust: <logger name="org.apache.nifi" level="INFO"/> to "DEBUG" instead of INFO.  It will be very noisy. Logback standard log levels:   OFF: This level turns off all logging. No messages will be outputted. ERROR: Indicates a serious error that might still allow the application to continue running, but requires attention. WARN: Indicates a potentially harmful situation that should be investigated, but does not necessarily prevent the application from continuing. INFO: Provides general information about the application's progress and significant events. DEBUG: Offers detailed information useful for debugging purposes, often including variable values and execution flow.   TRACE: Provides even finer-grained information than DEBUG, typically used for extremely detailed tracing of execution paths. ALL: This level enables all logging, including messages at all other levels. Keep in mind that just because you set DEBUG log level, does not mean every component will produce DEBUG level log messages. It all depends on what logging exists within the component class and dependent libraries.  When set to DEBUG, it will log DEBUG and all level below it (INFO, WARN, ERROR).  If you set "INFO", you also get WARN and ERROR logging. NiFi user authorization logging will go to the nifi-user.log.  This is logging related to access to NiFi.  Nifi-bootstrap.log has logging for you rNiFi bootstrap process.  The bootstrap is what is lauched when you execute the nifi.sh start command.  The bootstrap then starts the nifi main child process whcih loads your NiFi and dataflows.  The bootstrap then monitors that child process to make sure it is still live (restarts it automatically if it dies).   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@HoangNguyen  As long as you are running a new enough version of Apache NiFi, you'll have an option with the process group configuration to set a logging suffix. For each process group you want a separate log file, create a unique suffix. In above example I used the suffix "extracted". In my NiFi "logs" directory, I now have a new "nifi-app-extracted.log" file that contains the logging output of every component contained within that process group. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@asand3r  JVM Garbage collection is stop-the-world which would prevent for the duration of that GC event  the Kafka clients from communicating with Kafka. If that duration of pause is long enough I could cause Kafka to do a rebalance.  I can't say that you are experiencing that .   Maybe put the consumeKafka processor class in INFO level logging and monitor the nifi-app.log for any indication of rebalance happening. When it comes GC pauses, a common mistake I see is individuals setting the JVM heap settings in NiFi way to high simply because the server on which they have in stalled NiFi has a lot of installed memory.   Since GC only happens once the allocated JVM memory utilization reaches around 80%, large heaps could lead to long stop-the-world if there is a lot top clean-up to do.   Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more