Is there a way to debug a custom NiFi process that is running inside the dataflow?

[#1]

I have written a custom NiFi process for use with elasticsearch and I have tested it thoroughly outside of the DataFlow environment and it works perfect but when I drop it into the flow it throws an error message that is of no help so I was wondering if there is a way to debug the code from within the flow or produce a better error message. Any advice/help is appreciated, thanks.

[#2 · Accepted solution]

You can attach the debugger from your IDE to NiFI... In NiF's conf directory in bootstrap.conf there is a line commented out like this:

#java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000

If you uncomment that line and restart NiFi, the Java process will be listening for a debug connection on port 8000. If you want the process to wait for a connection before starting, you can also set suspend=y. Connecting your IDE debugger to the Java process will be specific to which IDE you are using.

You can also add addition logging statements in your processor code using the logger return from getLogger() to see how far it is getting. The log-levels are controlled through the logback.xml file in the conf directory. The default level for processors is WARN:

<loggername="org.apache.nifi.processors"level="WARN"/>

[#3]

Thank you sir! I will give it a shot.

[#4]

I know this is an old post, but in case if future search results turn up here and someone is unclear on how to do this part of @Bryan Bende's answer:

Connecting your IDE debugger to the Java process will be specific to which IDE you are using.

I wanted to share a link to a detailed tutorial I put together for how to do this in IntelliJ IDEA: https://community.hortonworks.com/content/kbentry/106931/nifi-debugging-tutorial.html