You can view the packaged version of Parquet in this pom.  ./nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/pom.xml ... View more

@vchhipa  If you are forced to "terminate" the processor after requesting it to stop, this can mean that you have a stuck or very long running process thread.  The "terminate" does not actually kill the thread, it disassociates that JVM thread from the processor and he current FlowFile that thread is associated with.  The terminated thread continues to run until it completes (NiFi does request thread to quit/exit when terminating it, but success of that varies since not all process support that ability). Any terminated threads still active will be represented in the processor by a small number in parentheses (1) displayed in its upper right corner.  The previously associated FlowFile is left on the inbound connection and will be handled based on queue priority  when the consuming processor is started again.  If the "terminated" thread should eventually complete, any output/return from that thread including logging is just sent to null. To figure out what is going on when you have a seemingly hung thread is to get a series of NiFi thread dumps: ./nifi.sh dump <dump-filename-nodenum-01.txt> Getting at least 3 dumps at an interval of 2-5 minutes apart is usually good. What you are looking for is a thread associated with your processor class (invokeHTTP in this case) where the same thread number exists in every thread dump collected.  Then you will want to luck that the thread stack to see if all are identical.   If thread output changes between thread dump outputs, it indicates that thread is not hung but rather just long running.  If thread dump output does not change, you'll want to dig in to that output to see what it is waiting on. Hope this helps, Matt ... View more

Hi, @murali2425 @vchhipa It seems some dependency issue while building your custom NiFi Processor,  org.apache.nifi:nifi-standard-services-api-nar dependency needs to be added in pom.xml of nifi-*-nar module. Ref here    <dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-standard-services-api-nar</artifactId> <version>1.11.3</version> <type>nar</type> </dependency>   Please modify your pom.xml and rebuild and see whether that fixes the issue. Please accept the answer you found most useful. ... View more

Hello @Love-Nifi  and @vchhipa ,   Thank you for posting your inquiry about timeouts. Without the full log, I can provide only some "if you see this, do that" kind of instructions.   If you see an ERROR message with: org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow, then follow the below is the steps to resolve the issue: 1. Go to NIFi UI > Global Menu > Cluster 2. Check which host is the coordinator and login to that host on the shell. 3. Go to flow.xml.gz file location. [default location is /var/lib/nifi/conf/] 4. Copy flow.xml.gz on the disconnected node and replace the original flow.xml.gz with copied flow.xml.gz file. 5. Check permissions and ownership of newly copied flow.xml.gz file and then restart Nifi on the disconnected node only.   If you are suspecting purely timeout issues, please attempt to tweak the below values in nifi.properties and restart the service: - nifi.cluster.node.protocol.threads=50 (Default 10) - nifi.cluster.node.connection.timeout=30 sec (Default 5 sec) - nifi.cluster.node.read.timeout=30 sec (Default 5 sec) Please find below a set of configurations that worth tuning on larger clusters based on https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html The below are some example values for larger clusters (you need to tune it based on your unique setup): nifi.cluster.node.protocol.threads=70 nifi.cluster.node.protocol.max.threads=100 nifi.zookeeper.session.timeout=30 sec nifi.zookeeper.connect.timeout=30 sec nifi.cluster.node.connection.timeout=60 sec nifi.cluster.node.read.timeout=60 sec nifi.ui.autorefresh.interval=900 sec nifi.cluster.protocol.heartbeat.interval=20 sec nifi.components.status.repository.buffer.size=300 nifi.components.status.snapshot.frequency=5 mins nifi.cluster.node.protocol.max.threads=120 nifi.cluster.node.protocol.threads=80 nifi.cluster.node.read.timeout=90 sec nifi.cluster.node.connection.timeout=90 sec nifi.cluster.node.read.timeout=90 sec   Please check if you notice any certificate related exception, like: WARN [Clustering Tasks Thread-2] o.apache.nifi.controller.FlowController Failed to send heartbeat due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'HEARTBEAT' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate In this case, create a new keystore and truststore and add client auth in the keystore.   Best regards: Ferenc ... View more

If you don't care about Avro format and want to go directly to JSON you can use the ExecuteSQLRecord processor where you can specify the output format.  ... View more

I followed the suggestions #1 in this answer and it worked.  If that doesn't work, to be sure that the files are identical on the servers, do md5sum of the files.  ... View more

What helped me was, first copying the file to the $NIFI_HOME/lib folder then giving the full path of the jar file in the ExecuteStreamCommand processor. So the config looked like "-jar; /opt/nifi-1.7.1/lib/mycode.jar". Couple things to ensure is that the jar is owned by the same user that NiFi is running as and the jar could be located anywhere as long as you give full path you should be fine. ... View more