Hi Samir, There shouldn't be an end-less loop... As you can see in the HDFS example diagram, there are two parts to the flow: - ListHDFS -> RPG (this part only runs on primary node) - Input Port -> FetchHDFS -> rest of the flow (this part runs on all nodes) The starting point of your flow should be something that has no input, like ListHDFS, so there can't be a circular loop back to that point. The end of the second part should end with wherever you are sending your data, like PutHDFS for example, after that it is dead end, no loop back to anywhere. If this is not clear, please provide a screen shot or template of your flow so we can see how you have connected the processors. Thanks, Bryan ... View more

No problem, glad it was helpful 🙂 If you are reaching the login screen, then it means your browser is not forwarding your credentials to NiFi. You could try setting the negotiate properties to just "myhost.de" instead of the full url with port. Another thing to look at might be the domain being used by your KDC... In this example I was using nifi.apache.org as the domain, so I had to add a mapping in /etc/hosts to map nifi.apache.org to localhost so I could use nifi.apache.org in my browser to access my local NiFi. If you are accessing myhost.de to get to your NiFi instance, but that isn't the domain in your KDC, then it won't line up and probably won't forward your credentials. ... View more

Nice article! You could also use the "Message Demarcator" property in PublishKafka (set to a new-line) and this way you never have to split up your flow file, it will stream the large flow file and read based on the demarcator so you still get each line sent as an individual message to Kafka. ... View more

Hello, this post is for ListenUDP, ListenTCP, ListenSyslog, and ListenRELP. The ListenWebSocket processor is implemented differently and does not necessarily follow what is described here. I'm not familiar with the websocket processor, but maybe others have experience with tuning it. ... View more

Something is not set up correctly because the log is showing alvin@NIFI.COM in some places and alvin@NIFI.COM in other places. What are you entering as the username when you login? What is entered for the Initial Admin in authorizers.xml? ... View more

If you are setting up authentication for users accessing NiFi's UI, then you only need the spnego properties as shown in this post. If you need NiFi to authenticate to other services, for example to talk Ranger when Ranger is kerberized, then you need the service principal and keytab. ... View more

Hi @sukesh nagaraja I think the results you got are expected behavior behavior... The extracting request handler has no way to know the field names for the data you sent in. It is generally used to extract text from files like PDFs, or Word documents, where you basically have a title and content, and everything just goes into the content mostly. For your scenario, you basically have a CSV where you know the field names. Take a look at Solr's CSV update handler: https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-CSVFormattedIndexUpdates You can use this from NiFi by setting the path to /update and setting the Content-Type to application/csv and then add a property fieldnames with your list of fields. I'd recommend playing around with the update handler outside of NiFi first, just by using curl or a browser tool like Postman, and then once you have the request working the way you want, then get it working in NiFi. ... View more

Created this: https://issues.apache.org/jira/browse/NIFI-3979 ... View more

Is there any pattern about the file that is missed? Is it always the latest modification time of all the files in the directory? You can turn on DEBUG logging for org.apache.nifi.processors.hadoop.ListHDFS by editing logback.xml and you should see some more information that might be helpful. ... View more

There is an expression language guide here: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html For your example you should be able to create an UpdateAttribute processor and add a new property like: myfilename = ${filename:substringAfterLast('/')} ... View more