@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi. ... View more

Please advice if the Event driven will be now available/implemented for NiFi 1.x and later versions? ... View more

The mergeContent Processor simply bins and merges the FlowFiles it sees on an incoming connection at run time. In you case you want each bin to have a min 100 FlowFiles before merging. So you will need to specify that in the "Minimum number of entries" property. I never recommend setting any minimum value without also setting the "Max Bin Age" property as well. Let say you only ever get 99 FlowFiles or the amount of time it takes to get to 100 exceeds the useful age of the data being held. Those Files will sit in a bin indefinitely or for excessive amount of time unless that exit age has been set. Also keep in mind that if you have more then one connection feeding your mergeContent processor, on each run it looks at the FlowFiles on only one connection. It moves in round robin fashion from connection to connection. NiFi provides a "funnel" which allows you to merge FlowFiles from many connections to a single connection. Matt ... View more

What is a task in this context ? reading a line ? ... View more

i think changing the run schedule worked. thank you. ... View more

@mclark Thanks for the response and appreciated. Do I need to configure something at back-end as well i.e. in nifi.properties or any other file in cluster or node because I am facing attached error. ... View more

+1 to mclarke's solution. ... View more

yeah thanks.. append works.. \\n (double backslash) doesnt.. I was doing this while writing count to file.. it now works.. The count is: ${executesql.row.count:append('\n')} ... View more

If I create a new template, it is creating a flow.xml.gz and working fine. However If I replace the flow.xml.gz with old flow.xml.gz (one backup taken earlier in same cluster node HDF2.X) nifi UI is not coming after login and giving error as "com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out". Have tried with all parameter tuning as described by "Matt Clarke" in some other post but no results. Again If I move old file and replace with new flow.xml.gz then nifi is working fine. Please let me know if anyone faces such issues and probable reason and working around. Thanks, Suman ... View more

@Michael Sobelman That DNS is not detectable by the node you are trying to access from. You can be fancy on aws and configure through routing tables by setting up a proper vpn between the EMR and NiFi nodes. Another option I used is route53 which will give you DNS publicly available. Lastly you can put a ELB infront of your EMR HBase master node. You may have to script it up (via boot scripts) to configure your ELB to point to new internal IP. ... View more