Yep what you describe with UpdateAttribute/MergeContent sounds perfectly fine. What you'll want there precisely will depend on how many relationships you have out of RouteText. As for concurrent tasks I'd say it would be 1 for GetFile 1 for SplitFile 2...4 or 5 or so on RouteText. No need to go too high generally. 1 for MergeContent 1 to 2 for PutHDFS You don't have to stress too much on those numbers out of the gate. You can run it with minimal threads first, find any bottlenecks and increase if necessary. ... View more

Thanks ! that was helpful . I don't see anywhere "Accept" ... View more

awesome solution mclark ... View more

@Sami Ahmad Instead of left clicking on the link in step three, right click and select "Save Link As..." option to save the xml template so it can be imported in to your NiFi. The dataflow template will show you all the components needed for this workflow. I believe the intent of this tutorial was not to teach users how to use the NiFi UI, but rather how to use a combination of specific NiFi components to build accomplish a particular workflow. Using the NIFi UI dataflow tools you can recreate the workflow as a UI dataflow building exercise. Thanks, Matt ... View more

@Matt Thank you, It works now! ... View more

@mclark @Bryan Bende I tried the same thing , I had the List-->Fetch-->Output port inside a process group on remote machine. My local NiFi was not able to find that port , But when I removed it from the process group and copied on to main canvas then my local NiFi was able to find it. Thank you both. ... View more

@Saikrishna Tarapareddy Your Regex above says the CSV file content must start with Tagname,Timestamp,Value,Quality,QualityDetail,PercentGood So, it should not route to "Header" unless the CSV starts with that. What is found later in the CSV file should not matter. I tried this and it seems to work as expected. If i removed the '^', then all files matched. Your processor is also loading 1 MB worth of the CSV content for evaluation; however, the string you are searching for is far fewer bytes. If you only want to match against the first line, reduce the size of the buffer from '1 MB' to maybe '60 b'. If I changed the buffer to '60 b' and removed the '^' from the regex above, only the files with the matching header were routed to "header". Thanks, Matt ... View more

@kishore sanchina you will need to use a protocol. If you simply want to "push" local files to nifi, you can use the ListenHTTP processor. Then simply curl the file to nifi. ... View more

Please advice if the Event driven will be now available/implemented for NiFi 1.x and later versions? ... View more

The mergeContent Processor simply bins and merges the FlowFiles it sees on an incoming connection at run time. In you case you want each bin to have a min 100 FlowFiles before merging. So you will need to specify that in the "Minimum number of entries" property. I never recommend setting any minimum value without also setting the "Max Bin Age" property as well. Let say you only ever get 99 FlowFiles or the amount of time it takes to get to 100 exceeds the useful age of the data being held. Those Files will sit in a bin indefinitely or for excessive amount of time unless that exit age has been set. Also keep in mind that if you have more then one connection feeding your mergeContent processor, on each run it looks at the FlowFiles on only one connection. It moves in round robin fashion from connection to connection. NiFi provides a "funnel" which allows you to merge FlowFiles from many connections to a single connection. Matt ... View more