Thanks mate. I did that change and ensured it is consistent on those three reference points in the yml file. However when I ran the minifi script , I am getting some weird errors :- Exception in thread "main" java.io.IOException: Unable to successfully transform the provided configuration at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1714) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.start(RunMiNiFi.java:1183) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values One thing though I have noticed is that when I create the YML file , somehow the Input Port ID is not reflecting right . Some other input port id number comes up and this is also reflected at the same 3 places but it doesn't match which the one created on Nifi UI which is kind of strange, I tried searching that incorrect input port id on the nifi canvas , and I got a match actually , but that was linking to the template that I created for the YML file and not the actual input port . I will have a look again and revert but thanks alot for your inputs. It has definitely given me a direction for sure. cheers Abhi ... View more

Hi @Vinicius Higa Murakami Thanks for your response. I havent made any change as such . I will just show you the screen shots so that you can tell me whether I am on the same page As seen above, are you referring to the Input Port like it is mentioned above . Basically i want to see data coming out from this Port to the log Attribute processor. The Id of the port is 7d8779b7-0164-1000-0000-00007fc774f5 as seen above, However when I look at my YML file , under the section for input ports i.e in my case FROM_CADLBIG03 i dont see this number. Please see the section below , higlighted is the id number . Do you think , I should update that id number with the input port id in the earlier screen shot ? please let me know ... View more

great . Thanks alot @Geoffrey Shelton Okot . I will go through this and come back to you ... View more

I have set my Transactions per batch to 10. Is that too low ? I am really looking at ways to process my queues faster . it always seems to be piling slowly over time ... View more

Hi @Shu Thanks - this is now fixed. We realized the Nifi Nodes werent added to the hadoop topology which is why we were getting that error. This has been added and the data is going through !! Thanks for all the help .. Have accepted your answer .. ... View more

Thanks for these details. I will try all this and see if I can get this fixed . The basic issue we are facing here is when we are trying to insert data into a partitioned hive table using "put hive streaming" processor . The issue that we are getting is as follows :- org.apache.nifi.util.hive.HiveWriter$CommitFailure: Commit of Txn 36315 failed on EndPoint: {metaStoreUri='thrift://caplifs0c.telecom.tcnz.net:9083', database='adhoc', table='nifi_s_order_test1', partitionVals=[2015-10-26] } Do you by any chance have an idea how to fix this issue . The "PutHiveStreaming" works fine for a non partitioned table but as soon as we have this set to a partitioned table , we start seeing these issues. Actually , we set all the required hive parameters in the update attribute processor just before the put hive streaming processor. Screen shot below :- Have you used the put hive streaming processor to insert data into a partitioned table ? Thanks @Shu . waiting for your response ... View more