Hi   All,   I have used three process groups in nifi (pg1,pg2,pg3). There is an output port called as pg1_Out coming out from pg1. This port is connected to pg2_in and pg3_in input ports.   pg1_out------>pg2_in   |   |   | pg3_in   Problem : when i trigger these process groups then flowfiles are passing to only pg3_in but not to pg2_in. Ideally , the flow files should get passed to both input port. I do not understand why am i seeing such behaviour in nifi.   Does anyone has faced such problem before? Please respose to me your thoughts on this. I would be really appreciated to your help.     ... View more

Hi All,  I am tyring to execute update query through putSQL processor. I want to update number of processed file count column of one of the transactional table in mysql.  The issue is that whenever  multiple flowfiles coming as input to putSQL processor at a time,the processed file count is getting incremented and updated only once. Ideally,the behaviour should be like increment the counter one by one for each flowfile and then update the processed file count column in MySQL table.   NiFi flow: Fetch processed file count using select query->updateattribute : processed file count+1->putSQL:update query.   Thanks in advance Please share you response on it. ... View more

Hello all,   Problem statement: We have 3 layers in our project,for each layer we have  nifi flow. Currently, the layers or nifi flow for each layer is getting executed one after another (sequentially).   Execution flow: layer1->layer2->layer3->  so on.   The trigger point or input parameters to the flow is nothing but number of source files. Which will be processed further in every layer one by one.   I am looking for such solution ,where my next nifi flow for next layer should not get triggered until completion of previous one for previous layer and for all files.   And if current layer got succeeded for all the files then only next one will kick off.   Please suggest me your thought process on the same. I will be very appreciated on your response.   Thank you in advance.     ... View more

Hi, I have installed below Hadoop services and all services are running fine services.png The problem is that when i am executing below query from Hive cli I am getting the exception in terminal as. Query : select count(*) from mytables; Exception: short exception description: Job Submission failed with exception 'java.io.FileNotFoundException(File does not exist: hdfs://<nanodeHostIp>:8020/hdp/apps/2.2.9.0-3393/mapreduce/mapreduce.tar.gz) please find attached exception file exception.txt and I have also checked path on HDFS "/hdp" which is not completely present on file system so my question is, As have done fresh hadoop installation through Apache ambari then why ambari did not created files or directories under "/hdp" directory automatically. What is the solution for above problem? Thanks in advance. ... View more