Hi @ChineduLB , You can use `.groupBy` and `concat_ws(",",collect_list)` functions and to generate `ID` use `row_number` window function.   val df=Seq(("1","User1","Admin"),("2","User1","Accounts"),("3","User2","Finance"),("4","User3","Sales"),("5","User3","Finance")).toDF("ID","USER","DEPT") import org.apache.spark.sql.expressions.Window df.groupBy("USER"). agg(concat_ws(",",collect_list("DEPT")).alias("DEPARTMENT")). withColumn("ID",row_number().over(w)). select("ID","USER","DEPARTMENT").show() ... View more

In   EvaluateJsonPath   processor add new property to extract   event_id   value from the flowfile. if flowfile is not having   event_id   then nifi adds empty value to the attribute. EvaluateJsonPath Configs:   Then by using   RouteOnAttribute   processor we can check the attribute value and route the flowfile accordingly. RouteOnAttribute Configs: not null value ${event_id:isEmpty():not()} null value ${event_id:isEmpty()} Then use null value and not null value relationships for further processing..!! ... View more

@Gerva    Select count(*) from <table>;  Query launches map reduce job and output will be displayed to the console. - If you want to store the output to file then use  Insert overwrite directory '<directory_name>' select count(*) from scenariox;   Now output of map reduce job will be stored into the given hdfs directory and you can find 00000_0 file in directory.. ... View more

@budati    For this case define your avro schema(with one field) to read incoming flowfile with some delimiter that doesn't exist in flowfile.   So that whole row will be read as string then we can filter out the records by using not like (or) using regex operator in apache calicite.   Select * from flowfile where col1 not like 'SKIP' Now output flowfile will not having any records that have SKIP in them and this solution will work dynamically for any number of columns. ... View more

@budati  i don't think there is a way to combine all 3 processors into one.   We still need to use ExecuteSQL -> ConvertAvroToJson -> EvaluateJsonPath to extract the values from the flowfile.   If the answer was helpful to resolve your issue, Accept the answer to close the thread 🙂 ... View more

@ANMAR  Try with this regex (?:"b"\s*:\s*)"(.*?)", This will extract only 24 - ny value from the given event ... View more