@jingyong zou The issue is with the flowfile format that is passed through the processor. As ConvertAvroToJson processor accepts only Avro format but i think you are passing Json format to the processor which is causing java.io.IOException: Not a data file error. ... View more

@Jean Paul Barddal TEZ initializes session when we run some hive job and based on the tez configurations we will have one YARN application initialized (as you can see in the Resource manager). The TEZ session will be same for all the queries that we are running until the session gets closed, In TEZ view we can see same application id for all the queries that we are running in Hive but DAG_ID will be different for all. In case of MapReduce we are not going to hold any of the resources like TEZ but we are going to run seperate applications for each query. - If the answer is helpful to resolve the issue, Login and Click on Accept button below to close this thread.This will help other community users to find answers quickly 🙂 ... View more

@3nomis ConvertRecord processor is still running as we can see running threads as one at top right corner of the processor. If it is running for ever (or) you are not convinced with performance then you can try splitting the 2GB file into smaller chunks by using SplitRecord processor then use MergeRecord processor to merge them back by using Defragment as merge strategy. ... View more

@Haijin Li Try with this and this approaches and also to test out the regex serde functionality, Create new file using vi editor in shell and move it to HDFS directory and create table on top of this directory. ... View more

@Patrick Hochstenbach Yes,You can consider custom processor/custom script for this, to do this in easier way and add the attribute to the flowfile with array size. ... View more

@jingyong zou Could you once change the split strategy and then run you select statement. Set hive.exec.orc.split.strategy=="ETL"; The available options are "BI", "ETL" and "HYBRID". ... View more