We have a requirement wherein we get different type of input files i.e avro, json and csv (example: client1_product.avro,client2_product.json,client3_product.csv, etc) and we have to convert these files to csv and load to db2. Currently I have designed a job with GetFile to fetch the avro files and ConvertRecordProcessor to convert records (avro to csv) and load it to db2 and this works fine but we need to design a generic job to read any file format (i.e avro, json and csv) and load it to db2 table. Can we design Nifi job to read different format files and convert it to a csv format in same job? Could you please advise.   Thanks, Avi ... View more

We are running spark jobs using spark-shell command : e.g: spark-shell --conf config_file_details --driver-memory 4G --executor-memory 4G -i spark_job.scala we are not using spark-submit command to submit job but instead of it we are using spark-shell command Could you please advice which will be fast in terms of performance. Thanks, ... View more