Thank You Sagar for your inputs! I am thinking on the following lines regarding the NiFi Flow: GetFile (To fetch input file from a specific directory) >> PutHDFS (Write Input File data to HDFS) >> PutHiveQL (Through "LOAD PATH INFILE..." load the data into Hive) >> SelectHiveQL (Execute the Semicolon delimited Hive QL's as Input & output in Avro format) >> ConvertAvroToJson (Direct Mapping from Avro field to JSON) >> SplitJson (Splits a JSON File into multiple separate FlowFiles) >> EvaluateJSONPath (Evaluate JSON Path expressions against content of a FlowFile & to assign the results to FlowFile attributes) >> ConvertJSONToSQL (Converts a JSON Formatted FlowFile into INSERT SQL Statement) >> PutSQL (Execute the SQL INSERT statement to actually insert the data into the Destination SQL Database) Please can you validate if the above flow will work ? Also I have a another question as follows: If I select multiple column values from a Hive Table in a HiveQL (e.g. "Select name, age, grade from student table" ) and retriving say 10 records for this query and further I want to insert these records / values into destination SQL Database table then what should the flow look like? Can I use the flow I described above to solve this requirement. Kindly share your thoughts / inputs. Thanks & Regards in advance !
... View more
