importing data from mysql to hive/hdfs using apache nifi

[#1]

am trying to import data from mysql to hive/hdfs but getting error.

1.

1. QueryDatabaseTable ---mysql data
2. ConvertAvroToJson --- output

   [{"emp_id": 467260, "emp_name": "Rob", "emp_age": 32}, {"emp_id": 467261, "emp_name": "Vijay", "emp_age": 32}, {"emp_id": 467258, "emp_name": "Jayaprakash", "emp_age": 26}, {"emp_id": 467259, "emp_name": "Kalyan", "emp_age": 32}, {"emp_id": 467262, "emp_name": "Andy", "emp_age": 20}, {"emp_id": 467263, "emp_name": "Ashley", "emp_age": 24}, {"emp_id": 467264, "emp_name": "Mounika", "emp_age": 24}]

3. splitjson -- how to split json file into single flow files

[#2 · Accepted solution]

What error are you getting? Also what version of NiFi/HDF are you using?

In SplitJson, the JSON Path expression you may want is $.*

As an alternative, you can try QueryDatabaseTable -> SplitAvro -> ConvertAvroToJson, this will split the Avro records first instead of converting the whole set to JSON then splitting the JSON.

In Apache NiFi 1.0.0 (and HDF 2.0), there will be a ConvertAvroToORC processor which will allow you to convert directly to ORC, then you can use PutHDFS and PutHiveQL (also in NiFi 0.7.0 and 1.0.0 and HDF 2.0) to transfer the files to HDFS and create a Hive table atop the target directory to make the data ready for querying.

[#3]

Hi,Matt Burgess, How to import data from mysql to hive use nifi PutHDFS processor and PutHiveQL processor? I already get ORC file, but i cannot put ORC file to hive? the NiFi1.0 I used.