Skip to main content
TimothySpann
Platinum ⭐️⭐️⭐️
TimothySpann
Platinum ⭐️⭐️⭐️
May 3, 2018

Converting CSV Files to Apache Hive Tables with Apache ORC Files

  • October 5, 2026 at 9:10 AM
  • 0 replies
  • 4790 views
  • Translate All

Converting CSV Files to Apache Hive Tables with Apache ORC Files


I received some CSV files of data to load into Apache Hive. There are many ways to do this, but I wanted to see how easy it was to do in Apache NiFi with zero code.

72565-cvsprocessing1.png

I read CSV files from a directory of files. Then I can Convert the CSV to AVRO directly with ConvertRecord.

72562-convertrecordcsv.png

I will need a schema, so I use the below settings for InferAvroSchema. if ever file is different, you will need to do this every time.

72563-avroschemafromcsv.png

CSV Reader

72564-csvreader.png

I use the Jackson CSV parser which works very well. The first line of the CSV is a header. It can figure out the fields from the header.

Once I have an Apache AVRO file it's easy to convert to Apache ORC and then store in HDFS.

Template:

csvprocess.xml

    Terms & ConditionsAccessibility statement