I am on HDP 2.3.4 and when I am running the above step with small data set it works fine but when I am trying to load 15 million rows, I am getting this error: Vertex failed, vertexName=Reducer 2, vertexId=vertex_1492812474981_9078_1_01, diagnostics=[Task failed, taskId=task_1492812474981_9078_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.io.IOException: Multiple family directories found in hdfs://apps/hive/warehouse/somedb.db/hive_cdi3/_temporary/0/_temporary I google it and it says that this is known bug: https://issues.apache.org/jira/browse/HIVE-13539 Please let me know if there is any work around. ... View more