Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop import suddenly fails after few years


Sqoop import suddenly fails after few years


we are importing data to hdfs with sqoop now for a few years.

Yesterday it started to fail with the following error:

19/02/20 05:14:00 INFO mapreduce.Job: Job job_1549278403763_0242 failed with state FAILED due to: Job commit failed: org.apache.hive.hcatalog.common.HCatException : 2012 : Moving of data failed during commit : Could not find a unique destination path for move: file = hdfs:// , src = hdfs://, dest = hdfs://

at org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.getFinalPath(

at org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.moveTaskOutputs(

at org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.moveTaskOutputs(

at org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.registerPartitions(

at org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.commitJob(



at java.util.concurrent.ThreadPoolExecutor.runWorker(

at java.util.concurrent.ThreadPoolExecutor$


The command we are using is:

/usr/bin/sqoop import --connect "jdbc:sap://IP:30015/?databaseName=TH1&instanceNumber=00&currentschema=schemaname" --username HADOOP --password Init1234 --driver --table BUFFER_INVOICE_HEADER --split-by datetimecreated --hcatalog-database dbname --hcatalog-table tablename --hcatalog-storage-stanza "stored as orc" --null-string null --null-non-string null --map-column-java ORIGINALXML=String --map-column-hive ORIGINALXML=String -m 28

The last file name in hdfs folder is "part-m-00027_a_998"

There are 27968 files in that hdfs directory.







--append does not work with HCatalog (Append mode for imports is not compatible with HCatalog. Please remove the parameter--append-mod)

Is there a way to overcome this limitation? According to this post limitation is 999 files, which is also our case.

Don't have an account?
Coming from Hortonworks? Activate your account here