Are you using hftp:// or webhdfs://? I'd recommend trying with the latter. For this specific exception in REST based copies, its usually not a fault with the network but a buggy state in the older Jetty used on the source cluster. Typically a rolling restart of the DataNodes will help resolve such a bad state of Jetty where it hangs up on a client mid-way during a response, causing the sudden EOF to the copying client in DistCp when it was expecting the rest of data. ... View more

The `delete` named column in your retailSKU table is actually a reserved word, and that's the central point of the issue. When Sqoop's building out the query, its currently not escaping the column names with backticks (``) which is necessary when not intending to mean using a reserved word ( https://dev.mysql.com/doc/refman/5.7/en/keywords.html). Would it be possible for you to alter the column name on the source retailSKU table, or pass a custom query instead which uses the right escape syntax, via --query: http://archive.cloudera.com/cdh5/cdh/5/sqoop/SqoopUserGuide.html#_free_form_query_imports ? Edit: Just noticed you've mentioned that free-form works just fine. ... View more

While MultipleInputs was designed for such a thing, your requirement is unique in that you need to process the same input 2x but with different params each time. It seems a bit redundant to me given that you can do it in a single task run vs. 2x the I/O cost… But I believe the way you can solve your identifier problem is by writing your own InputFormat wrapper over the existing InputFormat, which generates special types of InputSplit objects (wrapper over regular FileSplit classes). These input splits need to add in your identifiers as an extra field, and you can extract and cast the same from your context.getInputSplit() in the map-end to then differentiate the input. ... View more

When an RS exits, it logs a lot of content from the many threads it runs. Your log snippet is just the post-exit-decision messages, as you can note from the several "exiting" messages. Please instead pass us your entire log file, or look for the entire line and its following exception stack trace matching the word "FATAL", which would be in a line well before these logs appear. ... View more

What directory are you looking under for the keytab files? Ensure to look under the most recent role-related directory. Note that you don't need to access these directories directly and that you can find the same process files under CM at CM -> [Service] -> Instances tab -> [Instance of the service roles you are interested in] -> Processes tab -> In the files section, keytab should be available for download. ... View more

The "^A" is a visual shell representation of the ASCII hexadecimal byte "01".   You can run your command as so, making use of the actual byte representation than a visual helper:   sqoop export --connect jdbc:oracle:thin:olap7964/olap7964@192.168.2.135:1521:ORCLOBIA11G --username XXXXXXX --password XXXXXXX --export-dir /user/hive/warehouse/abc.db/sach_sport --table SACH_SPORT --fields-terminated-by $'\x01' -m 1 ... View more

Thank you for double checking! The "-56" is the right layout version for DNs, so that's not the issue. There is instead a bug in that we'd left the sorted field out during conversion of the BlockReportContext objects: https://github.com/cloudera/hadoop-common/blob/cdh5.7.1-release/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java#L2981-L2988 Internal report for this identified bug is CDH-40629, and the fix will be in the next update releases of 5.7.2 onwards and also in the 5.8.x series from 5.8.0 onwards. At present this shouldn't harm you, except the feature of HDFS-9260 won't really be in use and the perf. of NN would be the same as 5.6.x and lower. ... View more

Ensure your Hive MetaStore service is up and running. If you use packages: service hive-metastore status If you use Cloudera Manager: CM -> Hive -> Instances -> Hive MetaStore Server page ... View more

> Exact command cloudera gave is ADD JAR {{lib_dir}}/hive/lib/hive-contrib.jar; > 1)I am unable to find hive/lib directory on CDH 5 The {{lib_dir}} on CDH installed environments for Hive would either be /usr/lib/hive/ or /opt/cloudera/parcels/CDH/lib/hive/ (depending on packages or parcels being in use). ... View more