Hi @Garima Dosi If you look at the full yarn logs (i.e. 'yarn logs -applicationId <app id>'), do you see "Container killed on request" in the logs? The exit code may hold some clues. Best Regards, Craig 🙂 ... View more

Do you have the full stack trace? Also, what version of HDP/sqoop? Thanks ... View more

This JIRA has some info about why you need to set the property: https://issues.apache.org/jira/browse/SQOOP-2910 ... View more

Great to hear. Thanks for the feedback. ... View more

Set hive.exec.compress.output to false. Your cluster may be configured for compression by default, with the default codec. Alternatively, keep the compressed output but view it with the "hdfs dfs -text <file path>" command. ... View more

The Spark version will be displayed in the console log output... ... View more

A good way to sanity check Spark is to start Spark shell with YARN (spark-shell --master yarn) and run something like this: val x = sc.textFile("some hdfs path to a text file or directory of text files") x.count() This will basically do a distributed line count. If that looks good, another sanity check is for Hive integration. Run spark-sql (spark-sql --master yarn) and try to query a table that you know can be queried via Hive. ... View more

I noticed "Too many open files" so you might want to check your ulimit setting. Check here for guidance: https://community.hortonworks.com/questions/2029/best-practices-for-ulimits-number-of-open-file-dec.html ... View more