Community Articles
Find and share helpful community-sourced technical articles
Labels (1)

Problem:

When trying to distcp to AWS, this error is reported:

2015-12-03 09:50:01,132 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.s3a.S3AFileSystem could not be instantiated at java.util.ServiceLoader.fail(ServiceLoader.java:224)

and also:

Caused by: java.lang.ClassNotFoundException: com.amazonaws.event.ProgressListener

Solution:

In ambari, under yarn, configs please add: /usr/hdp/2.3.2.0-2950/hadoop/* and /usr/hdp/2.3.2.0-2950/hadoop/lib* to the yarn.application.classpath

Then under mapred, configs, do the same for mapreduce.application.classpath

restart all affected services and it should work.

548 Views
Don't have an account?
Version history
Revision #:
1 of 1
Last update:
‎12-03-2015 07:53 PM
Updated by:
 
Contributors
Top Kudoed Authors