Reply
New Contributor
Posts: 1
Registered: ‎05-09-2018

Sqoop Exception : java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem

[ Edited ]

I installed Hadoop 3.0.0 and Sqoop 1.4.6, I started all hadoop services and while working with sqoop import command i am receiving following java exception,

CHNBND110:libexec GVB$ sqoop import --connect jdbc:mysql://localhost/metastore --username root -P --table emp --m 1;
2018-05-18 11:25:31,790 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Enter password: 
2018-05-18 11:25:35,556 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
2018-05-18 11:25:35,556 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hive/2.3.1/libexec/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Fri May 18 11:25:36 IST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2018-05-18 11:25:37,061 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
2018-05-18 11:25:37,112 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
2018-05-18 11:25:37,135 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/Cellar/Hadoop/3.0.0
Note: /tmp/sqoop-GVB/compile/f12b2451001b1996c5a961ba73dbfccc/emp.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2018-05-18 11:25:43,997 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-GVB/compile/f12b2451001b1996c5a961ba73dbfccc/emp.jar
2018-05-18 11:25:44,060 WARN manager.MySQLManager: It looks like you are importing from mysql.
2018-05-18 11:25:44,060 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
2018-05-18 11:25:44,061 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
2018-05-18 11:25:44,061 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
2018-05-18 11:25:44,067 INFO mapreduce.ImportJobBase: Beginning import of emp
2018-05-18 11:25:44,591 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-05-18 11:25:44,638 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.DistributedFileSystem could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2558)
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2569)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2586)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(FileOutputFormat.java:160)
    at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:156)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
    at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
    at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/GlobalStorageStatistics$StorageStatisticsProvider
    at java.lang.Class.getDeclaredConstructors0(Native Method)
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
    at java.lang.Class.getConstructor0(Class.java:3075)
    at java.lang.Class.newInstance(Class.java:412)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
    ... 25 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.GlobalStorageStatistics$StorageStatisticsProvider
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 30 more 

 

Please let me know how to resolve this exception. Is my sqoop compatible with hadoop version.

Hadoop : 3.0.0, Hive : 2.3.1, Sqoop : 1.4.6.

 

Announcements