Reply
New Contributor
Posts: 1
Registered: ‎05-09-2018

Sqoop Exception" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem"

[ Edited ]

I installed Hadoop 3.0.0 and Sqoop 1.4.6, I started all hadoop services and while working with sqoop import command i am receiving following java exception,

 

CHND:Desktop GKM$ sqoop import --connect "jdbc:mysql://localhost/metastore" --username hivelogin -P --table emp -target-dir /user/hive/Warehouse -m 1
2018-05-28 15:28:05,667 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Enter password:
2018-05-28 15:28:08,432 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
2018-05-28 15:28:08,432 INFO tool.CodeGenTool: Beginning code generation
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hive/2.3.1/libexec/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hbase/1.2.6/libexec/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Mon May 28 15:28:09 IST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2018-05-28 15:28:09,934 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
2018-05-28 15:28:09,951 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `emp` AS t LIMIT 1
2018-05-28 15:28:09,958 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/Cellar/Hadoop/3.0.0
Note: /tmp/sqoop-gkm/compile/f337db0cb3ee1720a96571a3db70df1b/emp.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2018-05-28 15:28:11,845 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-gkm/compile/f337db0cb3ee1720a96571a3db70df1b/emp.jar
2018-05-28 15:28:11,861 WARN manager.MySQLManager: It looks like you are importing from mysql.
2018-05-28 15:28:11,861 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
2018-05-28 15:28:11,861 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
2018-05-28 15:28:11,862 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
2018-05-28 15:28:11,868 INFO mapreduce.ImportJobBase: Beginning import of emp
2018-05-28 15:28:12,049 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-05-28 15:28:12,069 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.DistributedFileSystem could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2558)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2569)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2586)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:352)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(FileOutputFormat.java:160)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:156)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/GlobalStorageStatistics$StorageStatisticsProvider
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
... 25 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.GlobalStorageStatistics$StorageStatisticsProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 30 more

Please let me know how to resolve this exception. Is my sqoop compatible with hadoop version.

Hadoop : 3.0.0, Hive : 2.3.1, Sqoop : 1.4.6.

Announcements