Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

CDH 5.3 and Sqoop2 startup and use error

avatar
Explorer

Hello.

 

I had a CDH 5.1.3 installation where I was able to use the Sqoop2 client to import data into Hive and HBase. After upgrading to CDH 5.3 I am unable to get neither the Sqoop2 service to start successfully, nor am I able to use the Sqoop2 client. What do I need to do to get this to work as it did before the upgrade?

 

The errors reported in /var/log/sqoop2/catalna log are as follows:

... ... ...

Jan 20, 2015 4:10:54 PM org.apache.tomcat.util.digester.Digester endElement
WARNING:   No rules found matching 'HTML/BODY'.
Jan 20, 2015 4:10:54 PM org.apache.tomcat.util.digester.Digester endElement
WARNING:   No rules found matching 'HTML'.
Jan 20, 2015 4:10:54 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 59 ms
Jan 20, 2015 4:10:54 PM org.apache.catalina.startup.Catalina start
SEVERE: Cannot start server. Server instance is not configured.

 

 

And, the Sqoop2 client throws an exception, and following is the output it displays:

15/01/20 19:58:13 FATAL conf.Configuration: error parsing conf core-default.xml
javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
        at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2375)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2337)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2254)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:861)
        at org.apache.sqoop.tool.SqoopTool.loadPluginsFromConfDir(SqoopTool.java:170)
        at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:140)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:208)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Exception in thread "main" java.lang.RuntimeException: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2493)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2337)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2254)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:861)
        at org.apache.sqoop.tool.SqoopTool.loadPluginsFromConfDir(SqoopTool.java:170)
        at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:140)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:208)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
        at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2375)
        ... 8 more

 

1 ACCEPTED SOLUTION

avatar
Explorer

For anyone else who may experience this:

The cause was a HSQLDB jar file present on the server outside of CDH installation and which was installed as part of another software. Removing this Jar file allowed Sqoop to import files from Sql Server (and MySql also). So problem solved.

View solution in original post

15 REPLIES 15

avatar
Explorer
Thanks for your reply.

I reinstalled the CDH 5.3.0 via Cloudera Manager, and found log under:
/var/log/sqoop2/sqoop-cmf-sqoop-SQOOP_SERVER-slc05hwl.us.oracle.com.log.out

error saying:


2015-01-30 09:03:04,822 ERROR org.apache.sqoop.server.ServerInitializer: Sqoop server failed to start
java.lang.RuntimeException: Failure in server initialization

.....

Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
......

Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
.....
Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true
at java.sql.DriverManager.getConnection(DriverManager.java:596)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:78)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)
... 31 more



And I added derby-10.11.1.1.jar to /var/lib/sqoop2/

then re-ran the Cluster Settings and error came out as you did mentioned


Do you have any idea about this?

Thank you very much

avatar
Explorer

Not exactly same but I had similar installation issue with Sqoop2. In my case I had to reinstall the OS and get past the issue. It was the most time saving solution for me after spending good amount of time trying to analyze the problem.

avatar
Explorer
well , that is lucky for you.

I re-imaged the server and issue still exists... no idea.


Any way, thank you ~~

avatar
New Contributor

I've upgraded CDH from 5.2.1 to 5.3.2 and got the same errors as in the first message. Sqoop2 service could not be started.

 

After several hours of investigations I've found out that new sqoop2 is not compatible with xercesImpl.jar from libxerces2-java (Debian Wheezy). It cannot be removed because other packages depend on it.

 

The workaround is to remove /usr/share/java/*.jar from common.loader=... in /var/lib/sqoop2/tomcat-deployment/conf/catalina.properties.

avatar
Explorer

Many Thanks, it worked for me!

I followed Yugune's reply.

 

The workaround is to remove /usr/share/java/*.jar from common.loader=... in /var/lib/sqoop2/tomcat-deployment/conf/catalina.properties.

avatar
Explorer
Encountered this myself on Red Hat Enterprise Linux Server release 7.2 (Maipo). Seems to be a conflict with this Red Hat's release of xceres-j2.