Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

sqoop2 cannot start in CDH5.4.9

Highlighted

sqoop2 cannot start in CDH5.4.9

Rising Star

Hi all,

 

I was testing sqoop2 for the past few days and everything works fine, however sqoop2 suddenly die and unable to start since 2 days ago, I have not changed any settings on sqoop2, I checked the /var/log/sqoop2 catalina log it says:

 

Mar 14, 2016 2:06:17 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Mar 14, 2016 2:06:18 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:18 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 2661 ms
Mar 14, 2016 2:06:18 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Mar 14, 2016 2:06:18 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.44
Mar 14, 2016 2:06:18 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive sqoop.war
Mar 14, 2016 2:06:32 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ROOT
Mar 14, 2016 2:06:33 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:33 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 15277 ms
Mar 14, 2016 2:06:33 PM org.apache.catalina.core.StandardServer await
SEVERE: StandardServer.await: create[8005]:
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:404)
at org.apache.catalina.startup.Catalina.await(Catalina.java:676)
at org.apache.catalina.startup.Catalina.start(Catalina.java:628)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)

Mar 14, 2016 2:06:33 PM org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:34 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Mar 14, 2016 2:06:34 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/sqoop] registered the JDBC driver [org.apache.derby.jdbc.AutoloadedDriver40] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Mar 14, 2016 2:06:35 PM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-12000

 

 

Does anyone have any idea?

 

Thanks!

2 REPLIES 2

Re: sqoop2 cannot start in CDH5.4.9

Rising Star

I found another error from sqoop-cmf-sqoop-SQOOP_SERVER-master.hadoop.log

 

2016-03-14 19:06:08,598 INFO org.apache.sqoop.core.SqoopServer: Booting up Sqoop server
2016-03-14 19:06:08,623 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Starting config file poller thread
2016-03-14 19:06:11,044 INFO org.apache.sqoop.security.authentication.SimpleAuthenticationHandler: Using simple/pseudo authentication, principal [sqoop2]
2016-03-14 19:06:11,045 INFO org.apache.sqoop.security.AuthenticationManager: Authentication loaded.
2016-03-14 19:06:11,057 INFO org.apache.sqoop.security.AuthorizationManager: Authorization loaded.
2016-03-14 19:06:11,059 INFO org.apache.sqoop.audit.AuditLoggerManager: Begin audit logger manager initialization
2016-03-14 19:06:11,060 INFO org.apache.sqoop.audit.AuditLoggerManager: Audit logger manager initialized: OK
2016-03-14 19:06:11,065 INFO org.apache.sqoop.repository.RepositoryManager: Setting system properties: {derby.stream.error.file=/var/log/sqoop2/derbyrepo.log}
2016-03-14 19:06:11,086 INFO org.apache.sqoop.repository.JdbcRepositoryContext: [repo-ctx] handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler, conn-url=jdbc:derby:/var/lib/sqoop2/repository/db;create=true, driver=org.apache.derby.jdbc.EmbeddedDriver, user=sa, password=*****, jdbc-props={}, tx-isolation=READ_COMMITTED, max-conn=10
2016-03-14 19:06:11,960 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: DerbyRepositoryHandler initialized.
2016-03-14 19:06:11,971 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepositoryProvider initialized
2016-03-14 19:06:11,971 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepository initialized.
2016-03-14 19:06:14,570 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:14,570 INFO org.apache.sqoop.repository.RepositoryManager: Repository Manager initialized: OK
2016-03-14 19:06:14,627 INFO org.apache.sqoop.connector.ConnectorManager: Connector config urls: [jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-hdfs-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kafka-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kite-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties]
2016-03-14 19:06:15,218 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.jdbc.GenericJdbcConnector] initialized.
2016-03-14 19:06:15,322 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.hdfs.HdfsConnector] initialized.
2016-03-14 19:06:15,368 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.kafka.KafkaConnector] initialized.
2016-03-14 19:06:15,491 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.kite.KiteConnector] initialized.
2016-03-14 19:06:16,439 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:16,442 INFO org.apache.sqoop.connector.ConnectorManager: Connectors loaded: {kite-connector={kite-connector:org.apache.sqoop.connector.kite.KiteConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kite-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, kafka-connector={kafka-connector:org.apache.sqoop.connector.kafka.KafkaConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kafka-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, hdfs-connector={hdfs-connector:org.apache.sqoop.connector.hdfs.HdfsConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-hdfs-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, generic-jdbc-connector={generic-jdbc-connector:org.apache.sqoop.connector.jdbc.GenericJdbcConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}}
2016-03-14 19:06:16,489 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:16,489 INFO org.apache.sqoop.driver.Driver: Driver initialized: OK
2016-03-14 19:06:16,510 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Initializing Map-reduce Submission Engine
2016-03-14 19:06:16,993 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file yarn-site.xml
2016-03-14 19:06:16,993 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file core-site.xml
2016-03-14 19:06:16,995 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file mapred-site.xml
2016-03-14 19:06:16,995 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file hdfs-site.xml
2016-03-14 19:06:20,083 INFO org.apache.sqoop.driver.JobManager: Starting submission manager purge thread
2016-03-14 19:06:20,083 INFO org.apache.sqoop.driver.JobManager: Purging old submissions
2016-03-14 19:06:20,084 INFO org.apache.sqoop.driver.JobManager: Submission manager initialized: OK
2016-03-14 19:06:20,091 INFO org.apache.sqoop.core.SqoopServer: Sqoop server has successfully boot up
2016-03-14 19:06:20,093 INFO org.apache.sqoop.driver.JobManager: Starting submission manager update thread
2016-03-14 19:06:20,180 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:20,427 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:20,739 INFO org.apache.sqoop.handler.LinkRequestHandler: LinkRequestHandler initialized
2016-03-14 19:06:20,755 INFO org.apache.sqoop.handler.ConnectorRequestHandler: ConnectorRequestHandler initialized
2016-03-14 19:06:20,759 INFO org.apache.sqoop.handler.DriverRequestHandler: DriverRequestHandler initialized
2016-03-14 19:06:20,800 INFO org.apache.sqoop.handler.JobRequestHandler: JobRequestHandler initialized
2016-03-14 19:06:20,812 INFO org.apache.sqoop.handler.SubmissionRequestHandler: SubmissionRequestHandler initialized
2016-03-14 19:06:20,815 INFO org.apache.sqoop.handler.JobRequestHandler: JobRequestHandler initialized
2016-03-14 19:06:20,817 INFO org.apache.sqoop.handler.ConnectorRequestHandler: ConnectorRequestHandler initialized
2016-03-14 19:06:20,822 INFO org.apache.sqoop.handler.DriverRequestHandler: DriverRequestHandler initialized
2016-03-14 19:06:20,829 INFO org.apache.sqoop.handler.LinkRequestHandler: LinkRequestHandler initialized
2016-03-14 19:06:20,836 INFO org.apache.sqoop.handler.VersionRequestHandler: VersionRequestHandler initialized
2016-03-14 19:06:22,957 INFO org.apache.sqoop.core.SqoopServer: Shutting down Sqoop server
2016-03-14 19:06:22,957 INFO org.apache.sqoop.driver.JobManager: Ending submission manager purge thread
2016-03-14 19:06:22,957 INFO org.apache.sqoop.driver.JobManager: Ending submission manager update thread
2016-03-14 19:06:22,957 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Destroying Mapreduce Submission Engine
2016-03-14 19:06:23,124 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: Embedded Derby shutdown raised SQL STATE 45000 as expected.
2016-03-14 19:06:23,124 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: Deregistering JDBC driver
2016-03-14 19:06:23,124 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread
2016-03-14 19:06:23,132 INFO org.apache.sqoop.core.SqoopServer: Sqoop server has been correctly terminated

Re: sqoop2 cannot start in CDH5.4.9

Rising Star

Managed to solve it as there was a port conflict on port 8005, changed to 12001 in Cloudera Management Console fixed it!

Don't have an account?
Coming from Hortonworks? Activate your account here