Reply
Contributor
Posts: 48
Registered: ‎05-30-2015

sqoop2 cannot start in CDH5.4.9

[ Edited ]

Hi all,

 

I was testing sqoop2 for the past few days and everything works fine, however sqoop2 suddenly die and unable to start since 2 days ago, I have not changed any settings on sqoop2, I checked the /var/log/sqoop2 catalina log it says:

 

Mar 14, 2016 2:06:17 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Mar 14, 2016 2:06:18 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:18 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 2661 ms
Mar 14, 2016 2:06:18 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Mar 14, 2016 2:06:18 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.44
Mar 14, 2016 2:06:18 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive sqoop.war
Mar 14, 2016 2:06:32 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ROOT
Mar 14, 2016 2:06:33 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:33 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 15277 ms
Mar 14, 2016 2:06:33 PM org.apache.catalina.core.StandardServer await
SEVERE: StandardServer.await: create[8005]:
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:404)
at org.apache.catalina.startup.Catalina.await(Catalina.java:676)
at org.apache.catalina.startup.Catalina.start(Catalina.java:628)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)

Mar 14, 2016 2:06:33 PM org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-12000
Mar 14, 2016 2:06:34 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Mar 14, 2016 2:06:34 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/sqoop] registered the JDBC driver [org.apache.derby.jdbc.AutoloadedDriver40] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Mar 14, 2016 2:06:35 PM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-12000

 

 

Does anyone have any idea?

 

Thanks!

Highlighted
Contributor
Posts: 48
Registered: ‎05-30-2015

Re: sqoop2 cannot start in CDH5.4.9

[ Edited ]

I found another error from sqoop-cmf-sqoop-SQOOP_SERVER-master.hadoop.log

 

2016-03-14 19:06:08,598 INFO org.apache.sqoop.core.SqoopServer: Booting up Sqoop server
2016-03-14 19:06:08,623 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Starting config file poller thread
2016-03-14 19:06:11,044 INFO org.apache.sqoop.security.authentication.SimpleAuthenticationHandler: Using simple/pseudo authentication, principal [sqoop2]
2016-03-14 19:06:11,045 INFO org.apache.sqoop.security.AuthenticationManager: Authentication loaded.
2016-03-14 19:06:11,057 INFO org.apache.sqoop.security.AuthorizationManager: Authorization loaded.
2016-03-14 19:06:11,059 INFO org.apache.sqoop.audit.AuditLoggerManager: Begin audit logger manager initialization
2016-03-14 19:06:11,060 INFO org.apache.sqoop.audit.AuditLoggerManager: Audit logger manager initialized: OK
2016-03-14 19:06:11,065 INFO org.apache.sqoop.repository.RepositoryManager: Setting system properties: {derby.stream.error.file=/var/log/sqoop2/derbyrepo.log}
2016-03-14 19:06:11,086 INFO org.apache.sqoop.repository.JdbcRepositoryContext: [repo-ctx] handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler, conn-url=jdbc:derby:/var/lib/sqoop2/repository/db;create=true, driver=org.apache.derby.jdbc.EmbeddedDriver, user=sa, password=*****, jdbc-props={}, tx-isolation=READ_COMMITTED, max-conn=10
2016-03-14 19:06:11,960 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: DerbyRepositoryHandler initialized.
2016-03-14 19:06:11,971 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepositoryProvider initialized
2016-03-14 19:06:11,971 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepository initialized.
2016-03-14 19:06:14,570 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:14,570 INFO org.apache.sqoop.repository.RepositoryManager: Repository Manager initialized: OK
2016-03-14 19:06:14,627 INFO org.apache.sqoop.connector.ConnectorManager: Connector config urls: [jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-hdfs-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kafka-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties, jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kite-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties]
2016-03-14 19:06:15,218 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.jdbc.GenericJdbcConnector] initialized.
2016-03-14 19:06:15,322 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.hdfs.HdfsConnector] initialized.
2016-03-14 19:06:15,368 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.kafka.KafkaConnector] initialized.
2016-03-14 19:06:15,491 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.kite.KiteConnector] initialized.
2016-03-14 19:06:16,439 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:16,442 INFO org.apache.sqoop.connector.ConnectorManager: Connectors loaded: {kite-connector={kite-connector:org.apache.sqoop.connector.kite.KiteConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kite-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, kafka-connector={kafka-connector:org.apache.sqoop.connector.kafka.KafkaConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-kafka-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, hdfs-connector={hdfs-connector:org.apache.sqoop.connector.hdfs.HdfsConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-hdfs-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}, generic-jdbc-connector={generic-jdbc-connector:org.apache.sqoop.connector.jdbc.GenericJdbcConnector:jar:file:/var/lib/sqoop2/tomcat-deployment/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.5-cdh5.4.9.jar!/sqoopconnector.properties}}
2016-03-14 19:06:16,489 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:16,489 INFO org.apache.sqoop.driver.Driver: Driver initialized: OK
2016-03-14 19:06:16,510 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Initializing Map-reduce Submission Engine
2016-03-14 19:06:16,993 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file yarn-site.xml
2016-03-14 19:06:16,993 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file core-site.xml
2016-03-14 19:06:16,995 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file mapred-site.xml
2016-03-14 19:06:16,995 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Found hadoop configuration file hdfs-site.xml
2016-03-14 19:06:20,083 INFO org.apache.sqoop.driver.JobManager: Starting submission manager purge thread
2016-03-14 19:06:20,083 INFO org.apache.sqoop.driver.JobManager: Purging old submissions
2016-03-14 19:06:20,084 INFO org.apache.sqoop.driver.JobManager: Submission manager initialized: OK
2016-03-14 19:06:20,091 INFO org.apache.sqoop.core.SqoopServer: Sqoop server has successfully boot up
2016-03-14 19:06:20,093 INFO org.apache.sqoop.driver.JobManager: Starting submission manager update thread
2016-03-14 19:06:20,180 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:20,427 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit
2016-03-14 19:06:20,739 INFO org.apache.sqoop.handler.LinkRequestHandler: LinkRequestHandler initialized
2016-03-14 19:06:20,755 INFO org.apache.sqoop.handler.ConnectorRequestHandler: ConnectorRequestHandler initialized
2016-03-14 19:06:20,759 INFO org.apache.sqoop.handler.DriverRequestHandler: DriverRequestHandler initialized
2016-03-14 19:06:20,800 INFO org.apache.sqoop.handler.JobRequestHandler: JobRequestHandler initialized
2016-03-14 19:06:20,812 INFO org.apache.sqoop.handler.SubmissionRequestHandler: SubmissionRequestHandler initialized
2016-03-14 19:06:20,815 INFO org.apache.sqoop.handler.JobRequestHandler: JobRequestHandler initialized
2016-03-14 19:06:20,817 INFO org.apache.sqoop.handler.ConnectorRequestHandler: ConnectorRequestHandler initialized
2016-03-14 19:06:20,822 INFO org.apache.sqoop.handler.DriverRequestHandler: DriverRequestHandler initialized
2016-03-14 19:06:20,829 INFO org.apache.sqoop.handler.LinkRequestHandler: LinkRequestHandler initialized
2016-03-14 19:06:20,836 INFO org.apache.sqoop.handler.VersionRequestHandler: VersionRequestHandler initialized
2016-03-14 19:06:22,957 INFO org.apache.sqoop.core.SqoopServer: Shutting down Sqoop server
2016-03-14 19:06:22,957 INFO org.apache.sqoop.driver.JobManager: Ending submission manager purge thread
2016-03-14 19:06:22,957 INFO org.apache.sqoop.driver.JobManager: Ending submission manager update thread
2016-03-14 19:06:22,957 INFO org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine: Destroying Mapreduce Submission Engine
2016-03-14 19:06:23,124 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: Embedded Derby shutdown raised SQL STATE 45000 as expected.
2016-03-14 19:06:23,124 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: Deregistering JDBC driver
2016-03-14 19:06:23,124 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread
2016-03-14 19:06:23,132 INFO org.apache.sqoop.core.SqoopServer: Sqoop server has been correctly terminated

Contributor
Posts: 48
Registered: ‎05-30-2015

Re: sqoop2 cannot start in CDH5.4.9

Managed to solve it as there was a port conflict on port 8005, changed to 12001 in Cloudera Management Console fixed it!