question Re: Sqoop Error: Could not start job in Support Questions

Sqoop Error: Could not start job

cdhtsk — Fri, 16 Sep 2022 10:39:01 GMT

Hi,

After the upgrade CDH from 5.7.0 to 5.8.0 I can't start any sqoop job in Hue: Sqoop transfer.

Hue only displays: Error: Could not start job.

Could you please help me what is the problem?

In /var/log/sqoop2/localhost.2016-09-13.log is an error:

Sep 13, 2016 11:51:38 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet v1.JobServlet threw exception
java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:255)
at org.apache.sqoop.driver.JobManager.start(JobManager.java:288)
at org.apache.sqoop.handler.JobRequestHandler.startJob(JobRequestHandler.java:380)
at org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:116)
at org.apache.sqoop.server.v1.JobServlet.handlePutRequest(JobServlet.java:96)
at org.apache.sqoop.server.SqoopProtocolServlet.doPut(SqoopProtocolServlet.java:79)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)

Re: Sqoop Error: Could not start job

MuraliAdiReddy — Tue, 13 Sep 2016 19:47:44 GMT

Hi,

Looks like one of the required jar jackson-mapper-asl.jar is missing under /var/lib/sqoop2/lib .

Can you check ls -l /var/lib/sqoop2/lib/jackson*.jar to make sure the required jar is avaialbe in the specified path.

Re: Sqoop Error: Could not start job

cdhtsk — Wed, 14 Sep 2016 10:11:47 GMT

Hi,

thanks for your response.

I don't have any lib directory or jar in /var/lib/sqoop2/.

/var/lib/sqoop2# ls -l /var/lib/sqoop2/lib/jackson*.jar
ls: cannot access /var/lib/sqoop2/lib/jackson*.jar: No such file or directory

/var/lib/sqoop2# ls /var/lib/sqoop2
ojdbc6.jar postgresql-9.0-801.jdbc4.jar tomcat-deployment

Re: Sqoop Error: Could not start job

tolwi — Tue, 20 Sep 2016 14:08:46 GMT

having the same problem .. is there any fix or workaround ?

Re: Sqoop Error: Could not start job

JonRobinson — Tue, 20 Sep 2016 15:07:06 GMT

I've recently started working on a quickstart VM image of CDH5.8 and I'm also getting same class not found error for JsonMappingException class/

I can see the jar it should be contained within included in the classpath of the Sqoop2 server:

/usr/lib/sqoop2/client-lib/jackson-mapper-asl-1.8.8.jar

The right version of that jar file exists in the location specified in the classpath:

[cloudera@quickstart client-lib]$ pwd

/usr/lib/sqoop2/client-lib

[cloudera@quickstart client-lib]$ ls -ltra jackson-mapper-asl-1.8.8.jar

-rwxr-xr-x 1 root root 668564 Jun 16 12:36 jackson-mapper-asl-1.8.8.jar

If i check the contents of the jar file i can see the class is included in there:

[cloudera@quickstart client-lib]$ jar tf jackson-mapper-asl-1.8.8.jar | grep -i JsonMappingException.class

org/codehaus/jackson/map/JsonMappingException.class

org/codehaus/jackson/map/RuntimeJsonMappingException.class

And I can see the jar file in the running process:

[cloudera@quickstart client-lib]$ lsof -p 7621 | grep jackson-mapper-asl-1.8.8.jar

java 7621 cloudera mem REG 253,0 668564 1056186 /usr/lib/sqoop2/client-lib/jackson-mapper-asl-1.8.8.jar

java 7621 cloudera 40r REG 253,0 668564 1056186 /usr/lib/sqoop2/client-lib/jackson-mapper-asl-1.8.8.jar

Any ideas to resolve would be appreciated.

Re: Sqoop Error: Could not start job

tolwi — Tue, 20 Sep 2016 15:18:52 GMT

cd /usr/lib/hadoop/client
sudo ln -s /usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar .
sudo ln -s /usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar .

Re: Sqoop Error: Could not start job

luck — Wed, 21 Sep 2016 04:48:12 GMT

It does't work for me.

Re: Sqoop Error: Could not start job

tolwi — Wed, 21 Sep 2016 07:58:30 GMT

My prev sollution was for the case when sqoop installed from packages.

When sqoop installed from parcels:

cd /opt/cloudera/parcels/CDH/lib/hadoop/client
sudo ln -s ../../hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar .
sudo ln -s ../../hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar .

Re: Sqoop Error: Could not start job

luck — Wed, 21 Sep 2016 08:06:49 GMT

[root@hadoop-controller-1 lib]# cd /opt/cloudera/parcels/CDH/lib/hadoop/client
[root@hadoop-controller-1 client]# sudo ln -s ../../hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar .
[root@hadoop-controller-1 client]# sudo ln -s ../../hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar .
[root@hadoop-controller-1 client]# exit
exit
[centos@hadoop-controller-1 ~]$ sqoop2
Sqoop home directory: /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/sqoop2
Sqoop Shell: Type 'help' or '\h' for help.

sqoop:000> start job -j 1
0    [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception has occurred during processing command
Exception: org.apache.sqoop.common.SqoopException Message: CLIENT_0001:Server has returned exception - <html><head><title>Apache Tomcat/6.0.44 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - Servlet execution threw an exception</h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u>Servlet execution threw an exception</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>javax.servlet.ServletException: Servlet execution threw an exception
	org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
	org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
	org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
</pre></p><p><b>root cause</b> <pre>java.lang.NoClassDefFoundError: org/codehaus/jackson/map/JsonMappingException
	org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
	org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
	org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
	org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:255)
	org.apache.sqoop.driver.JobManager.start(JobManager.java:288)
	org.apache.sqoop.handler.JobRequestHandler.startJob(JobRequestHandler.java:380)
	org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:116)
	org.apache.sqoop.server.v1.JobServlet.handlePutRequest(JobServlet.java:96)
	org.apache.sqoop.server.SqoopProtocolServlet.doPut(SqoopProtocolServlet.java:79)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
	org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
	org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
	org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
</pre></p><p><b>root cause</b> <pre>java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingException
	java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	java.security.AccessController.doPrivileged(Native Method)
	java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
	org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
	org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
	org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.ja

...

I did exactly what you told me to do, but still faild.

Re: Sqoop Error: Could not start job

tolwi — Thu, 22 Sep 2016 04:38:38 GMT

did you restart sqoop before trying to run the job?

Re: Sqoop Error: Could not start job

luck — Fri, 23 Sep 2016 05:46:05 GMT

suddenly worked, stranger. I did not restart the sqoop.

Re: Sqoop Error: Could not start job

cdhtsk — Tue, 27 Sep 2016 13:02:19 GMT

Worked! Thank You! 🙂

Re: Sqoop Error: Could not start job

CannedFish — Fri, 09 Jun 2017 06:54:00 GMT

Thanks for your solution.