Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Could not get connectors ( Hue + Sqoop2 ) on secured cluster

Highlighted

Could not get connectors ( Hue + Sqoop2 ) on secured cluster

New Contributor

Hello all, 
 
I've been trying to access the "Sqoop Transfer" tool in hue on my secured cluster but i keep getting an error when displaying the pages "Error : Could not get connectors".  My current setup is a CDH5.8 secured with Kerberos on which I installed the SQOOP2 server ( through the interface not manually ).
 
When looking at the logs of Hue, the error comes from the call to the sqoop server (GET mysqoopserver:12000/sqoop/v1/connectors ) which answers "Status 401 - Authentication required".  After searching a lot, It looks like Hue, at startup, loads the sqoop.properties. But instead of loading the file Generated by CDH in ( /var/run/cloudera-scm-agent/process/XXX-sqoop-SQOOP_SERVER/sqoop.properties it loads the default /etc/sqoop2/conf/sqoop.properties ) leading to Hue sending unauthenticated requests to my SQOOP2 server. If I replace the /etc/sqoop2/conf/sqoop.properties by a symlink to the one deployed in /var/run/... and chmod the file to grant read access to hue, the page displays correctly. Unfortunately, as this folder changes everytime, this is not a valid solution.
 
Also, after temporary fixing this problem, I created 2 links in the interface ( One from an Oracle Database and One to my HDFS ), I created a job, But When I hit Save and Run, I receive a "Could not start job" with following error in the Hue logs : 

 

[25/Aug/2016 03:02:40 -0700] exception ERROR <html><head><title>Apache Tomcat/6.0.44 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - Servlet execution threw an exception</h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u>Servlet execution threw an exception</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>javax.servlet.ServletException: Servlet execution threw an exception
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
</pre></p><p><b>root cause</b> <pre>java.lang.NoClassDefFoundError: org/codehaus/jackson/map/JsonMappingException
org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:255)
org.apache.sqoop.driver.JobManager.start(JobManager.java:288)
org.apache.sqoop.handler.JobRequestHandler.startJob(JobRequestHandler.java:380)
org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:116)
org.apache.sqoop.server.v1.JobServlet.handlePutRequest(JobServlet.java:96)
org.apache.sqoop.server.SqoopProtocolServlet.doPut(SqoopProtocolServlet.java:79)
javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
</pre></p><p><b>root cause</b> <pre>java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingException
java.net.URLClassLoader$1.run(URLClassLoader.java:366)
java.net.URLClassLoader$1.run(URLClassLoader.java:355)
java.security.AccessController.doPrivileged(Native Method)
java.net.URLClassLoader.findClass(URLClassLoader.java:354)
java.lang.ClassLoader.loadClass(ClassLoader.java:425)
java.lang.ClassLoader.loadClass(ClassLoader.java:358)
org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submitToCluster(MapreduceSubmissionEngine.java:274)
org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.submit(MapreduceSubmissionEngine.java:255)
org.apache.sqoop.driver.JobManager.start(JobManager.java:288)
org.apache.sqoop.handler.JobRequestHandler.startJob(JobRequestHandler.java:380)
org.apache.sqoop.handler.JobRequestHandler.handleEvent(JobRequestHandler.java:116)
org.apache.sqoop.server.v1.JobServlet.handlePutRequest(JobServlet.java:96)
org.apache.sqoop.server.SqoopProtocolServlet.doPut(SqoopProtocolServlet.java:79)
javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:622)
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:291)
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
</pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/6.0.44 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.44</h3></body></html>
[25/Aug/2016 03:02:40 -0700] exception ERROR Could not start job.
[25/Aug/2016 03:02:40 -0700] kerberos_ ERROR handle_mutual_auth(): Mutual authentication unavailable on 500 response
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG handle_mutual_auth(): Handling: 500
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG handle_response(): returning <Response [500]>
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG handle_401(): returning <Response [500]>
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG authenticate_user(): returning <Response [500]>
[25/Aug/2016 03:02:40 -0700] connectionpool DEBUG "PUT /sqoop/v1/job/1/start HTTP/1.1" 500 4653
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG authenticate_user(): Authorization header: Negotiate YIICigYJKoZIhvcSAQICAQBuggJ5MIICdaADAgEFoQMCAQ6iBwMFACAAAACjggF9YYIBeTCCAXWgAwIBBaEKGwhDTE9VREVSQaImMCSgAwIBA6EdMBsbBEhUVFAbE3F1aWNrc3RhcnQuY2xvdWRlcmGjggE4MIIBNKADAgESoQMCAQKiggEmBIIBIkQaSlB8Pj98yfY9kfLkjSmTJDGKMy4zxK4Qoprba4kH5XR6ztK1Ypbctn4H+MsVuLt3XNq8Mjh7OOZ+qcWcxT2l0NOhy0dGGXVtKv0CFLObAvuUgsyTnnmnwYuYik/vypcS6qExG1cv9i1adiTYpRNG8cL1yiSmxWlj0YmzWEt4RiQeJ9B2VlHYAs2dAnTI+CWR3ROoX52efpaxg5EmtCLR4pjGF6jblxPKDEhv3wQ0GQ7QHX7FPB3zsjYGFC9HyHryVlr+A70/PW6LjnRtFjoSH9eb9xMEAMXbOj2ILBbfd9O15KKG/LJhYpOPtW2SXYQvoweFifUMcxKadcl9lYz9LHvisMk04i69V0Gx2w5AxJdGU3hOieUA/I5+mNzWiKDvpIHeMIHboAMCARKigdMEgdBgVIj4btQWifueF5K9puo08bjCtkecM51/8h2DOJ0A35SZfybsYsrmKBMCVUGRZA59gbKHEfcFlk9iO+Vt7tTOhaDiomV4yyt06vQp15TCTm5jEiB7NKRSIsI7gYU4RLYV8QXbmWVa5ynlOwsunnpj49REjuqI3WZMAjAdyi48/Bd0kKwGeieGA/QWHF2axYfdc7PNZeGrnm8omE0Vg0CQVrbLToA21DRxH67reLTMLe7Zom72Wqu3FvrmvVyWiigZ2oGs71NXmECie84V+r/B
[25/Aug/2016 03:02:40 -0700] kerberos_ DEBUG handle_401(): Handling: 401
[25/Aug/2016 03:02:40 -0700] connectionpool DEBUG "PUT /sqoop/v1/job/1/start HTTP/1.1" 401 997

 


Has someone had the same problem ? Did I screwd up something in my configuration ?
 
I'd appreciate any input
 
Regards

2 REPLIES 2
Highlighted

Re: Could not get connectors ( Hue + Sqoop2 ) on secured cluster

New Contributor

After more testing, I've come to the conclusion that it's a bug with the CDH5.8.0

 

I have 4 Clusters :

2 clusters on CDH5.7.0 for which it works perfectly ( One installed with packages, One with parcels )

2 clusters on CDH5.8.0 with the same Error ( also one installed with packages, One with parcels )

 

I just took one of my 5.7.0 Cluster who was working fine upgraded it to 5.8.0 and now it doesn't work anymore.

 

I'll just hope it will be fixed in the next release and do it in command line until then

 

L

 

Highlighted

Re: Could not get connectors ( Hue Sqoop2 ) on secured cluster

I created a CM jira for having Hue get the sqoop.properties automatically.

About the second error, it seems more between Sqoop and MapReduce. Ask
there?
http://community.cloudera.com/t5/Data-Ingestion-Integration/bd-p/SqoopFlume

Sqoop2 in Hue will get some robutness when Sqoop2 gets GA (~ end of next
year AFAIK)
Don't have an account?
Coming from Hortonworks? Activate your account here