Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Integration sQoop2 and huE 3.9

Integration sQoop2 and huE 3.9

Super Collaborator

Hi:

I added in the hue.ini the sqoop2 config this

  1. [sqoop]
  2. server_url=http://xxx.xx.xx.xx:12000/sqoop
  3. sqoop_conf_dir=/opt/sqoop-1.99.7-bin-hadoop200/conf

and in the sqoop2 node i have started the server with this config

  1. 2016-10-10 23:47:33,569 INFO [org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:744)] Started o.e.j.s.ServletContextHandler@3bbf6abe{/sqoop,null,AVAILABLE}
  2. 2016-10-10 23:47:33,577 INFO [org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:266)] Started ServerConnector@327fbda7{HTTP/1.1}{0.0.0.0:12000}
  3. 2016-10-10 23:47:33,577 INFO [org.eclipse.jetty.server.Server.doStart(Server.java:379)] Started @5039ms
  4. 2016-10-10 23:47:33,577 INFO [org.apache.sqoop.server.SqoopJettyServer.startServer(SqoopJettyServer.java:128)] Started Sqoop Jetty server.

I export HADOOP_COMMON_HOME=/usr/hdp/current/hadoop-client

  1. export HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
  2. export HADOOP_MAPRED_HOME=/usr/hdp/current/hadoop-mapreduce-client
  3. export HADOOP_YARN_HOME=/usr/hdp/current/hadoop-yarn-client
  4. export SQOOP_SERVER_EXTRA_LIB=/var/lib/sqoop2/

but from hue UI i get this error:

  1. Sqoop error: Could not get connectors.

Any suggestion??

6 REPLIES 6

Re: Integration soqop2 and huE 3.9

Is Sqoop2 working from the command line?

Note that Hue is not testing with Sqoop2 currently as it is not ready yet.

Re: Integration soqop2 and huE 3.9

Super Collaborator

My sqoop server is running but I can access via http, but its working via command line:

 

HTTP ERROR: 404

Problem accessing /sqoop/. Reason:

    Not Found
Powered by Jetty://
./sqoop2-shell
Setting conf dir: /usr/lib/sqoop/bin/../conf
Sqoop home directory: /usr/lib/sqoop
Sqoop Shell: Type 'help' or '\h' for help

sqoop:000> set server --host xxx.xx.xx.xx --port 12000 --webapp sqoop --url http://xxx.xx.xx.xx:12000/sqoop
--host, --port or --webapp option is ignored, because --url option is given.
Server is set successfully
sqoop:000> show version --all
client version:
Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
server version:
Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
API versions:
[v1]

 So, i dont know if some configuration is missing.

 

I think i need to set the host into the jetty config file

 

Re: Integration soqop2 and huE 3.9

Super Collaborator

Hi:

 

from CHROME if i type this http://xxx.xx.xx.xx:12000/sqoop/v1/connector/ i can see that:

{"message":"Connector:  doesn't exist","stack-trace":[{"file":"HandlerUtils.java","line":54,"class":"org.apache.sqoop.handler.HandlerUtils","method":"getConnectorFromConnectorName"},{"file":"ConnectorRequestHandler.java","line":70,"class":"org.apache.sqoop.handler.ConnectorRequestHandler","method":"handleEvent"},{"file":"ConnectorServlet.java","line":48,"class":"org.apache.sqoop.server.v1.ConnectorServlet","method":"handleGetRequest"},{"file":"SqoopProtocolServlet.java","line":50,"class":"org.apache.sqoop.server.SqoopProtocolServlet","method":"doGet"},{"file":"HttpServlet.java","line":687,"class":"javax.servlet.http.HttpServlet","method":"service"},{"file":"HttpServlet.java","line":790,"class":"javax.servlet.http.HttpServlet","method":"service"},{"file":"ServletHolder.java","line":808,"class":"org.eclipse.jetty.servlet.ServletHolder","method":"handle"},{"file":"ServletHandler.java","line":1669,"class":"org.eclipse.jetty.servlet.ServletHandler$CachedChain","method":"doFilter"},{"file":"AuthenticationFilter.java","line":614,"class":"org.apache.hadoop.security.authentication.server.AuthenticationFilter","method":"doFilter"},{"file":"DelegationTokenAuthenticationFilter.java","line":294,"class":"org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter","method":"doFilter"},{"file":"AuthenticationFilter.java","line":573,"class":"org.apache.hadoop.security.authentication.server.AuthenticationFilter","method":"doFilter"},{"file":"ServletHandler.java","line":1652,"class":"org.eclipse.jetty.servlet.ServletHandler$CachedChain","method":"doFilter"},{"file":"ServletHandler.java","line":585,"class":"org.eclipse.jetty.servlet.ServletHandler","method":"doHandle"},{"file":"ContextHandler.java","line":1127,"class":"org.eclipse.jetty.server.handler.ContextHandler","method":"doHandle"},{"file":"ServletHandler.java","line":515,"class":"org.eclipse.jetty.servlet.ServletHandler","method":"doScope"},{"file":"ContextHandler.java","line":1061,"class":"org.eclipse.jetty.server.handler.ContextHandler","method":"doScope"},{"file":"ScopedHandler.java","line":141,"class":"org.eclipse.jetty.server.handler.ScopedHandler","method":"handle"},{"file":"HandlerWrapper.java","line":97,"class":"org.eclipse.jetty.server.handler.HandlerWrapper","method":"handle"},{"file":"Server.java","line":499,"class":"org.eclipse.jetty.server.Server","method":"handle"},{"file":"HttpChannel.java","line":310,"class":"org.eclipse.jetty.server.HttpChannel","method":"handle"},{"file":"HttpConnection.java","line":257,"class":"org.eclipse.jetty.server.HttpConnection","method":"onFillable"},{"file":"AbstractConnection.java","line":540,"class":"org.eclipse.jetty.io.AbstractConnection$2","method":"run"},{"file":"ThreadPoolExecutor.java","line":1145,"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker"},{"file":"ThreadPoolExecutor.java","line":615,"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run"},{"file":"Thread.java","line":745,"class":"java.lang.Thread","method":"run"}],"error-code":"SERVER_0006","class":"org.apache.sqoop.common.SqoopException","error-code-message":"Entity requested doesn't exist"}

So the server is running.

please where hue is looking a connectors jars??

Re: Integration soqop2 and huE 3.9

Re: Integration soqop2 and huE 3.9

Super Collaborator
Hi:

I have Tried with sqoop 1.99.5 and Hue 3.11 and now it worked for me, so,
the problem is the sqoop version.
Highlighted

Re: Integration sQoop2 and huE 3.9

New Contributor
Has this problem been solved ? I encountered the same situation.
Don't have an account?
Coming from Hortonworks? Activate your account here