Integration sQoop2 and huE 3.9 [ Edited ]

[pacosoplas]

Hi:

I added in the hue.ini the sqoop2 config this

1. [sqoop]
2. server_url=http://xxx.xx.xx.xx:12000/sqoop
3. sqoop_conf_dir=/opt/sqoop-1.99.7-bin-hadoop200/conf

and in the sqoop2 node i have started the server with this config

1. 2016-10-10 23:47:33,569 INFO [org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:744)] Started o.e.j.s.ServletContextHandler@3bbf6abe{/sqoop,null,AVAILABLE}
2. 2016-10-10 23:47:33,577 INFO [org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:266)] Started ServerConnector@327fbda7{HTTP/1.1}{0.0.0.0:12000}
3. 2016-10-10 23:47:33,577 INFO [org.eclipse.jetty.server.Server.doStart(Server.java:379)] Started @5039ms
4. 2016-10-10 23:47:33,577 INFO [org.apache.sqoop.server.SqoopJettyServer.startServer(SqoopJettyServer.java:128)] Started Sqoop Jetty server.

I export HADOOP_COMMON_HOME=/usr/hdp/current/hadoop-client

1. export HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client
2. export HADOOP_MAPRED_HOME=/usr/hdp/current/hadoop-mapreduce-client
3. export HADOOP_YARN_HOME=/usr/hdp/current/hadoop-yarn-client
4. export SQOOP_SERVER_EXTRA_LIB=/var/lib/sqoop2/

but from hue UI i get this error:

1. Sqoop error: Could not get connectors.

Any suggestion??

[Romainr]

Is Sqoop2 working from the command line?

Note that Hue is not testing with Sqoop2 currently as it is not ready yet.

[pacosoplas]

My sqoop server is running but I can access via http, but its working via command line:

HTTP ERROR: 404

Problem accessing /sqoop/. Reason:

    Not Found
Powered by Jetty://

./sqoop2-shell
Setting conf dir: /usr/lib/sqoop/bin/../conf
Sqoop home directory: /usr/lib/sqoop
Sqoop Shell: Type 'help' or '\h' for help

sqoop:000> set server --host xxx.xx.xx.xx --port 12000 --webapp sqoop --url http://xxx.xx.xx.xx:12000/sqoop
--host, --port or --webapp option is ignored, because --url option is given.
Server is set successfully
sqoop:000> show version --all
client version:
 Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
 Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
server version:
 Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
 Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
API versions:
 [v1]

 So, i dont know if some configuration is missing.

I think i need to set the host into the jetty config file

[pacosoplas]

Hi:

from CHROME if i type this http://xxx.xx.xx.xx:12000/sqoop/v1/connector/ i can see that:

{"message":"Connector:  doesn't exist","stack-trace":[{"file":"HandlerUtils.java","line":54,"class":"org.apache.sqoop.handler.HandlerUtils","method":"getConnectorFromConnectorName"},{"file":"ConnectorRequestHandler.java","line":70,"class":"org.apache.sqoop.handler.ConnectorRequestHandler","method":"handleEvent"},{"file":"ConnectorServlet.java","line":48,"class":"org.apache.sqoop.server.v1.ConnectorServlet","method":"handleGetRequest"},{"file":"SqoopProtocolServlet.java","line":50,"class":"org.apache.sqoop.server.SqoopProtocolServlet","method":"doGet"},{"file":"HttpServlet.java","line":687,"class":"javax.servlet.http.HttpServlet","method":"service"},{"file":"HttpServlet.java","line":790,"class":"javax.servlet.http.HttpServlet","method":"service"},{"file":"ServletHolder.java","line":808,"class":"org.eclipse.jetty.servlet.ServletHolder","method":"handle"},{"file":"ServletHandler.java","line":1669,"class":"org.eclipse.jetty.servlet.ServletHandler$CachedChain","method":"doFilter"},{"file":"AuthenticationFilter.java","line":614,"class":"org.apache.hadoop.security.authentication.server.AuthenticationFilter","method":"doFilter"},{"file":"DelegationTokenAuthenticationFilter.java","line":294,"class":"org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter","method":"doFilter"},{"file":"AuthenticationFilter.java","line":573,"class":"org.apache.hadoop.security.authentication.server.AuthenticationFilter","method":"doFilter"},{"file":"ServletHandler.java","line":1652,"class":"org.eclipse.jetty.servlet.ServletHandler$CachedChain","method":"doFilter"},{"file":"ServletHandler.java","line":585,"class":"org.eclipse.jetty.servlet.ServletHandler","method":"doHandle"},{"file":"ContextHandler.java","line":1127,"class":"org.eclipse.jetty.server.handler.ContextHandler","method":"doHandle"},{"file":"ServletHandler.java","line":515,"class":"org.eclipse.jetty.servlet.ServletHandler","method":"doScope"},{"file":"ContextHandler.java","line":1061,"class":"org.eclipse.jetty.server.handler.ContextHandler","method":"doScope"},{"file":"ScopedHandler.java","line":141,"class":"org.eclipse.jetty.server.handler.ScopedHandler","method":"handle"},{"file":"HandlerWrapper.java","line":97,"class":"org.eclipse.jetty.server.handler.HandlerWrapper","method":"handle"},{"file":"Server.java","line":499,"class":"org.eclipse.jetty.server.Server","method":"handle"},{"file":"HttpChannel.java","line":310,"class":"org.eclipse.jetty.server.HttpChannel","method":"handle"},{"file":"HttpConnection.java","line":257,"class":"org.eclipse.jetty.server.HttpConnection","method":"onFillable"},{"file":"AbstractConnection.java","line":540,"class":"org.eclipse.jetty.io.AbstractConnection$2","method":"run"},{"file":"ThreadPoolExecutor.java","line":1145,"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker"},{"file":"ThreadPoolExecutor.java","line":615,"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run"},{"file":"Thread.java","line":745,"class":"java.lang.Thread","method":"run"}],"error-code":"SERVER_0006","class":"org.apache.sqoop.common.SqoopException","error-code-message":"Entity requested doesn't exist"}

So the server is running.

please where hue is looking a connectors jars??

[Romainr]

Created jira https://issues.cloudera.org/browse/HUE-5124

[pacosoplas]

Hi:

I have Tried with sqoop 1.99.5 and Hue 3.11 and now it worked for me, so,
the problem is the sqoop version.

[zhengw]

Has this problem been solved ? I encountered the same situation.