Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hue user cannot load data into Hive via Sqoop 1

avatar
Explorer

Hi All,

 

I am trying to load data from SQL Server into Hive in Hue using Sqoop 1. The following statement is working:

 

import --connect jdbc:sqlserver://xxx:1433;database=INFA_SOURCE --username infa_source --password B1source --table Personen -m 1

However when I add --hive-import it's not working any more. There are two jobs running and both have status succeeded, but workflow is in status killed. In the log file of first job I can see that there is a failure at the end, but it's not specified what exactly failed:

 

2018-01-09 10:59:10,544 [main] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: SELECT t.* FROM [Personen] AS t WHERE 1=0
2018-01-09 10:59:10,569 [main] INFO  org.apache.sqoop.hive.HiveImport  - Loading uploaded data into Hive

<<< Invocation of Sqoop command completed <<<

Hadoop Job IDs executed by Sqoop: job_1515423392261_0009

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

As the same statement works via command line, I think the issue is with Hue user that doesn't have access to /user/hive/warehouse folder, because of a sticky bit.

 

I found information about enabling security with Sentry, but I am not sure if that's really the issue and if this could be a solution?.

 

I have mainly two questions:

- Can I remove sticky bit to test the --hive-import with no issues and then bring it back?

- If the issue is with priviledges, is Sentry a possible solution?

 

Please let me know if you see here another problem or how else I could check it 🙂

 

Thanks

Anna

12 REPLIES 12

avatar
Champion
Hey is your Hue ini looks good with Hive configuration ?

avatar
Champion

@Annkaa My bad lost in the thread , couldnt respond to you quickly .

Looks like there is no metastore URI being provided in the ini file or the URI is wrong  . 

Based on the exception . 

Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
    at org.apache.hadoop.hive.metastore.HiveMe

Let me know if that helps 

avatar
New Contributor

yes @csguna I thinks thats the Issue. @Annkaa you can verify that part by running a small query in your Hive shell and Beeline shell separately. In your case the query should work in Hive and Beeline should promt "No current connection". As a matter of fact every external connectivity to hive metastore goes through beeline. That must be the issue as you are unable to accomplish it through Hue.