Reply
New Contributor
Posts: 3
Registered: ‎04-09-2016

after sqoop hive-import, tables not listed

environment:  standalone mac mini running OS/X el capitan 10.11.2

hive:  1.2.1

sqoop:  1.4.6

hadoop:  2.7.1

 

hello everyone,

 

i am attempting to import a database table into hive using sqoop.  i see the data files in HDFS, but do not see the new table after running 'show tables' in hive.  after i manually give the create table statement (which appears in the output of sqoop --verbose), then i can see the table and select from it.

 

i have hive configured to use derby as its metastore, using a well-known directory:

 

  <property>

    <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true</value>

    <description>JDBC connect string for a JDBC metastore</description>

  </property>

 

i think the problem is that sqoop cannot interact with the hive metastore.  every time i run the sqoop command:

 

sqoop import --verbose --connect jdbc:mysql://localhost/mylocaldb --username ..... --password ..... --table mytable  --hive-import

 

it leaves a directory called metastore_db in the current directory.  so i think the fix is to get sqoop to use the hive metastore, instead of doing whatever it's doing.  setting HIVE_CONF_DIR to $HIVE_HOME/conf had no effect.

 

am i on the right track?  how can i get this to work?

 

many thanks,

d

 

 

New Contributor
Posts: 1
Registered: ‎07-20-2016

Re: after sqoop hive-import, tables not listed

The solution that I kind of stumbled upon which worked for me is to put hive-site.xml in the folder where you are running the sqoop command. Similiar approach should work from oozie as well.

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: after sqoop hive-import, tables not listed

Sqoop will internally trigger Hive CLI command, so you will need to have proper hive-site.xml for Hive CLI to function. Can you please confirm that you can create tables when running Hive CLI by itself? (Not Beeline).

 

The Sqoop's --verbose output should give you some hint on whether table creation failed or succeeded, can you put the full output here for review?

 

Thanks

New Contributor
Posts: 3
Registered: ‎05-03-2017

Re: after sqoop hive-import, tables not listed

I have the same issue but I am running my sqoop command as part of an oozie workflow in HUE.  My job finishes successfully but the table does not get created under the database that I specify or under the default database.  I can see the files that get imported but they are under my users default directory.

 

Any suggestions ?

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: after sqoop hive-import, tables not listed

What happens when you run exact same sqoop command manually from command line? Do you get the exact same result? What does the verbose output tell you?

New Contributor
Posts: 3
Registered: ‎05-03-2017

Re: after sqoop hive-import, tables not listed

Sadly I dont think the Oracle jdbc driver is in place for me to try that which is why I used HUE in the first place.  I'll see if someone can install it for me.

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: after sqoop hive-import, tables not listed

Please also enabled verbose logging for sqoop action in Oozie, and get the full output of the sqoop command when running in Oozie, this can help us understand how the job was run inside Oozie anc compare it with when running by itself.

Announcements
New solutions