Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

after sqoop hive-import, tables not listed

after sqoop hive-import, tables not listed

New Contributor

environment:  standalone mac mini running OS/X el capitan 10.11.2

hive:  1.2.1

sqoop:  1.4.6

hadoop:  2.7.1

 

hello everyone,

 

i am attempting to import a database table into hive using sqoop.  i see the data files in HDFS, but do not see the new table after running 'show tables' in hive.  after i manually give the create table statement (which appears in the output of sqoop --verbose), then i can see the table and select from it.

 

i have hive configured to use derby as its metastore, using a well-known directory:

 

  <property>

    <name>javax.jdo.option.ConnectionURL</name>

    <value>jdbc:derby:;databaseName=/var/lib/hive/metastore/metastore_db;create=true</value>

    <description>JDBC connect string for a JDBC metastore</description>

  </property>

 

i think the problem is that sqoop cannot interact with the hive metastore.  every time i run the sqoop command:

 

sqoop import --verbose --connect jdbc:mysql://localhost/mylocaldb --username ..... --password ..... --table mytable  --hive-import

 

it leaves a directory called metastore_db in the current directory.  so i think the fix is to get sqoop to use the hive metastore, instead of doing whatever it's doing.  setting HIVE_CONF_DIR to $HIVE_HOME/conf had no effect.

 

am i on the right track?  how can i get this to work?

 

many thanks,

d

 

 

6 REPLIES 6

Re: after sqoop hive-import, tables not listed

New Contributor

The solution that I kind of stumbled upon which worked for me is to put hive-site.xml in the folder where you are running the sqoop command. Similiar approach should work from oozie as well.

Highlighted

Re: after sqoop hive-import, tables not listed

Guru

Sqoop will internally trigger Hive CLI command, so you will need to have proper hive-site.xml for Hive CLI to function. Can you please confirm that you can create tables when running Hive CLI by itself? (Not Beeline).

 

The Sqoop's --verbose output should give you some hint on whether table creation failed or succeeded, can you put the full output here for review?

 

Thanks

Re: after sqoop hive-import, tables not listed

Explorer

I have the same issue but I am running my sqoop command as part of an oozie workflow in HUE.  My job finishes successfully but the table does not get created under the database that I specify or under the default database.  I can see the files that get imported but they are under my users default directory.

 

Any suggestions ?

Re: after sqoop hive-import, tables not listed

Guru

What happens when you run exact same sqoop command manually from command line? Do you get the exact same result? What does the verbose output tell you?

Re: after sqoop hive-import, tables not listed

Explorer

Sadly I dont think the Oracle jdbc driver is in place for me to try that which is why I used HUE in the first place.  I'll see if someone can install it for me.

Re: after sqoop hive-import, tables not listed

Guru

Please also enabled verbose logging for sqoop action in Oozie, and get the full output of the sqoop command when running in Oozie, this can help us understand how the job was run inside Oozie anc compare it with when running by itself.