Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

metastore issue running sqoop with Oozie

Highlighted

metastore issue running sqoop with Oozie

New Contributor

Hello,

I am unable to run sqoop with Oozie. I am able to get a successful run using the shell, but keeps getting the same error using Oozie. I tried researching the error but nothing works so far. This is the HDP sandbox and I thought the hive metastore would be pre-configured with Derby ?? Any guidance will be very helpful. Thanks.

2018-10-10 02:10:30,917 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:30,965 [main] INFO  org.apache.hive.hcatalog.common.HiveClientCache  - Initializing cache: eviction-timeout=120 initial-capacity=50 maximum-capacity=50                                           
2018-10-10 02:10:31,349 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:31,350 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore  - 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore                                 
2018-10-10 02:10:31,396 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore  - ObjectStore, initialize called                                                                                                
2018-10-10 02:10:31,639 [main] INFO  DataNucleus.Persistence  - Property hive.metastore.integral.jdo.pushdown unknown - will be ignored                                                                            
2018-10-10 02:10:31,641 [main] INFO  DataNucleus.Persistence  - Property datanucleus.cache.level2 unknown - will be ignored                                                                                        
2018-10-10 02:10:33,961 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:33,963 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore  - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Databa
se,Type,FieldSchema,Order"                                                                                                                                                                                         
2018-10-10 02:10:34,138 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:35,513 [main] WARN  DataNucleus.Query  - Query for candidates of org.apache.hadoop.hive.metastore.model.MDatabase and subclasses resulted in no possible candidates                               
Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCre
ateTables"                                                                                                                                                                                                         
org.datanucleus.store.rdbms.exceptions.MissingTableException: Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData i
s incorrect, or you need to enable "datanucleus.schema.autoCreateTables" 

Here is the workflow

   <action name="sqoop_import">
        <sqoop xmlns="uri:oozie:sqoop-action:0.4">
            <job-tracker>${resourceManager}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>oozie.action.sharelib.for.sqoop</name>
                    <value>sqoop,hive,hcatalog</value>
                </property>
                <property>
                    <name>hive.metastore.local</name>
                    <value>false</value>
                </property>
            </configuration>
            <arg>export</arg>
            <arg>--connect</arg>
            <arg>jdbc:sqlserver://dev01;databasename=Test;schema=dbo;</arg>
            <arg>--username</arg>
            <arg>***</arg>
            <arg>--password</arg>
            <arg>***</arg>
            <arg>--table</arg>
            <arg>test_summary</arg>
            <arg>--hcatalog-database</arg>
            <arg>Test</arg>
            <arg>--hcatalog-table</arg>
            <arg>test_summary</arg>
        </sqoop>
        <ok to="end"/>
        <error to="kill"/>
    </action>