Support Questions
Find answers, ask questions, and share your expertise

metastore issue running sqoop with Oozie

metastore issue running sqoop with Oozie

Explorer

Hello,

I am unable to run sqoop with Oozie. I am able to get a successful run using the shell, but keeps getting the same error using Oozie. I tried researching the error but nothing works so far. This is the HDP sandbox and I thought the hive metastore would be pre-configured with Derby ?? Any guidance will be very helpful. Thanks.

2018-10-10 02:10:30,917 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:30,965 [main] INFO  org.apache.hive.hcatalog.common.HiveClientCache  - Initializing cache: eviction-timeout=120 initial-capacity=50 maximum-capacity=50                                           
2018-10-10 02:10:31,349 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:31,350 [main] INFO  org.apache.hadoop.hive.metastore.HiveMetaStore  - 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore                                 
2018-10-10 02:10:31,396 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore  - ObjectStore, initialize called                                                                                                
2018-10-10 02:10:31,639 [main] INFO  DataNucleus.Persistence  - Property hive.metastore.integral.jdo.pushdown unknown - will be ignored                                                                            
2018-10-10 02:10:31,641 [main] INFO  DataNucleus.Persistence  - Property datanucleus.cache.level2 unknown - will be ignored                                                                                        
2018-10-10 02:10:33,961 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:33,963 [main] INFO  org.apache.hadoop.hive.metastore.ObjectStore  - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Databa
se,Type,FieldSchema,Order"                                                                                                                                                                                         
2018-10-10 02:10:34,138 [main] WARN  org.apache.hadoop.hive.conf.HiveConf  - HiveConf of name hive.metastore.local does not exist                                                                                  
2018-10-10 02:10:35,513 [main] WARN  DataNucleus.Query  - Query for candidates of org.apache.hadoop.hive.metastore.model.MDatabase and subclasses resulted in no possible candidates                               
Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCre
ateTables"                                                                                                                                                                                                         
org.datanucleus.store.rdbms.exceptions.MissingTableException: Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData i
s incorrect, or you need to enable "datanucleus.schema.autoCreateTables" 

Here is the workflow

   <action name="sqoop_import">
        <sqoop xmlns="uri:oozie:sqoop-action:0.4">
            <job-tracker>${resourceManager}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>oozie.action.sharelib.for.sqoop</name>
                    <value>sqoop,hive,hcatalog</value>
                </property>
                <property>
                    <name>hive.metastore.local</name>
                    <value>false</value>
                </property>
            </configuration>
            <arg>export</arg>
            <arg>--connect</arg>
            <arg>jdbc:sqlserver://dev01;databasename=Test;schema=dbo;</arg>
            <arg>--username</arg>
            <arg>***</arg>
            <arg>--password</arg>
            <arg>***</arg>
            <arg>--table</arg>
            <arg>test_summary</arg>
            <arg>--hcatalog-database</arg>
            <arg>Test</arg>
            <arg>--hcatalog-table</arg>
            <arg>test_summary</arg>
        </sqoop>
        <ok to="end"/>
        <error to="kill"/>
    </action>