Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

i am getting this error for sqoop to hbase incremental import and scheduling in oozie??please help me ??

Labels:
avatar
author-rank omkarmotoe
Rising Star

Created ‎06-01-2016 05:08 AM 

2016-06-01 04:46:03,257  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] Start action [0000003-160601041645443-oozie-root-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:46:03,263  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] [***0000003-160601041645443-oozie-root-W@:start:***]Action status=DONE
2016-06-01 04:46:03,264  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] [***0000003-160601041645443-oozie-root-W@:start:***]Action updated in DB!
2016-06-01 04:46:03,396  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@:start:
2016-06-01 04:46:03,402  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W
2016-06-01 04:46:03,433  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Start action [0000003-160601041645443-oozie-root-W@sqoop-import] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:46:10,745  INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Trying to get job [job_1464701959467_0011], attempt [1]
2016-06-01 04:46:10,806  INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] checking action, hadoop job ID [job_1464701959467_0011] status [RUNNING]
2016-06-01 04:46:10,809  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] [***0000003-160601041645443-oozie-root-W@sqoop-import***]Action status=RUNNING
2016-06-01 04:46:10,810  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] [***0000003-160601041645443-oozie-root-W@sqoop-import***]Action updated in DB!
2016-06-01 04:46:10,912  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@sqoop-import
2016-06-01 04:47:18,879  INFO CallbackServlet:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] callback for action [0000003-160601041645443-oozie-root-W@sqoop-import]
2016-06-01 04:47:19,397  INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Trying to get job [job_1464701959467_0011], attempt [1]
2016-06-01 04:47:22,078  INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] action completed, external ID [job_1464701959467_0011]
2016-06-01 04:47:22,209  WARN SqoopActionExecutor:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, com/yammer/metrics/core/MetricsRegistry
2016-06-01 04:47:22,209  WARN SqoopActionExecutor:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Launcher exception: com/yammer/metrics/core/MetricsRegistry
java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:732)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:777)
at org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:255)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:163)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.core.MetricsRegistry
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 31 more
2016-06-01 04:47:22,348  INFO ActionEndXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] ERROR is considered as FAILED for SLA
2016-06-01 04:47:22,460  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] Start action [0000003-160601041645443-oozie-root-W@kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:47:22,466  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] [***0000003-160601041645443-oozie-root-W@kill***]Action status=DONE
2016-06-01 04:47:22,466  INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] [***0000003-160601041645443-oozie-root-W@kill***]Action updated in DB!
2016-06-01 04:47:22,678  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@kill
2016-06-01 04:47:22,678  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W
2016-06-01 04:47:22,678  INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@sqoop-import

JOB.PROPERTIES

jobTracker=sandbox.hortonworks.com:8050                                                                                                              
mapreduce.job.user.name=root                                                                                                                         
nameNode=hdfs://sandbox.hortonworks.com:8020                                                                                                         
queueName=default                                                                                                                                    
oozie.use.system.libpath=true                                                                                                                        
oozie.libpath=${nameNode}/user/root/share/lib                                                                                                        
oozie.wf.application.path=hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqoop/                                                         
user.name=root

WORKFLOW.XML

<workflow-app name="sqoop-import-hbase" xmlns="uri:oozie:workflow:0.4">                                                                              

<start to="sqoop-import"/>                                                                                                                           
<action name="sqoop-import" cred="hbase">                                                                                                            
<sqoop xmlns="uri:oozie:sqoop-action:0.2">                                                                                                           
<job-tracker>${jobTracker}</job-tracker>                                                                                                             
<name-node>${nameNode}</name-node>                                                                                                                   
<job-xml>/user/root/hbase-site.xml</job-xml>                                                                                                         

<configuration>                                                                                                                                      
<property>                                                                                                                                           
<name>mapred.job.queue.name</name>                                                                                                                   
<value>${queueName}</value>                                                                                                                          
</property>                                                                                                                                          
</configuration>                                                                                                                                     

<command>import --connect "jdbc:sqlserver://ip;port;database=ESP_MFR" --table bundle_plan_package --username tariff --password tariffsql --in
cremental append --check-column packageid --hbase-table bundle_plan_package --column-family bundleid</command>                                       

<file>/user/root/hbase-site.xml#hbase-site.xml</file>                                                                                                
<file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file>                                                                                                    
<file>/user/root/share/lib/sqoop/sqljdbc_6.0#sqljdbc_6.0</file>                                                                                      
<file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file>                                                  
<file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file>                                                  
<file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file>                                              
<file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file>                                                      
<file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file>                                                  
<file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file>                                    
<file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file>                                                                      
<file>/user/root/hbase/netty-all-4.0.23.Final.jar#netty-all-4.0.23.Final.jar</file>                                                                  
<file>/user/root/hbase/htrace-hbase-4.1.0-incubating.jar#htrace-hbase-4.1.0-incubating.jar</file>                                                    
<file>/user/root/hbase/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar#hive-hbase-handler-1.2.1000.2.4.0.0-169.jar</file>                                
<file>/user/root/hbase/hive-hbase-handler.jar#hive-hbase-handler.jar</file>                                                                          
<file>/user/root/hbase/guava-12.0.1.jar#guava-12.0.1.jar</file>                                                                                      
<file>/user/root/hbase/zookeeper.jar#zookeeper.jar</file>                                                                                            
</sqoop>                                                                                                                                             
<ok to="end"/>                                                                                                                                       
<error to="kill"/>                                                                                                                                   
</action>                                                                                                                                            
<kill name="kill">                                                                                                                                   
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>                                                              
</kill>                                                                                                                                              
<end name="end"/>                                                                                                                                    
</workflow-app>
2,761 Views
1 ACCEPTED SOLUTION

avatar
author-rank omkarmotoe
Rising Star

Created ‎06-02-2016 10:17 AM

The error that i am getting is related to the metrics core jar file missing in the configuration?

so i have added the metrics-core-2.2.0.jar which is in hbase lib directory to <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> in workflow.xml

so this way i resolved the error

View solution in original post

2,110 Views
3 REPLIES 3

avatar
author-rank KuldeepK author-rank
Master Guru

Created ‎06-01-2016 05:46 AM

@omkar pathallapalli - You are getting class not found exception for com.yammer.metrics.core.MetricsRegistry

For this, you should include hadoop-common-<version>.jar file in your oozie-workflow.xml

You can also check if oozie sharelib has this class or not, if not then please add it in sharelib and update the sharelib using below command

oozie admin -oozie http://<oozie-server>:11000/oozie -sharelibupdate

Please run above command as oozie user on oozie host.

2,110 Views

avatar
author-rank omkarmotoe
Rising Star

Created ‎06-01-2016 06:45 AM

I have tried but i am getting same error @Kuldeep Kulkarni

2,110 Views
0 Kudos

avatar
author-rank omkarmotoe
Rising Star

Created ‎06-02-2016 10:17 AM

The error that i am getting is related to the metrics core jar file missing in the configuration?

so i have added the metrics-core-2.2.0.jar which is in hbase lib directory to <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> in workflow.xml

so this way i resolved the error

2,111 Views
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements