Member since
04-20-2016
34
Posts
11
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 654 | 06-02-2016 11:50 AM | |
| 84 | 06-02-2016 10:17 AM |
07-21-2016
12:03 PM
hi @hoda moradi I had this same problem. This problem went aways when I added the mysql-connector-java.jar library into the lib directory located inside the oozie project root directory where the job.properties and workflow.xml files are located.
... View more
06-08-2016
01:47 PM
1)Bundle.xml <bundle-app name='bundle-app'
xmlns='uri:oozie:bundle:0.1'>
<controls>
<kick-off-time>${jobStart}</kick-off-time>
</controls>
<coordinator name='coord-1'>
<app-path>hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqooptest/coordinator.xml</app-path>
</coordinator>
<coordinator name='coord-2'>
<app-path>hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqoop/coordinator.xml</app-path>
</coordinator> </bundle-app> 2)Job.properties oozie.bundle.application.path=hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/bundle nameNode=hdfs://sandbox.hortonworks.com:8020 jobTracker=sandbox.hortonworks.com:8050 queueName=default start=2016-06-08T05:00Z end=2016-06-09T21:00Z user.name=root 3)coordinator.xml coord-1 <coordinator-app name="wf_scheduler" frequency="30" start="2016-06-08T05:00Z" end="2016-06-09T21:00Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.1"> <action> <workflow> <app-path>hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqooptest</app-path> </workflow> </action> </coordinator-app> 4)coordinator.xml coord-2 <coordinator-app name="wf_scheduler" frequency="60" start="2016-06-08T05:00Z" end="2016-06-09T21:00Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.1"> <action> <workflow> <app-path>hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqoop/</app-path> </workflow> </action> </coordinator-app> 5)Workflow.xml For coordinator.xml coord-1 <workflow-app name="sqoop-import-hbase" xmlns="uri:oozie:workflow:0.4"> <start to="sqoop-import"/> <action name="sqoop-import"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <job-xml>/user/root/hbase-site.xml</job-xml> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <command>import --connect "jdbc:sqlserver://ip:1433;database=ESP_MFR" --table bundle_plan --username tari --password tarif --incremental append --check-column bundleid --hbase-table bundle_plan --column-family bundleid</command> <file>/user/root/hbase/hadoop-common-2.7.1.2.4.0.0-169.jar#hadoop-common-2.7.1.2.4.0.0-169.jar</file> <file>/user/root/hbase-site.xml#hbase-site.xml</file> <file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file> <file>/user/root/share/lib/sqoop/sqljdbc_6.0#sqljdbc_6.0</file> <file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file> <file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file> <file>/user/root/hbase/netty-all-4.0.23.Final.jar#netty-all-4.0.23.Final.jar</file> <file>/user/root/hbase/htrace-hbase-4.1.0-incubating.jar#htrace-hbase-4.1.0-incubating.jar</file> <file>/user/root/hbase/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar#hive-hbase-handler-1.2.1000.2.4.0.0-169.jar</file> <file>/user/root/hbase/hive-hbase-handler.jar#hive-hbase-handler.jar</file> <file>/user/root/hbase/guava-12.0.1.jar#guava-12.0.1.jar</file> <file>/user/root/hbase/zookeeper.jar#zookeeper.jar</file> <file>/user/root/hbase/metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> </sqoop> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> 6)Workflow.xml For coordinator.xml coord-2 <workflow-app name="sqoop-import-hbase" xmlns="uri:oozie:workflow:0.4"> <start to="sqoop-import"/> <action name="sqoop-import"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <job-xml>/user/root/hbase-site.xml</job-xml> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <command>import --connect "jdbc:sqlserver://ip:1433;database=ESP_MFR" --table bundle_plan_package --username tari --password tarif --incremental append --check-column packageid --hbase-table bundle_plan_package --column-family bundleid</command> <file>/user/root/hbase/hadoop-common-2.7.1.2.4.0.0-169.jar#hadoop-common-2.7.1.2.4.0.0-169.jar</file> <file>/user/root/hbase-site.xml#hbase-site.xml</file> <file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file> <file>/user/root/share/lib/sqoop/sqljdbc_6.0#sqljdbc_6.0</file> <file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file> <file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file> <file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file> <file>/user/root/hbase/netty-all-4.0.23.Final.jar#netty-all-4.0.23.Final.jar</file> <file>/user/root/hbase/htrace-hbase-4.1.0-incubating.jar#htrace-hbase-4.1.0-incubating.jar</file> <file>/user/root/hbase/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar#hive-hbase-handler-1.2.1000.2.4.0.0-169.jar</file> <file>/user/root/hbase/hive-hbase-handler.jar#hive-hbase-handler.jar</file> <file>/user/root/hbase/guava-12.0.1.jar#guava-12.0.1.jar</file> <file>/user/root/hbase/zookeeper.jar#zookeeper.jar</file> <file>/user/root/hbase/metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> </sqoop> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>
7)<u>Error</u> Error: E0501 : E0501: Could not perform authorization operation, Call From sandbox.hortonworks.com/10.22.2.125 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: <a href="http://wiki.apache.org/hadoop/ConnectionRefused">http://wiki.apache.org/hadoop/ConnectionRefused</a>
Log:-
org.apache.oozie.service.AuthorizationException:
E0501: Could not perform authorization operation, Call From
sandbox.hortonworks.com/10.22.2.125 to l
ocalhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/Con
nectionRefused
at
org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:406)
at
org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:266)
at
org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:304)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Call
From sandbox.hortonworks.com/10.22.2.125 to localhost:8020 failed on connection
exception: java.net.Connec
tException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1431)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1315)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1311)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1311)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at
org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:378)
... 26 more
Caused by: java.net.ConnectException:
Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710)
at
org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
at org.apache.hadoop.ipc.Client.call(Client.java:1397)
... 43 more
2016-06-08 04:41:00,199 WARN V1JobsServlet:523 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://s
andbox.hortonworks.com:11000/oozie/v2/jobs]
error[E0501], E0501: Could not perform authorization operation, Call From
sandbox.hortonworks.com/10.22.2
.125 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/h
adoop/ConnectionRefused
org.apache.oozie.servlet.XServletException:
E0501: Could not perform authorization operation, Call From
sandbox.hortonworks.com/10.22.2.125 to localh
ost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/Connecti
onRefused
at
org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:271)
at
org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:99)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at
org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:304)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:615)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:574)
at
org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.oozie.service.AuthorizationException: E0501: Could not perform
authorization operation, Call From sandbox.hortonworks.com/10.22
.2.125 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org
/hadoop/ConnectionRefused
at
org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:406)
at
org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:266)
... 25 more
Caused by: java.net.ConnectException: Call
From sandbox.hortonworks.com/10.22.2.125 to localhost:8020 failed on connection
exception: java.net.Connec
tException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1431)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1315)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1311)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1311)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at
org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:378)
... 26 more
Caused by: java.net.ConnectException:
Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
at org.apache.hadoop.ipc.Client.call(Client.java:1397)
... 43 more
... View more
Labels:
06-03-2016
12:37 PM
2016-06-03 11:47:22,552
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@:start:]
Start action [0000001-160602113811107-oozie-root-W@:start:] with user-retry
state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-03 11:47:22,558
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@:start:]
[***0000001-160602113811107-oozie-root-W@:start:***]Action status=DONE 2016-06-03 11:47:22,559
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@:start:]
[***0000001-160602113811107-oozie-root-W@:start:***]Action updated in DB! 2016-06-03 11:47:22,929
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[] No Notification URL is defined. Therefore nothing to notify for job
0000001-160602113811107-oozie-root-W 2016-06-03 11:47:22,935
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@:start:] No Notification URL is
defined. Therefore nothing to notify for job
0000001-160602113811107-oozie-root-W@:start: 2016-06-03 11:47:23,031
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] Start action
[0000001-160602113811107-oozie-root-W@sqoop-import] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-03 11:47:34,390
INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] Trying to get job
[job_1464701959467_0046], attempt [1] 2016-06-03 11:47:34,572
INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@sqoop-import]
checking action, hadoop job ID [job_1464701959467_0046] status [RUNNING] 2016-06-03 11:47:34,575
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@sqoop-import]
[***0000001-160602113811107-oozie-root-W@sqoop-import***]Action status=RUNNING 2016-06-03 11:47:34,575
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import]
[***0000001-160602113811107-oozie-root-W@sqoop-import***]Action updated in DB! 2016-06-03 11:47:34,632
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] No Notification URL
is defined. Therefore nothing to notify for job 0000001-160602113811107-oozie-root-W@sqoop-import 2016-06-03 11:49:36,786
INFO CallbackServlet:520 - SERVER[sandbox.hortonworks.com] USER[-]
GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] callback for action
[0000001-160602113811107-oozie-root-W@sqoop-import] 2016-06-03 11:49:50,525
INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@sqoop-import]
Trying to get job [job_1464701959467_0046], attempt [1] 2016-06-03 11:49:52,879
INFO CallbackServlet:520 - SERVER[sandbox.hortonworks.com] USER[-]
GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] callback for action
[0000001-160602113811107-oozie-root-W@sqoop-import] 2016-06-03 11:49:54,562
INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] action completed,
external ID [job_1464701959467_0046] 2016-06-03 11:49:54,792
INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] action produced
output 2016-06-03 11:50:00,341
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@end] Start action
[0000001-160602113811107-oozie-root-W@end] with user-retry state :
userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2016-06-03 11:50:00,356
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@end]
[***0000001-160602113811107-oozie-root-W@end***]Action status=DONE 2016-06-03 11:50:00,356
INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@end]
[***0000001-160602113811107-oozie-root-W@end***]Action updated in DB! 2016-06-03 11:50:00,640
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@end] No Notification URL is
defined. Therefore nothing to notify for job
0000001-160602113811107-oozie-root-W@end 2016-06-03 11:50:00,641
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[] No Notification URL is defined. Therefore nothing to notify for job
0000001-160602113811107-oozie-root-W 2016-06-03 11:50:00,641
INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com]
USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-160602113811107-oozie-root-W]
ACTION[0000001-160602113811107-oozie-root-W@sqoop-import] No Notification URL
is defined. Therefore nothing to notify for job
0000001-160602113811107-oozie-root-W@sqoop-import 2016-06-03 11:50:08,018
WARN ActionCheckXCommand:523 - SERVER[sandbox.hortonworks.com]
USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase]
JOB[0000001-160602113811107-oozie-root-W] ACTION[0000001-160602113811107-oozie-root-W@sqoop-import]
E0815: Action pending=[false], status=[OK]. Skipping ActionCheck Execution,
Error Code: E0815 job.properties jobTracker=sandbox.hortonworks.com:8050
mapreduce.job.user.name=root
nameNode=hdfs://sandbox.hortonworks.com:8020
queueName=default
oozie.use.system.libpath=true
oozie.libpath=${nameNode}/user/${user.name}/share/lib
oozie.wf.application.path=${nameNode}/user/${user.name}/examples/apps/sqoop/
user.name=root
work flow.xml <workflow-app name="sqoop-import-hbase" xmlns="uri:oozie:workflow:0.4">
<start to="sqoop-import"/>
<action name="sqoop-import">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/root/hbase-site.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<command>import --connect "jdbc:sqlserver://<ip >:<port>;database=ESP_MFR" --table bundle_plan_package --username tariff --password tariffsql --in
cremental append --check-column packageid --hbase-table bundle_plan_package --column-family bundleid</command>
<file>/user/root/hbase/hadoop-common-2.7.1.2.4.0.0-169.jar#hadoop-common-2.7.1.2.4.0.0-169.jar</file>
<file>/user/root/hbase-site.xml#hbase-site.xml</file>
<file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file>
<file>/user/root/share/lib/sqoop/sqljdbc_6.0#sqljdbc_6.0</file>
<file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file>
<file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file>
<file>/user/root/hbase/netty-all-4.0.23.Final.jar#netty-all-4.0.23.Final.jar</file>
<file>/user/root/hbase/htrace-hbase-4.1.0-incubating.jar#htrace-hbase-4.1.0-incubating.jar</file>
<file>/user/root/hbase/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar#hive-hbase-handler-1.2.1000.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hive-hbase-handler.jar#hive-hbase-handler.jar</file>
<file>/user/root/hbase/guava-12.0.1.jar#guava-12.0.1.jar</file>
<file>/user/root/hbase/zookeeper.jar#zookeeper.jar</file>
<file>/user/root/hbase/metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
... View more
- Tags:
- Data Processing
- Oozie
Labels:
06-02-2016
11:58 AM
i got it finally that i dind not follow the correct syntax job-xml should be above the configuration <job-xml>/user/root/hbase-site.xml</job-xml> <configuration> <\configuration>
... View more
06-02-2016
11:50 AM
1 Kudo
i resolved this by adding hbase/lib metrics-core-2.2.0.jar to /user/root/hbase/ and definer in <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file>
... View more
06-02-2016
10:17 AM
3 Kudos
The error that i am getting is related to the metrics core jar file missing in the configuration? so i have added the metrics-core-2.2.0.jar which is in hbase lib directory to <file>metrics-core-2.2.0.jar#metrics-core-2.2.0.jar</file> in workflow.xml so this way i resolved the error
... View more
06-01-2016
06:45 AM
I have tried but i am getting same error @Kuldeep Kulkarni
... View more
06-01-2016
05:08 AM
2 Kudos
2016-06-01 04:46:03,257 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] Start action [0000003-160601041645443-oozie-root-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:46:03,263 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] [***0000003-160601041645443-oozie-root-W@:start:***]Action status=DONE
2016-06-01 04:46:03,264 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] [***0000003-160601041645443-oozie-root-W@:start:***]Action updated in DB!
2016-06-01 04:46:03,396 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@:start:
2016-06-01 04:46:03,402 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W
2016-06-01 04:46:03,433 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Start action [0000003-160601041645443-oozie-root-W@sqoop-import] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:46:10,745 INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Trying to get job [job_1464701959467_0011], attempt [1]
2016-06-01 04:46:10,806 INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] checking action, hadoop job ID [job_1464701959467_0011] status [RUNNING]
2016-06-01 04:46:10,809 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] [***0000003-160601041645443-oozie-root-W@sqoop-import***]Action status=RUNNING
2016-06-01 04:46:10,810 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] [***0000003-160601041645443-oozie-root-W@sqoop-import***]Action updated in DB!
2016-06-01 04:46:10,912 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@sqoop-import
2016-06-01 04:47:18,879 INFO CallbackServlet:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] callback for action [0000003-160601041645443-oozie-root-W@sqoop-import]
2016-06-01 04:47:19,397 INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Trying to get job [job_1464701959467_0011], attempt [1]
2016-06-01 04:47:22,078 INFO SqoopActionExecutor:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] action completed, external ID [job_1464701959467_0011]
2016-06-01 04:47:22,209 WARN SqoopActionExecutor:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], main() threw exception, com/yammer/metrics/core/MetricsRegistry
2016-06-01 04:47:22,209 WARN SqoopActionExecutor:523 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] Launcher exception: com/yammer/metrics/core/MetricsRegistry
java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:732)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:777)
at org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:255)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:163)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.core.MetricsRegistry
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 31 more
2016-06-01 04:47:22,348 INFO ActionEndXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] ERROR is considered as FAILED for SLA
2016-06-01 04:47:22,460 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] Start action [0000003-160601041645443-oozie-root-W@kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2016-06-01 04:47:22,466 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] [***0000003-160601041645443-oozie-root-W@kill***]Action status=DONE
2016-06-01 04:47:22,466 INFO ActionStartXCommand:520 - SERVER[sandbox.hortonworks.com] USER[root] GROUP[-] TOKEN[] APP[sqoop-import-hbase] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] [***0000003-160601041645443-oozie-root-W@kill***]Action updated in DB!
2016-06-01 04:47:22,678 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@kill] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@kill
2016-06-01 04:47:22,678 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W
2016-06-01 04:47:22,678 INFO WorkflowNotificationXCommand:520 - SERVER[sandbox.hortonworks.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-160601041645443-oozie-root-W] ACTION[0000003-160601041645443-oozie-root-W@sqoop-import] No Notification URL is defined. Therefore nothing to notify for job 0000003-160601041645443-oozie-root-W@sqoop-import JOB.PROPERTIES jobTracker=sandbox.hortonworks.com:8050
mapreduce.job.user.name=root
nameNode=hdfs://sandbox.hortonworks.com:8020
queueName=default
oozie.use.system.libpath=true
oozie.libpath=${nameNode}/user/root/share/lib
oozie.wf.application.path=hdfs://sandbox.hortonworks.com:8020/user/root/examples/apps/sqoop/
user.name=root
WORKFLOW.XML <workflow-app name="sqoop-import-hbase" xmlns="uri:oozie:workflow:0.4">
<start to="sqoop-import"/>
<action name="sqoop-import" cred="hbase">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/root/hbase-site.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<command>import --connect "jdbc:sqlserver://ip;port;database=ESP_MFR" --table bundle_plan_package --username tariff --password tariffsql --in
cremental append --check-column packageid --hbase-table bundle_plan_package --column-family bundleid</command>
<file>/user/root/hbase-site.xml#hbase-site.xml</file>
<file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file>
<file>/user/root/share/lib/sqoop/sqljdbc_6.0#sqljdbc_6.0</file>
<file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file>
<file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file>
<file>/user/root/hbase/netty-all-4.0.23.Final.jar#netty-all-4.0.23.Final.jar</file>
<file>/user/root/hbase/htrace-hbase-4.1.0-incubating.jar#htrace-hbase-4.1.0-incubating.jar</file>
<file>/user/root/hbase/hive-hbase-handler-1.2.1000.2.4.0.0-169.jar#hive-hbase-handler-1.2.1000.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hive-hbase-handler.jar#hive-hbase-handler.jar</file>
<file>/user/root/hbase/guava-12.0.1.jar#guava-12.0.1.jar</file>
<file>/user/root/hbase/zookeeper.jar#zookeeper.jar</file>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
... View more
- Tags:
- Hadoop Core
- Oozie
Labels:
05-31-2016
07:05 AM
Hi @Benjamin Leonhardi I am trying to import a table from SQL Server to hbase using sqoop and through incremental import in sqoop trying to update the hbase table using empid and schedule the sqoop job using oozie workflow in order to make the job runs on particular time basis. Eg) SQL TABLE create table employee(empid int primary key,empname varchar(35),designation varchar(30),salary int); insert into employee values(300,'Azhar','MD',50000);
insert into employee values(301,'vijay','GM',40000);
insert into employee values(302,'rahul','Asst GM',35000);
insert into employee values(303,'khanna','accountant',25000);
insert into employee values(304,'vikram','sales manager',20000); IMPORTING DATA INTO HBASE USING SQOOP sqoop import --connect "jdbc:sqlserver://localhost:1433;database=US_DB" --username sa--password 12345 --table employee --hbase-table hb_emp --column-family empid --hbase-create-table INCREMENTAL IMPORT IN SQOOP FOR SQL-HBASE TABLE
sqoop import --connect "jdbc:mysql://localhost;database=US_DB" --username root -P --table employee --hbase-table hb_emp --column-family cfemp --incremental append --check-column empid --last-value 304 SCHEDULING SQOOP INCREMENTAL JOB USING OOZIE FOR HBASE TABLE Here is my job.properties and workflow.xml configuration job.properties
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
nameNode=hdfs://sandbox.hortonworks.com:8020
jobTracker=sandbox.hortonworks.com:8050
queue.Name=default
examplesRoot=examples
oozie.use.system.libpath=true
oozie.libpath=${nameNode}/user/root/share/lib
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/sqoop/
workflow.xml
<workflow-app name="sqoop-hbase-wf" xmlns="uri:oozie:workflow:0.2">
<start to="sqoop-import"/>
<action name="sqoop-import">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/root/hbase-site.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<command>sqoop import --connect "jdbc:mysql://localhost;database=US_DB" --username root -P --table employee --hbase-table hb_emp --column-family cfemp
--incremental append --check-column empid --last-value 304</command>
<file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file>
<file>/user/root/hbase/hbase-client-1.1.2.2.4.0.0-169.jar#hbase-client-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-common-1.1.2.2.4.0.0-169.jar#hbase-common-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-protocol-1.1.2.2.4.0.0-169.jar#hbase-protocol-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file>
<file>/user/root/hbase/hbase-server-1.1.2.2.4.0.0-169.jar#hbase-server-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar#hbase-hadoop-compat-1.1.2.2.4.0.0-169.jar</file>
<file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file>
</sqoop>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app> ERROR Getting struck while running oozie with sqoop-main exception. Please help me to solve this issue. What are the compatible versions for this task to be completed usinf hdp 2.4. ENVIRONMENT
Hortonworks 2.4 hdp
Thanks
... View more
05-30-2016
12:23 PM
1 Kudo
He is my job. properties and workflow.xml job.properties
nameNode=hdfs://sandbox.hortonworks.com:8020
jobTracker=sandbox.hortonworks.com:8050
queueName=default
examplesRoot=examples
oozie.use.system.libpath=true
oozie.libpath=${nameNode}/user/${user.name}/share/lib
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/apps/sqoop/
here is my workflow.xml
<workflow-app name="sqoop-hbase-wf" xmlns="uri:oozie:workflow:0.4">
<start to="sqoop-import"/>
<action name="sqoop-import">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>/user/root/hbase-site.xml</job-xml>
<command>import --connect "jdbc:sqlserver://ip:port;database=ESP" --table bundle_plan_package --username omkar --password tarpass --incremental append --check-column packageid --hbase-table bundle_plan_package --column-family bundleid</command>
<file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file>
<file>/user/root/hbase/hbase-client.jar#hbase-client.jar</file>
<file>/user/root/hbase/hbase-common.jar#hbase-common.jar</file>
<file>/user/root/hbase/hbase-protocol.jar#hbase-protocol.jar</file>
<file>/user/root/hbase/htrace-core-3.1.0-incubating.jar#htrace-core-3.1.0-incubating.jar</file>
<file>/user/root/hbase/hbase-server.jar#hbase-server.jar</file>
<file>/user/root/hbase/hbase-hadoop-compat.jar#hbase-hadoop-compat.jar</file>
<file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file>
</sqoop>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
... View more
Labels:
05-30-2016
06:04 AM
Thanks @Predrag Minovic
... View more
05-27-2016
03:19 AM
Hi @Kuldeep Kulkarni here is my work flow <workflow-app name="sqoop-wf" xmlns="uri:oozie:workflow:0.2"> <start to="sqoop-import"/> <action name="sqoop-import"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> <delete path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data/sqoop"/> <mkdir path="${nameNode}/user/${wf:user()}/${examplesRoot}/output-data"/> </prepare> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <job-xml>/user/root/hbase-site.xml</job-xml> <command>import --connect "jdbc:sqlserver://<ip:port>;database=ESP_MFR" --table bundle --username uname --password password --incremental append --check-column packageid --hbase-table bundle --column-family bundleid</command> <file>/user/root/sqljdbc4.jar#sqljdbc4.jar</file> <file>/user/root/hbase/hbase-client.jar#hbase-client.jar</file> <file>/user/root/hbase/hbase-common.jar#hbase-common.jar</file> <file>/user/root/hbase/hbase-protocol.jar#hbase/hbase-protocol.jar</file> <file>/user/root/hbase/htrace-core3.1.0-incubating.jar#htrace-core3.1.0-incubating.jar</file> <file>/user/root/hbase/hbase-server.jar#hbase-server.jar</file> <file>/user/root/hbase/hbase-hadoop-compat.jar#hbase-hadoop-compat.jar</file> <file>/user/root/hbase/high-scale-lib-1.1.1.jar#high-scale-lib-1.1.1.jar</file> </sqoop> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>
... View more
05-26-2016
01:27 PM
In apache oozie book i found this and please help me how to XML version 0.4 my XML version is 0.2 Common
Errors and Debugging In
this section, we cover some common errors and give various tips and solutions. We
hope this section will save you some time by being a quick reference guide for some
of the common mistakes we have seen repeated on Oozie. A
lot of the Oozie-level errors that the server throws are usually E0710
or E0701. You will see these errors in the Oozie UI and these errors
usually mean it’s not a Hadoop issue, but something in the Oozie system. Workflow
XML schema errors: Always
be aware of the XML schema version and features.
When you see errors like the one shown here, check the schema: Error:
E0701 : E0701: XML schema error, cvc-complex-type.2.4.a: Invalid
content was found starting with element 'global'. One of '{"uri:oozie:workflow:0.3":credentials,
"uri:oozie:workflow:0.3":start}' is
expected. Issues
with the <global> section: The
example error previously shown relates to support for
the <global> section. It’s only supported in workflow XML
version 0.4 or higher,
as shown here: <workflow-app
xmlns="uri:oozie:workflow:0.4" name="my-test-wf"> Also, with some of
the extension actions like hive or shell, the <global> section might not work.
Remember that the action definitions have their own schema version as well and confirm
that you are using supported features both at the workflow level and the action level. Schema
version errors with action types: The action schema
versions are different and often a lower
number than the workflow schema version. Sometimes, users cut and paste the same
version from the workflow header and that may not be the right version number. If
you see the following error with a Hive action for instance, you should check the
version number (it is probably too high): Error: E0701 : E0701:
XML schema error, cvc-complex-type.2.4.c: The matching wildcard
is strict, but no declaration can be found for element
... View more
Labels:
05-24-2016
06:32 AM
Thank you Very Much @Josh Elser
... View more
05-24-2016
04:33 AM
capture.png1.pngcapture2.png
... View more
Labels:
04-29-2016
07:36 AM
i have given like this sqoop import jdbe:sqlserver;//xxx.xxx.xx.xx:1433/sample --table TEST--username uname --password pwd -m 1
... View more
04-28-2016
04:58 AM
@ sandeep please paste full log
... View more
