About amit_kannaujiya

amit_kannaujiya · ‎02-08-2017

Hi, Which package ? you all ready excluding it from the storm core .

amit_kannaujiya · ‎02-07-2017

Hi @trupti, There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help . Thank You

amit_kannaujiya · ‎02-03-2017

Hi, I have to schedule my hadoop jobs with oozie bundle . 1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2). I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1. How can i configure this from coordinator or workflow ? my bundle : <?xml version="1.0" encoding="UTF-8"?> <bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2"> <controls> <kick-off-time>${kickOffTime}</kick-off-time> </controls> <coordinator name="ods-ds-cms-coordinator" > <app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path> </coordinator> </bundle-app> coordinator : <coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}" frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4"> <action> <workflow> <app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path> <configuration> <property> <name>nameNode</name> <value>${nameNode}</value> </property> <property> <name>jobTracker</name> <value>${jobTracker}</value> </property> <property> <name>exampleDir</name> <value>${nameNode}/custom/oozie</value> </property> </configuration> </workflow> </action> </coordinator-app> workflow 1: <?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow"> <global> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queue}</value> </property> </configuration> </global> <start to="cms-checker"/> <action name="cms-checker"> <java> <main-class>com.insense.helper.CMSPullChecker</main-class> <arg>${cmsChecker}</arg> <arg>${cmsType}</arg> <capture-output/> </java> <ok to="trigger_next_job"/> <error to="kill"/> </action>  <action name="trigger_next_job"> <sub-workflow> <app-path>${exampleDir}/ods-next-task.workflow</app-path> <propagate-configuration/> </sub-workflow> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Something went wrong in ods-workflow</message> </kill> <end name="end"/> </workflow-app> workflow 2: <?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow"> <global> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queue}</value> </property> </configuration> </global> <start to="campaign-customer-eligibility-mr"/> <action name="campaign-customer-eligibility-mr"> <java> <main-class>com.insense.MyMapReduceJobDriver</main-class> <capture-output/> </java> <ok to="done"/> <error to="kill"/> </action> <kill name="kill"> <message>Something went wrong in ods-workflow</message> </kill> <end name="done"/> </workflow-app> How can I do this from oozie coordinator or workflow . I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this . Thanks

amit_kannaujiya · ‎01-28-2017

Initially Hbase services are working fine on all the client nodes , but this connection problem started when i do restart of hbase service after changes via ambari server . Same problem is not there in sandbox (you can start / stop any no of time) . Is there any special instruction need to follow before doing start/ stop hbase (or any other service) on multi node aws cluster on HDP 2.4.0 ?

amit_kannaujiya · ‎01-28-2017

no, there are not different aws instances .

amit_kannaujiya · ‎01-27-2017

Checking logs of Hbase master node : vim /var/log/hbase/hbase-hbase-master-ods-node2.log 2017-01-27 10:48:15,699 INFO [localhost:16000.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 472400 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms. 2017-01-27 10:48:17,203 INFO [localhost:16000.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 473904 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms. 2017-01-27 10:48:18,707 INFO [localhost:16000.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 475408 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms. 2017-01-27 10:48:20,212 INFO [localhost:16000.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 476913 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms. 2017-01-27 10:48:21,716 INFO [localhost:16000.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 478417 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms. Checking logs of region server node : vim /var/log/hbase/hbase-hbase-master-ods-node3.log 2017-01-27 10:41:22,369 WARN [regionserver/localhost/127.0.0.1:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying. 2017-01-27 10:41:25,369 INFO [regionserver/localhost/127.0.0.1:16020] regionserver.HRegionServer: reportForDuty to master=localhost,16000,1485513620999 with port=16020, startcode=1485513635588 2017-01-27 10:41:25,370 WARN [regionserver/localhost/127.0.0.1:16020] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2270) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:894) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:410) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:716) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:887) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1200) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)

amit_kannaujiya · ‎01-27-2017

Hi, I had updated the log4j properties of Hbase from ambari server in hdp 2.4 cluster . Then i do restart affected services, then hbase master nodes connection refused . please suggest how to solve this .

Online	Offline
Last Visited	‎04-26-2018 10:44 AM

Date Registered	‎01-27-2017 09:41 AM
Last Visited	‎04-26-2018 10:44 AM
Total Messages Posted	10
Total Kudos Received	1

Re: Storm 1.0 on HDP 2.5: java.lang.NoClassDefFoun...

Re: delay specific workflow action of oozie coordi...

delay specific workflow action of oozie coordinato...

Re: Connection failed: [Errno 111] Connection refu...

Re: Connection failed: [Errno 111] Connection refu...

Re: Connection failed: [Errno 111] Connection refu...

Connection failed: [Errno 111] Connection refused ...