Here is how I would do it but I am missing any requirement, please feel free to add more details (without revealing any secret sauce of your logic 😉 - Assumptions first - Job 1 - Executes Step A and Step B at 00:01 AM every morning (two step job) Job 2 - Executes Step B every hour between 01:01 - 23:01 through out the day. (single step job) Note: The timings can obviously be adjusted but assumption here is that the time of execution of two step is fixed and is mutually exclusive with the other 23 executions of the single step job. These two steps could be any action supported by Oozie like Hive, Pig, Email, SSH etc. So the workflow definitions will have duplicate Step B action in both jobs. Coordinator Definitions - The exact time of execution and frequency can be controlled by specifying the values of validity and frequency. For Job1, Validity = 00:00 hours of the day when you want the job to start executing. Frequency = ${coord:days(int n)} See section 4.4.1. The coord:days(int n) and coord:endOfDays(int n) EL functionsat - http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html For Job 2, Validity = 01:00 hours of the same day as Job 1 Frequency = frequency="* 1-23 * * *" Note: that instead of using fixed frequency we are using cron type syntax, which is super cool See section 4.4.3. Cron syntax in coordinator frequency at - http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html Hope this helps. ... View more

Hi Ravi, As you know that property dfs.datanode.handler.count defines the number of server threads for the datanode, this property is at the datanode level. In other words, this property value is driven more by the I/O requests to the datanode rather than the size of the cluster. So, hypothetically speaking, if you have a cluster (large or small) being used for online archiving use case such that the data is not read very often, you do not need a large number of parallel threads. As the traffic / I/O goes up, there may be benefit in increasing the number of parallel threads in datanode. Here is the code that uses this property. If there is a way to isolate the heavy workers from light workers then you can create Ambari configuration groups to have different values for these properties. ... View more

This post by Lester Martin sums up really well - https://martin.atlassian.net/wiki/pages/viewpage.action?pageId=36044812 Here is a summary - Since most OS patching / upgrade requires reboot, it is best to schedule such an activity around a scheduled outage. It is also recommended to go through the exercise in a lower level environment prior to applying changes in a PROD environment. In order to be apply the changes while the cluster is up, the patch/upgrade will have to be applied in rolling manner by first stopping the components in the host from Ambari, then applying changes, rebooting host and then starting the Hadoop services from Ambari. Repeat for each host. This process will have to be scripted for a large size cluster. The steps of stopping and starting the cluster can be performed by uzing Ambari APIs. ... View more