schedule the processor to run only every 10sec so that you don’t query too
As the Rest call outputs the application details in Json format, lets use a
SplitJson processor to separate
Provide “JsonPath Expression” value as “$.apps.app” in the configuration.
GetHTTP to SplitJson for success relation and auto
4. Lets add
processor to extract required fields and add them to flow-file
attribute: Configure it as below:
to EvaluateJsonPath for success
6. Create and start two controller services:
DistributedMapCacheClientService, DistributedMapCacheServer so that we
keep track of all the applications and don’t sent out duplicate alerts for same
7. Add a
processor to update the cache with latest apps that fails/killed. Configure
it as below adding Distributed cache service.
Lets auto terminate Failure relationship and connect success relationship to
PutEmail processorwhich will sent out email for any new failed/killed application.
Make sure you have formatted the email body and subject to have all information
about the failed job:
Auto terminate success and failure relationship for
PutEmail processor. Once you start the Flow, you will get alerts
for each Killed/Failed Yarn application. My Alert would look like below:
Note: Now you can configure your GetHTTP Processor to query YARN to find long running applications