Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi InvokeHttp: How to work with a list of Remote URL's?

Nifi InvokeHttp: How to work with a list of Remote URL's?

New Contributor

Hi, we get every day a dynamic list if host with the InvokeHTTP processor from our source system.

On this base we have to request the source system again with InvokeHTTP host by host to get detailed data.

1. InvokeHTTP Request:

https://xyz.abc.com/prod/check_mk/view.py?view_name=hostgroup&hostgroup=vmware-host&limit=hard&outpu... Output : [{"host_state":"UP","host“:“xxx.yyy.com“,“host_icons":"menu","num_services_ok":"28","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“yyy.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“zzz.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017“}]

From this output we have to grep the host values (“xxx.yyy.com“, ....) and build an request for every host:

2. InvokeHTTP Requests

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"xxx.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"yyy.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"zzz.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}} ...

How can we challenge this? Thanks Timo

2 REPLIES 2

Re: Nifi InvokeHttp: How to work with a list of Remote URL's?

  • InvokeHttp to make your first request
  • SplitJson on $. to get a flow file for each host
  • EvaluateJsonPath with host = $.host to extract the host field from the JSON to a flow file attribute
  • UpdateAttribute with request = {"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"${host}"}],"data_range":{"time_range":[1491174000,1491260340]}}
  • InvokeHttp
Highlighted

Re: Nifi InvokeHttp: How to work with a list of Remote URL's?

New Contributor

Can this list of URLs be run concurrently by selecting Concurrent tasks to > 1? If yes, how should the response and downstream processors are handled (they need to be current as well)?

Don't have an account?
Coming from Hortonworks? Activate your account here