Nifi InvokeHttp: How to work with a list of Remote URL's?

[#1]

Hi, we get every day a dynamic list if host with the InvokeHTTP processor from our source system.

On this base we have to request the source system again with InvokeHTTP host by host to get detailed data.

1. InvokeHTTP Request:

https://xyz.abc.com/prod/check_mk/view.py?view_name=hostgroup&hostgroup=vmware-host&limit=hard&outpu... Output : [{"host_state":"UP","host“:“xxx.yyy.com“,“host_icons":"menu","num_services_ok":"28","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“yyy.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“zzz.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017“}]

From this output we have to grep the host values (“xxx.yyy.com“, ....) and build an request for every host:

2. InvokeHTTP Requests

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"xxx.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"yyy.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"zzz.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}} ...

How can we challenge this? Thanks Timo

[#2]

• InvokeHttp to make your first request
• SplitJson on $. to get a flow file for each host
• EvaluateJsonPath with host = $.host to extract the host field from the JSON to a flow file attribute
• UpdateAttribute with request = {"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"${host}"}],"data_range":{"time_range":[1491174000,1491260340]}}
• InvokeHttp

[#3]

Can this list of URLs be run concurrently by selecting Concurrent tasks to > 1? If yes, how should the response and downstream processors are handled (they need to be current as well)?