Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi InvokeHttp: How to work with a list of Remote URL's?

avatar

Hi, we get every day a dynamic list if host with the InvokeHTTP processor from our source system.

On this base we have to request the source system again with InvokeHTTP host by host to get detailed data.

1. InvokeHTTP Request:

https://xyz.abc.com/prod/check_mk/view.py?view_name=hostgroup&hostgroup=vmware-host&limit=hard&outpu... Output : [{"host_state":"UP","host“:“xxx.yyy.com“,“host_icons":"menu","num_services_ok":"28","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“yyy.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017"},{"host_state":"UP","host“:“zzz.yyy.com“,“host_icons":"menu","num_services_ok":"34","num_services_warn":"0","num_services_unknown":"0","num_services_crit":"0","num_services_pending":"0","ctime":"Wed Jun 21 06:00:02 CEST 2017“}]

From this output we have to grep the host values (“xxx.yyy.com“, ....) and build an request for every host:

2. InvokeHTTP Requests

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"xxx.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"yyy.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}}

https://xyz.abc.com/prod/check_mk/webapi.py?action=get_graph&_username=ansible&_secret=XXXXXXXXX&request={"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"zzz.yyy.com"}],"data_range":{"time_range":[1491174000,1491260340]}} ...

How can we challenge this? Thanks Timo

2 REPLIES 2

avatar
Master Guru
  • InvokeHttp to make your first request
  • SplitJson on $. to get a flow file for each host
  • EvaluateJsonPath with host = $.host to extract the host field from the JSON to a flow file attribute
  • UpdateAttribute with request = {"specification":["template",{"service_description":"Filesystem /","site":"prod","graph_index":0,"host_name":"${host}"}],"data_range":{"time_range":[1491174000,1491260340]}}
  • InvokeHttp

avatar
Explorer

Can this list of URLs be run concurrently by selecting Concurrent tasks to > 1? If yes, how should the response and downstream processors are handled (they need to be current as well)?