Support Questions
Find answers, ask questions, and share your expertise

Nifi pull CSV from Splunk then convert to an API?

Nifi pull CSV from Splunk then convert to an API?

New Contributor

I have a Splunk instance that I am feeding CSV's into (outside of Nifi). I would like Nifi to see when stuff is added into Splunk then initiate a pull of the new data then turn around and for an API command with the info from the CSV in it. I am working with the GetSplunk processor and I can confirm it is retrieving the data I need and I think I figured out how to have it drop duplicate data as when I view data providence I do see a receive followed by a drop. However this wont scale as it pulls the same info over and over and once there is a ton of info the pull will be huge. Any help would be great as this is my first experience with Nifi and while I did find some youtube videos I am still struggling.

Recap

How do I get new CSV data from splunk into Nifi

IE "RuleDeployer,permanent,http://www.badguy.com,true,drop"

Then put the values from that out of Nifi as a API command

IE curl -E /etc/secret/certs/pem/sensor.dec.pem --cacert /etc/secret/certs/pem/cacert.pem -v -H "Content-Type: application/json" -X POST -d " \"ruleSet\":\"FromCSV(RuleDeployer)\",\"ruleKind\":\"FromCSV(permanent)\",\"originId\":1,\"mitigationAction\":\"FromCSV(drop)\",\"capture\":\"FromCSV(True)\",\"parameters\":{\ "url\":\"FromCSV(http://www.badguy.com)\"}}" https://localhost:8443/api/httpmiti/create