Support Questions
Find answers, ask questions, and share your expertise

Nifi invokehttp processor to make web API calls to kerberized Solr

Solved Go to solution

Nifi invokehttp processor to make web API calls to kerberized Solr

Explorer

I'm trying to create a standard ingest flow for nifi that will process java application logs in json format and index them to Solr. I want this flow to create the necessary Solr collections if they don't already exist. I can do this with a simple API call to the /admin/collection endpoint; however, I don't know how kerberos authentication will work from the invokehttp processor. Does anyone have any knowledge on the subject to help get me going in the right direction on this?

Another option would be to invoke the "solrctl" command on the OS; I'm running CDH 5.11. The main issue there is that I don't know of a way to execute a script or command while still using the nifi expression language to pass parameters from flow file attributes.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nifi invokehttp processor to make web API calls to kerberized Solr

Ok, how about ExecuteStreamCommand which accepts incoming flow files?

View solution in original post

3 REPLIES 3

Re: Nifi invokehttp processor to make web API calls to kerberized Solr

As far as I know, none of the HTTP processors in NiFi support Kerberos authentication, so I don't think you'll be able to do the first idea. For the second idea, you should be able to use the ExecuteProcess processor, the command arguments property supports expression language.

Re: Nifi invokehttp processor to make web API calls to kerberized Solr

Explorer

Thank you for your suggestion. However, this will not work because ExecuteProcessor does not accept an incoming relationship. I have a preceding processor that has an initialize relationship as an output. Upon initialization of my dataflow a flowfile is is sent to this relationship and contains the necessary information to create the Solr Collection.

The more I look at this issue the more appropriate a CreateSolrCollection processor sounds. For now I'm going to continue with my scripted processor to run the solrctl command. This just requires the nifi user to have authority to invoke the necessary api calls. Unless there's something I'm missing, I think that's the best solution for my question.

Re: Nifi invokehttp processor to make web API calls to kerberized Solr

Ok, how about ExecuteStreamCommand which accepts incoming flow files?

View solution in original post