Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to add a apache nifi flowfile from kafka into a httpresponse

Highlighted

How to add a apache nifi flowfile from kafka into a httpresponse

Explorer

I have a working webservice with nifi where I get a httprequest and can respone to it. For test I use ReplaceText to add a specifi content to the response. This works, but my goal is to add a flowfile from Kafka (or the value of it) as the content of the httpresponse.

The usecase is, that I can send information as a response of an httprequest (REST Webservice).

I was searching and testing for a solution, but nothing works. Is it possbile with a standard processor of Apache NiFi (Hortonworks HDF)?

best regards jan

6 REPLIES 6
Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

Jan, are you trying to send binary content response?

Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

Explorer

I edit my post. My problem is, that GetKafka doesn't accepts incoming flow files. I tried MergeContent and ReplaceText but it doesn't work.

Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

I think there's a confusion around the architecture. What would you expect to send to GetKafka? It's purpose is to listen on a topic and consume incoming messages. It's not a request-response processor. Maybe share more details around your pipeline?

Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

Explorer

Actually it is just a test example. Actually I have to webservices. Input and Output. I receive data from my input webservice and stores them into Kafka. To test the output webservice, I want to send the data which is stored in Kafka, but this is not possible. I should put them into hbase and them through hive or directly from hbase. Right?

Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

One can't 'query' Kafka, it's not designed for that. If you want to serve data identified by some incoming ID, it's necessary to save things in random-access storage.

Highlighted

Re: How to add a apache nifi flowfile from kafka into a httpresponse

I think the request is for GetKafka to support being triggered by an incoming FlowFile, right now it does not support that and it consumers from the topic each time the processor runs, which is based on the run schedule of the processor. In theory it could be possible to allow an incoming FlowFile to trigger it so it would only consume when triggered by an external request.

Don't have an account?
Coming from Hortonworks? Activate your account here