Support Questions

Find answers, ask questions, and share your expertise

Is it possible to ingest data into kafka from REST API calls?

HDP 2.3.2

We have data sitting on prem and we are planning to ingest data into Kafka using REST calls? Kafka will be running in the cloud.

1 ACCEPTED SOLUTION

@Artem Ervits

This is the answer that I got from the client https://github.com/confluentinc/kafka-rest

That's the answer without buying HDF.

View solution in original post

10 REPLIES 10

Nifi is not the option yet.

Mentor

Mentor

@Neeraj Sabharwal OK I see your nifi comment

@Guilherme Braccialli

Guil shared some ideas on using Flume.

Mentor

please close @Neeraj Sabharwal

@Artem Ervits updated...feel free to accept my answer

@Artem Ervits

This is the answer that I got from the client https://github.com/confluentinc/kafka-rest

That's the answer without buying HDF.

Expert Contributor

Does Confluent Inc Support cost extra if someone wants to install this in Production? Is there a hortonworks out of the box solution to get RESTful APIs data to be ingested into HDFS.

Nifi and Kafka example for pulling REST API end point data -

https://dzone.com/articles/real-time-stock-processing-with-apache-nifi-and-ap

Super Guru

Okay so I wrote an example nifi process to do it

 

https://www.datainmotion.dev/2020/01/flank-stack-nifi-processor-for-kafka.html

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.