Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is it possible to ingest data into kafka from REST API calls?

SOLVED Go to solution
Highlighted

Is it possible to ingest data into kafka from REST API calls?

HDP 2.3.2

We have data sitting on prem and we are planning to ingest data into Kafka using REST calls? Kafka will be running in the cloud.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is it possible to ingest data into kafka from REST API calls?

@Artem Ervits

This is the answer that I got from the client https://github.com/confluentinc/kafka-rest

That's the answer without buying HDF.

9 REPLIES 9

Re: Is it possible to ingest data into kafka from REST API calls?

Nifi is not the option yet.

Re: Is it possible to ingest data into kafka from REST API calls?

Mentor

Re: Is it possible to ingest data into kafka from REST API calls?

Mentor

@Neeraj Sabharwal OK I see your nifi comment

Re: Is it possible to ingest data into kafka from REST API calls?

@Guilherme Braccialli

Guil shared some ideas on using Flume.

Re: Is it possible to ingest data into kafka from REST API calls?

Mentor

please close @Neeraj Sabharwal

Re: Is it possible to ingest data into kafka from REST API calls?

@Artem Ervits updated...feel free to accept my answer

Re: Is it possible to ingest data into kafka from REST API calls?

@Artem Ervits

This is the answer that I got from the client https://github.com/confluentinc/kafka-rest

That's the answer without buying HDF.

Re: Is it possible to ingest data into kafka from REST API calls?

Expert Contributor

Does Confluent Inc Support cost extra if someone wants to install this in Production? Is there a hortonworks out of the box solution to get RESTful APIs data to be ingested into HDFS.

Re: Is it possible to ingest data into kafka from REST API calls?

New Contributor

Nifi and Kafka example for pulling REST API end point data -

https://dzone.com/articles/real-time-stock-processing-with-apache-nifi-and-ap