Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Real-Time Data Ingestion for mission critical applications;

avatar
Contributor

Our use case ;

In some text files coming data from some mission critical applications,these are not click stream data or something like that.

We have to catch every row without data losing.

At the beginning, daily approximately 15,000,000 rows expected.

30,000 rows/minute.

Somehow we have to use kafka to store data.

Some consumers take data from kafka topics and than write to hbase or phoenix.Here is clear for us.

The most important thing is all rows in these text files must be readed anyway.

Question 1.

Which solution is best practice for that ?

1. Flume & Kafka ?

2. Spark streaming & Kafka ?

3. Only Spark streaming ?

4. Storm & Kafka ?

5. Flume --> to hbase or phoenix ?

6. any other solutions ?

Question 2.

Can we use best practice solution with Nifi ?

Thanks in advance,

1 ACCEPTED SOLUTION

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
1 REPLY 1

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login