Hi @heta desai You could be using HDF (NiFi) as your primary ingestion tool and not have to worry about the other options necessarily. That said, sqoop is primarily used to move data from an existing RDBMS to Hadoop (or vice versa). Flume was the main tool previously to ingest flat files, csv, etc, but has fallen out of favour and is often being replaced with HDF/NiFi now. Kafka is a distributed messaging system which can be used as a pub/sub model for data ingest, including streaming. So all three are a bit different. The right tool for the job depends on your use case, but as I said, HDF/NiFi can pretty much cover the gambit, so if you are starting out now, you may want to look at that first. Here is another good write up on the same subject: https://community.hortonworks.com/questions/23337/best-tools-to-ingest-data-to-hadoop.html As always, if you find this post useful, please accept the answer.
... View more
