I am very new to Nifi and HDF and hence finding it tough to understand the USP of Nifi with respect to other data transport mechanisms. So any help would be grateful.
- Is NiFi’s primary interaction only through UI?
- How different is Nifi from Kafka or any enterprise ESB apart from the visual data flow aspect?
- Especially when comparing with Kafka, what is common between them and where does they differ?
My understanding of the Nifi features with respect to Kafka
- Visual Command and Control - Not available in Kafka
- Data lineage - Something that can be done with Apache Atlas for Kafka?
- Data prioritisation - I presume this can be controlled with a combination of topics and consumers and consumer groups in Kafka
- Back pressure - As Kafka can retain data, consumers can always replay the data and catch-up
- Control Latency vs Throughput - Similar to back pressure and prioritisation, this can be controlled with consumers and topics with data retention
- Security - Kafka also has got security implementation
- Scaling - Build a Kafka cluster