how can i use listentcp processor in nifi and pass the stream to publish kafka processor

[#1]

@Mohamed Hossam

Are you looking for an example of how to use the ListenTCP processor?

[#2]

@Mohamed Hossam

So, after you receive the data via ListenTCP, then you want to publish to Kafka and then to Storm? Or do you want to read data from Kafka and then send to Storm?

[#3]

@Wynner

sorry of not replaying for all this time ,,, i want to have a live streaming of network traffic then publish it to kafka then to storm.

all i want to process the data of network traffic if there's another tool can replace the kafka and storm no problem , i read that i can make this job also using spark stream , which better storm or spark stream.

[#4]

yes sure it will help very much

but the main target for me to pass message from kafka to storm using nifi

[#5]

@Mohamed Hossam

Can you provide more details about your flow?

[#6]

@Wynner i need to capture my network traffic ,, this is my flow .

[#7]

@Mohammed

If your application is tollerable for near realtime processing (ie between 500ms to 2 sec ) latency then you can use spark streaming.

If you application need complex processing (like join few extracted value with another stream of data and conclude some result) and also need to be real time processing then you need to go with Storm.

If it you want to ingest the data and do some simple transformation then you may go with Nifi.

Irrespective of above if you want to handle the reliability, congestion control , back pressure then it is good to use kafka , because the stream data will be first put in kafka and then nifi/spark/storm can pull from kafka for processing.

I hope i could explain each components usage 🙂

[#8]

@tthomas , thank you for this great help . and i think now i can determine which tool i"ll use for my application