Hello there, I am using NiFi 1.8.0 I wanna transfer data (about 50,000,000 millions records or upper than it , 6G) from my local to HDFS. I am using tailFile, mergContent and HDFS processors. With the default config of NiFi and processors (thread = 4) I transfer data about 2:20 ( 390 event per seconds) but data generator 's time is 900,000. How can I increase performance for this situation? ... View more

I need to run python code on zeppelin with spark engine and adding python interpreter on zeppelin then run python code but, I dont know, is python code run by spark engine? or not? if not, how can run python code by spark engine on zeppelin? ... View more

We are using Snort to send network packets to Apache Metron, and we want to calculate the average delay between packets (do a statistical analysis). Our solution is to write a java program by changing existing Snort topology inside Apache Storm (we want to make the least changes inside prepared java codes by Metron's team). First of all, is it a right solution?! Currently, we are able to send packets via Nifi Site-to-Site from a remote machine to the Metron server and see the results in Elasticsearch. To do the job, which file(s) in Metron should be changed, and where should I save the output(s), HDFS for instance? In the second step, we need to consider windowing to calculate the average value in specific periods. Any advice would be appreciated. ... View more