@David Sheard I am facing the same issue, please let me know in case you were able to resolve it. ... View more

Hi @Timothy Spann, Can you pls tell me why Storm was chosen as the first framework for HDF? Is it because it offers a real-time streaming environment, as opposed to batch streaming by Spark? Thanks. ... View more

Thanks for the reply @Jobin George, can you tell me how to send the results from Spark back to NiFi? There is a NiFiReceiver but no NiFiSender. Does the former have a method to send back the results? (I currently use the Java version) Thanks. ... View more

@mclark can you give us an approximate month when NiFi 1.0 will be available to the community? Thanks. ... View more

Hi, I have tried building a web service with NiFi and am able to get the incoming requests and pass it to Spark/Storm. Assuming that I compute the prediction inside Spark, I wish to know how to send back the score/result as response to NiFi. If that is currently not possible, what are the chances of creating a custom processor in R to predict the scores and pass it on as response? Thanks. ... View more

Thank you very much for the reply. If its the case of priority then I understand. 🙂 Sure, I will open the ML query as a separate one. ... View more

Hi @Benjamin Leonhardi, Thanks for replying. You are right, there is a connector to receive the data from NiFi to Spark. I had tried it by transferring the data to Spark through an output port and it works pretty well 🙂 My question is more on why Hortonworks decided not to include it. I am also curious on the Machine Learning integration. Please share if there are any good ways to accept requests from NiFi, make predictions using a trained classifier and finally give back the scores as response. Here again, NiFi can accept requests and give back a response, but I am not able to give it to a classifier and get the scores back. Thanks again. ... View more

Hi @Jobin George, Thanks for the tutorial! I am connecting to Spark from Nifi using the Output Port. I tried an example - sending a HTTP POST request using the HandleHttpRequest processor with random JSON message which is processed by Spark (code in Java) and is ultimately stored in a MongoDB collection. As a next step, I am going to send a real-world JSON request (let's say IoT data) and predict using a model (which I had already trained using MLLib). I want to predict the score and *return the result* back to Nifi's HandleHttpResponse processor as a response. I am not able to connect an output relationship from the Output Port to the Response processor. Is there another way to send the response (from Spark side) ? Thanks, Sai ... View more