question Re: Machine Learning in Apache Nifi in Support Questions

question Re: Machine Learning in Apache Nifi in Support Questions https://community.cloudera.com/t5/Support-Questions/Machine-Learning-in-Apache-Nifi/m-p/124309#M87053 Hi <A rel="user" href="https://community.cloudera.com/users/7309/rendi7936.html" nodeid="7309" target="_blank">@Rendiyono Wahyu Saputro</A>,What are you trying to build is what we call <A href="http://fr.hortonworks.com/solutions/connected-data-platforms/" rel="nofollow noopener noreferrer" target="_blank">Connected Data Platform</A> at Hortonworks. You need to understand that you have two types of workloads/requirements and you need to use HDF and HDP jointly.<OL><LI>ML model construction: the first step towards you goal is to build your machine learning model. This require processing lot of historical data (data at rest) to detect some pattern related to what you are trying to predict. This phase is called "training phase".The best tool do this is HDP and more specifically Spark.</LI><LI>Applying the ML model: once step1 completed, you will have a model that you can apply to new data to predict something. In my understanding you want to apply this at real time data coming from twitter (data at motion). To get the data in real time and transform to what the ML model needs, you can use NiFi. Next, NiFi send the data to Storm or Spark Streaming that applies the model and get the prediction.</LI></OL>So you will have to use HDP to construct the model, HDF to get and transform the data, and finally a combination of HDF/HDP to apply the model and make the prediction. <img src="https://community.cloudera.com/t5/image/serverpage/image-id/22724i43985E3A52F4381D/image-size/medium?v=v2&px=400" role="button" title="3874-screen-shot-2016-04-30-at-45216-pm.png" alt="3874-screen-shot-2016-04-30-at-45216-pm.png" />To build a web service with NiFi you need to use several processors: one to listen to incoming requests, one or several processors to implement your logic (transformation, extraction, etc), one to publish the result. You can check this page that contains several data flow examples. The "<A href="https://cwiki.apache.org/confluence/download/attachments/57904847/Hello_NiFi_Web_Service.xml?version=1&modificationDate=1449369797000&api=v2" rel="nofollow noopener noreferrer" target="_blank">Hello_NiFi_Web_Service.xml</A>" gives an example on how to do it.<A href="https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates" target="_blank" rel="nofollow noopener noreferrer">https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates</A> Mon, 19 Aug 2019 10:14:56 GMT ahadjidj 2019-08-19T10:14:56Z