Member since
04-19-2016
13
Posts
4
Kudos Received
0
Solutions
09-06-2016
11:32 PM
Good Morning, guys Before i will go with the question, first i must to tell you this. I'm college student (IT) in Indonesia. This year is my final year and i want to make a project in Big Data. So, i want to get some data with Apache Nifi dan send it to Output. Afterwards, i process it with Spark and Storm. 1. What Performance Comparison should i do ? Response Time ? Throughput ? Real Time Proccessing Speed ? 2. Is it possible to do that ? Because, i know someone in the internet says benchmark between Spark and Storm inaccurate and there is no paper or journal about this. ( http://www.slideshare.net/ptgoetz/apache-storm-vs-spark-streaming ) 3. If you have a better idea about my final project. Let me know it. Thanks before, Have a nice day.
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache Storm
08-29-2016
06:23 AM
Good Morning, everyone. Before i go with my question, i will start why i need fuzzy algorithm. In my project, i use Arduino to collect some data about temperature, etc. After i collect a lot of data, i want to use Fuzzy Algorithm to predict rainfall in some area. But, i dont find fuzzy algorithm in MLlib's Apache Spark ? What should i do ? How i can use fuzzy algorithm in HDF, or there is another way ? Or can i use python library in Apache Spark ? ; Because i heard there are some fuzzy library in Python, maybe i can use it in Apache Spark Thanks a lot.
... View more
Labels:
- Labels:
-
Apache Spark
08-16-2016
06:44 AM
@Davide Isoardi Good Afternoon, sir.
May i ask you something ? Is http://www.ecubecenter.it/Hadoop-Big-Data-Twitter-Map/ using this https://github.com/disoardi/tweetsdemo_with_ambari_views. It has been a while that i lost my mind to work my undergraduate project. Maybe, i will start again from this. Thanks before
... View more
05-02-2016
03:04 PM
@Ravi Mutyala Hmmm..., I dont know if there are more tutorial about Apache Nifi in this forum than Hortonworks Tutorial. Better to check it, and thanks a lot.
... View more
05-02-2016
03:01 PM
@Pierre Villard I want to send Twitter username from smartphone to HDP. When ListenHTTP get the data, it will send to GetTwitter to get some tweet according username. After that, i will use spark to calculate Machine Learning Algorithm to predict user personality.
... View more
05-02-2016
02:57 PM
@Kirk Haslbeck Thanks, i will try it, one of them as fast as possible
... View more
04-30-2016
02:53 AM
3 Kudos
Hello guys, First, i want to make a project with Apache Nifi, but there is some problem that i face it now. 1. Can i make Machine Learning Calculation in Apache Nifi ? I know that i must use Mahout and Spark, but i dont find Mahout and Spark Processor in Apache Nifi. What should i do ? Can you suggest me another idea. 2. I want to make web service with Apache Nifi ? First time, i use ListenHTTP Processor, but the text that i send become Flow File. How can i convert it to text so i can put it in GetTwitter Processor ? Thanks for reply 🙂
... View more
Labels:
- Labels:
-
Apache NiFi
-
Apache Pig
-
Apache Spark
04-21-2016
09:20 AM
Hmmmm, so the key of this is Apache Nifi. In Nifi, we can create a sequence from get the data, process it and save it according the Processor what we choose. If in the first time, i can load "aloha.csv" and make a table from it manually (Click Files Views and Hive View). So, with Nifi i can load "aloha.csv" and make a table automatically using the Processor. Is it correct ? Well, maybe i try it. I'm sorry if i ask too much because i need to clarify this. Because, if i cant implement this (my final project in the college), i will step back to search another idea.
... View more
04-21-2016
02:32 AM
@Davide Isoardi Thanks for reply my question. But, i have another question. When an App using web service, the sequence will be like this. Android --[JSON Data]--> Web Service --[SQL Query]--> Database. In this case, to proccess the data, we only using Web Service and send SQL Query to Database. From the your answer this is what i get. Android --[Twitter Data]--> Nifi --> Solr --> Spark --> Another Solr Collection --[JSON Format]--> Android In this case, to process the data we need many app to send data. From Android to Nifi, from Nifi to Solr, etc. Can Solr receive data from Nifi after that send to Spark ? Because i never know how to communicate (send/receive data) from one app to another app in HDP automatically like your idea ? I'm sorry, if my question makes you confuse 😉
... View more
04-21-2016
01:37 AM
@Andrea D'Orio Thanks for your support 😉
... View more
04-20-2016
08:43 AM
1 Kudo
Hello anyone, First, i want say sorry because maybe my question is out of the topic. But, i need to ask it because i dont find another forum that discuss about Big Data and how to use it with your own (Using Tutorial that provided by Hortonworks). My question is 1. Can i get data from twitter to HDP ? Is it true that i can use Apache Nifi or Can i use another Software like Apache Kafka ? But How ? 2. When i get the data form Twitter, how to process it ? Can i use Spark or Mahout ? In this case, i want to find the most people personality from twitter, which is the best option ? 3. Then, if i can calculate the data, how i can access it with android aplication ? In my mind, i think i can use web service but i dont know how to make it with HDP ? I will make a clear explanation about this, a. User want to use my app. But, user must login to Twitter before use this app. b. After this, i want to collect some twitter data from user. c. Twitter data passes to HDP using Web services. d. HDP will calculate the data with certain algorithm. e. After that, HDP will send the result the data to android app. f. Voila, use can see his/her personality using twitter data. Can i implement this ? Or maybe i think too much how use HDP ? 4. Do you some another idea for my final project in the college ? Actually, my older lecturer use HDP but only single cluster (Sandbox). And i want to make it multi node, but i dont know how to maximize the potential how to use HDP and real Bid Data. I hope my question with not confuse you guys, why i ask to many question because my lack knowledege how to use HDP and Big Data. I hope i can get the answer and have a nice day :).
... View more
- Tags:
- Data Ingestion & Streaming
- data retrieve from hive
- nifi rss feeds store data into hive tables
- oracle big data discovery
Labels: