question Re: Data Analysis using Mahout in Archives of Support Questions (Read Only)

Data Analysis using Mahout

Gayathridevi — Fri, 16 Sep 2022 13:02:18 GMT

Re: Data Analysis using Mahout

gkupcovaite — Tue, 27 Mar 2018 20:23:32 GMT

If you are beginner and want to start with ML, I'd suggest ditching Mahout and better learn Spark. Mahout is older project, which uses MapReduce. Spark on the other hand is in memory processing, and way more developed.

There's literally no one that uses Mahout... everyone is focusing on Spark:

https://hortonworks.com/apache/spark/

Check out some nice Spark tutorials we have:

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

https://hortonworks.com/hadoop-tutorial/interacting-with-data-on-hdp-using-scala-and-apache-spark/

Re: Data Analysis using Mahout

Gayathridevi — Wed, 28 Mar 2018 09:23:36 GMT

@Ginta

Thanks.

As you suggested i M using spark now. Must i use MLIB ?

I have a csv file in spark now. val mk = sc.textFile("hdfs:path filenmae.csv");

I have 4 string values and 3 double values.

I need to calculate the outlier now and need to apply any prediction modelling with results in a visualized way.

What can i use now? Any suggestions? Thanks.

Re: Data Analysis using Mahout

asirna — Wed, 28 Mar 2018 13:06:09 GMT

@Gayathri Devi,

This prediction depends on the date you have. You may have labelled or unlabelled data based on which you have different algorithms.

Assuming your data is labelled, then you have to find if you are trying to solve a regression problem or a classification problem. Based on that you can choose the algorithms.

Since you have written that you want to find outliers , I'm assuming that it is a regression problem. Then you can use algorithms like Linear Regression, Support Vector Regression, Decision tree regression, Random forest regression etc.

If your data is unlabelled, you have to use a unsupervised learning method. You will have algorithms like K-Means clustering, Hierarchical clustering etc.

The main part of any solving machine learning problem is learning what your data is and choosing the right algorithm for your problem. So you may need to spend more time in analysing data and choosing the right algorithm.

Here are few links for the concepts mentioned above. You can find these algorithms in spark.

https://spark.apache.org/docs/latest/ml-guide.html

https://machinelearningmastery.com/classification-versus-regression-in-machine-learning/

https://www.quora.com/What-is-the-main-difference-between-classification-problems-and-regression-problems-in-machine-learning

https://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/

https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabeled-data

Happy machine learning 🙂

-Aditya