Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Data Analysis using Mahout

Solved Go to solution

Data Analysis using Mahout

New Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Data Analysis using Mahout

@Gayathri Devi,

This prediction depends on the date you have. You may have labelled or unlabelled data based on which you have different algorithms.

Assuming your data is labelled, then you have to find if you are trying to solve a regression problem or a classification problem. Based on that you can choose the algorithms.

Since you have written that you want to find outliers , I'm assuming that it is a regression problem. Then you can use algorithms like Linear Regression, Support Vector Regression, Decision tree regression, Random forest regression etc.

If your data is unlabelled, you have to use a unsupervised learning method. You will have algorithms like K-Means clustering, Hierarchical clustering etc.

The main part of any solving machine learning problem is learning what your data is and choosing the right algorithm for your problem. So you may need to spend more time in analysing data and choosing the right algorithm.

Here are few links for the concepts mentioned above. You can find these algorithms in spark.

https://spark.apache.org/docs/latest/ml-guide.html

https://machinelearningmastery.com/classification-versus-regression-in-machine-learning/

https://www.quora.com/What-is-the-main-difference-between-classification-problems-and-regression-pro...

https://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/

https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabeled-da...

Happy machine learning :)

.

-Aditya

3 REPLIES 3

Re: Data Analysis using Mahout

New Contributor

If you are beginner and want to start with ML, I'd suggest ditching Mahout and better learn Spark. Mahout is older project, which uses MapReduce. Spark on the other hand is in memory processing, and way more developed.

There's literally no one that uses Mahout... everyone is focusing on Spark:

https://hortonworks.com/apache/spark/

Check out some nice Spark tutorials we have:

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

https://hortonworks.com/hadoop-tutorial/interacting-with-data-on-hdp-using-scala-and-apache-spark/

Highlighted

Re: Data Analysis using Mahout

New Contributor

@Ginta

Thanks.

As you suggested i M using spark now. Must i use MLIB ?

I have a csv file in spark now. val mk = sc.textFile("hdfs:path filenmae.csv");

I have 4 string values and 3 double values.

I need to calculate the outlier now and need to apply any prediction modelling with results in a visualized way.

What can i use now? Any suggestions? Thanks.

Re: Data Analysis using Mahout

@Gayathri Devi,

This prediction depends on the date you have. You may have labelled or unlabelled data based on which you have different algorithms.

Assuming your data is labelled, then you have to find if you are trying to solve a regression problem or a classification problem. Based on that you can choose the algorithms.

Since you have written that you want to find outliers , I'm assuming that it is a regression problem. Then you can use algorithms like Linear Regression, Support Vector Regression, Decision tree regression, Random forest regression etc.

If your data is unlabelled, you have to use a unsupervised learning method. You will have algorithms like K-Means clustering, Hierarchical clustering etc.

The main part of any solving machine learning problem is learning what your data is and choosing the right algorithm for your problem. So you may need to spend more time in analysing data and choosing the right algorithm.

Here are few links for the concepts mentioned above. You can find these algorithms in spark.

https://spark.apache.org/docs/latest/ml-guide.html

https://machinelearningmastery.com/classification-versus-regression-in-machine-learning/

https://www.quora.com/What-is-the-main-difference-between-classification-problems-and-regression-pro...

https://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/

https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabeled-da...

Happy machine learning :)

.

-Aditya

Don't have an account?
Coming from Hortonworks? Activate your account here