Do you have a target variable that you can predict? Or do you have logic that will allow you to convert a "low" CPU value into a target variable? Spark has a wide variety of models that are available for classification modeling: https://spark.apache.org/docs/latest/mllib-classification-regression.html If you are interested in seeing which factor is contributing to a specific instance, I would recommend starting with a logistic regression model as that will provide more explanatory power -- providing more insight into which factor is contributing to a particular CPU failure ... View more

@Gayathri Devi, This prediction depends on the date you have. You may have labelled or unlabelled data based on which you have different algorithms. Assuming your data is labelled, then you have to find if you are trying to solve a regression problem or a classification problem. Based on that you can choose the algorithms. Since you have written that you want to find outliers , I'm assuming that it is a regression problem. Then you can use algorithms like Linear Regression, Support Vector Regression, Decision tree regression, Random forest regression etc. If your data is unlabelled, you have to use a unsupervised learning method. You will have algorithms like K-Means clustering, Hierarchical clustering etc. The main part of any solving machine learning problem is learning what your data is and choosing the right algorithm for your problem. So you may need to spend more time in analysing data and choosing the right algorithm. Here are few links for the concepts mentioned above. You can find these algorithms in spark. https://spark.apache.org/docs/latest/ml-guide.html https://machinelearningmastery.com/classification-versus-regression-in-machine-learning/ https://www.quora.com/What-is-the-main-difference-between-classification-problems-and-regression-problems-in-machine-learning https://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/ https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabeled-data Happy machine learning 🙂 . -Aditya ... View more

@Gayathri Devi Could you try with below query as we are reading 2018-02-27T02:00 value and converting as timestamp value. Query:- hive> select from_unixtime(unix_timestamp('2018-02-27T02:00',"yyyy-MM-dd'T'hh:mm"),'yyyy-MM-dd hh:mm:ss'); +----------------------+--+ | _c0 | +----------------------+--+ | 2018-02-27 02:00:00 | +----------------------+--+ (or) By using regexp_replace function we can replace T in your timestamp value hive> select regexp_replace('2018-02-27T02:00','T',' '); +-------------------+--+ | _c0 | +-------------------+--+ | 2018-02-27 02:00 | +-------------------+--+ And use concat function to add missing :00 value to make above value as hive timestamp. hive> select concat(regexp_replace('2018-02-27T02:00','T',' '),":00"); +----------------------+--+ | _c0 | +----------------------+--+ | 2018-02-27 02:00:00 | +----------------------+--+ ... View more

@Gayathri Devi, You can use the below script . beeline -u "{connection-string}" -e "show tables" | grep $1 if [ $? -eq 0 ] then echo "table found" else echo "table not found" fi But the content in a file say checktable.sh and run the below steps chmod +x checktable.sh ./checktable.sh {tablename to check} Thanks, Aditya ... View more

@Gayathri Devi, Can you try this query insert into table tblename select * from (select from_unixtime(unix_timestamp('161223000001', 'yyMMddHHmmss')))b; #2) If you have timestamp as '1506614501' hive> select from_unixtime(unix_timestamp('1506614501', 'yyMMddHHmm')); OK 2015-08-01 21:01:00 Time taken: 0.257 seconds, Fetched: 1 row(s) Thanks, Aditya ... View more