Member since
11-07-2016
637
Posts
253
Kudos Received
144
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2286 | 12-06-2018 12:25 PM | |
2341 | 11-27-2018 06:00 PM | |
1814 | 11-22-2018 03:42 PM | |
2881 | 11-20-2018 02:00 PM | |
5230 | 11-19-2018 03:24 PM |
04-08-2018
02:46 PM
@Michael Bronson, I do not see any error logs in the logs pasted. I see only warnings. Is something failing because of this.?
... View more
04-08-2018
01:52 PM
@Henry Leung, You can use ssh to run the script in remote machine. ssh username@hostname.com "python script.py" . Aditya
... View more
04-08-2018
01:47 PM
@Michael Bronson, Yes. 'master01.sys200.ve.com' is valid domain name. Make sure that domain names are in lower case only. It should not be in mixed case/ capitals. . -Aditya
... View more
04-01-2018
10:45 AM
@Aishwarya Sudhakar, There is a difference between /demo/dataset.csv and demo/dataset.csv. The slash makes a difference. If -cat demo/dataset.csv gives you the file output, then you have to use the same path in spark scale code Change your code like below val df = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load("demo/dataset.csv") . Aditya
... View more
04-01-2018
09:51 AM
As you can see the output of ls /demo , there is no dataset.csv in /demo folder. May be it is in /demo/data. Please check the correct path for the input csv file. Can you please try running this command su hdfs hdfs dfs -ls -R /demo
... View more
04-01-2018
08:12 AM
@Aishwarya Sudhakar, Can you please paste the output of below commands su hdfs
hdfs dfs -ls /demo Also try giving load("hdfs://{namnodehost}:8020/demo/dataset.csv")
... View more
03-29-2018
10:08 AM
1 Kudo
Failed to list function from phoenix sqlline.py Command run : select * from SYSTEM.FUNCTION; Error: ERROR 604 (42P00): Syntax error. Mismatched input. Expecting "NAME", got "FUNCTION" at line 1, column 22. (state=42P00,code=604)
org.apache.phoenix.exception.PhoenixParserException: ERROR 604 (42P00): Syntax error. Mismatched input. Expecting "NAME", got "FUNCTION" at line 1, column 22.
at org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
at org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1280)
at org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1363)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1434)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: MismatchedTokenException(65!=99)
at org.apache.phoenix.parse.PhoenixSQLParser.recoverFromMismatchedToken(PhoenixSQLParser.java:360)
at org.apache.phoenix.shaded.org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at org.apache.phoenix.parse.PhoenixSQLParser.parseNoReserved(PhoenixSQLParser.java:9986)
at org.apache.phoenix.parse.PhoenixSQLParser.identifier(PhoenixSQLParser.java:9953)
at org.apache.phoenix.parse.PhoenixSQLParser.from_table_name(PhoenixSQLParser.java:9606)
at org.apache.phoenix.parse.PhoenixSQLParser.table_factor(PhoenixSQLParser.java:6261)
at org.apache.phoenix.parse.PhoenixSQLParser.table_ref(PhoenixSQLParser.java:6083)
at org.apache.phoenix.parse.PhoenixSQLParser.table_list(PhoenixSQLParser.java:6019)
at org.apache.phoenix.parse.PhoenixSQLParser.parseFrom(PhoenixSQLParser.java:5984)
at org.apache.phoenix.parse.PhoenixSQLParser.single_select(PhoenixSQLParser.java:4612)
at org.apache.phoenix.parse.PhoenixSQLParser.unioned_selects(PhoenixSQLParser.java:4714)
at org.apache.phoenix.parse.PhoenixSQLParser.select_node(PhoenixSQLParser.java:4780)
at org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:789)
at org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:508)
at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
... 9 more Issue: FUNCTION is a key word so this command doesn't work. Resolution : Run select * from SYSTEM."FUNCTION"; to list functions
... View more
Labels:
03-28-2018
07:49 AM
1 Kudo
@Archana Thirumala Reddy, Spark 1.6 is used for the exam. Please find more details in the links. https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/05/HDCD_Spark_Data_Sheet.pdf https://hortonworks.com/services/training/certification/exam-objectives/#hdpcdspark Please accept the answer if this helps. This will be useful for other community users. . Aditya
... View more
03-28-2018
06:06 AM
3 Kudos
@Gayathri Devi, This prediction depends on the date you have. You may have labelled or unlabelled data based on which you have different algorithms. Assuming your data is labelled, then you have to find if you are trying to solve a regression problem or a classification problem. Based on that you can choose the algorithms. Since you have written that you want to find outliers , I'm assuming that it is a regression problem. Then you can use algorithms like Linear Regression, Support Vector Regression, Decision tree regression, Random forest regression etc. If your data is unlabelled, you have to use a unsupervised learning method. You will have algorithms like K-Means clustering, Hierarchical clustering etc. The main part of any solving machine learning problem is learning what your data is and choosing the right algorithm for your problem. So you may need to spend more time in analysing data and choosing the right algorithm. Here are few links for the concepts mentioned above. You can find these algorithms in spark. https://spark.apache.org/docs/latest/ml-guide.html https://machinelearningmastery.com/classification-versus-regression-in-machine-learning/ https://www.quora.com/What-is-the-main-difference-between-classification-problems-and-regression-problems-in-machine-learning https://machinelearningmastery.com/supervised-and-unsupervised-machine-learning-algorithms/ https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabeled-data Happy machine learning 🙂 . -Aditya
... View more
03-27-2018
04:24 PM
1 Kudo
@Michael Bronson, This is the curl call for start curl -u $USER:$PASSWD -i -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo":{"context":"_PARSE_.START.AMBARI_METRICS","operation_level":{"level":"SERVICE","cluster_name":"hdp","service_name":"AMBARI_METRICS"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' http://localhost:8080/api/v1/clusters/hdp/services/AMBARI_METRICS
... View more