@Aravindan Vijayan yes, Ive done that part. Its been working since the upgrade. Maybe the documentation should be updated so it is not confusing. I remember running into sth similar whne upgrading to 2.4 last year. Speaking of documentation, it should be more obvious with regards to grafana when upgrading from 2.5 to 2.6, since in 2.5 Grafana is already a part of the stack, but the upgrade docs consider it as a service that must be installed and not how to upgrade it (or if there is an upgrade). ... View more

I may be teaching wrong religion and not helping you directly here: I went away from Zeppelin in Ambari and installed it on a client manually - not a biggie. Have full control over it now. Good luck. ... View more

@Simon Jespersen I am not answering to your question directly, just sharing my XML experience: I am using Spark to read XML files and DataFrames are saved to Hive tables for further analysis. Thanks for the Nifi idea, I will look into it. ... View more

So the answer to my question would be: File add_to_group.amb: { "MemberInfo/user_name":"user1", "MemberInfo/group_name":"group1" } curl -iv -u admin:admin -H "X-Requested-By: ambari" -X PUT -d @add_to_group.amb http://ambari-server:8080/api/v1/groups/group1/members This works! Thank you, @sbhat ... View more

@sbchat I had to unaccept the answer. What happens now is that the user is added, but the old users from the group are removed from it. Can you check if this behaviour occurs on your side as well? ... View more

@sbhat I ran what you recommended under adding user to a group and Im getting 405 Method not allowed. I tried with -X POST rather than PUT and Im getting this error: { "status" : 400, "message" : "The properties [MemberInfo/user_name, MemberInfo/group_name] specified in the request or predicate are not supported for the resource type Group." * Connection #0 to host ambari-server left intact } so I updated the string by adding members/ in front of MemberInfo, like this: { "members/MemberInfo/user_name":"user1", "members/MemberInfo/group_name":"group1" } and the error is the same. Im using Ambari 2.2.2.0 if this has to do with anything. ... View more

@Jorge de la peña Have you checked the log files? Btw, if you can upgrade to Spark 1.6.0, then you should do it. ... View more

@Jorge de la peña Jorge, next time you should quote the user you are talking to. Pure luck I saw your post. Thank for posting this, btw. It motivated me to finally put together the How-I-did-it https://markobigdata.wordpress.com/2016/03/08/building-apache-zeppelin-0-6-0-on-spark-1-5-2-in-a-cluster-mode/ Hope it helps! ... View more

@Neeraj Sabharwal The way I have tested RZeppelin twice now is the following: I ran the RInterpreter and it worked fine. As soon as I try something on my own, I get this error. The second time I installed, I wanted to run your flights example and when I wanted to load the four libraries I already got this error message. btw, Im surprised your installation was successful without installing Cairo package. I had to install cairo package before running install install_github('IRkernel/repr'). If I run the following command in R: #list all installed packages and versions: ip <- as.data.frame(installed.packages()[,c(1,3:4)]) rownames(ip) <- NULL ip <- ip[is.na(ip$Priority),1:2,drop=FALSE] print(ip, row.names=FALSE) Packages and versions you mention in your git (plus Cairo): base64enc 0.1-3 Cairo 1.5-9 caret 6.0-64 devtools 1.10.0 dplyr 0.4.3 evaluate 0.8 repr 0.4 This is the list of all installed packages: r-installed-pkgs.txt My R version: > version _ platform x86_64-pc-linux-gnu arch x86_64 os linux-gnu system x86_64, linux-gnu status major 3 minor 2.3 year 2015 month 12 day 10 svn rev 69752 language R version.string R version 3.2.3 (2015-12-10) nickname Wooden Christmas-Tree ... View more

@Neeraj Sabharwal Ive set up Spark 1.6.0, installed sparkR environment, and built and started zeppelin with R interpreter. I managed to run the RInterpreter notebook. Then I tried to run your example with flights data, I got the same erro message. Now it seems as if I cannot run the zeppelin example either. Same error message: unable to start device X11cairo ... View more

@Neeraj Sabharwal Just to help you help me, here is what Ive installed to make it work before the error message: sudo apt-get install libxml2-dev -y sudo apt-get install libcurl4-openssl-dev -y sudo apt-get install libssl-dev -y sudo apt-get install libcurl4-gnutls-dev -y sudo apt-get install libcairo2-dev -y in R (as root): install.packages("evaluate", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("base64enc", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("devtools", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("dplyr", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("caret", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("Cairo", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages('googleVis', dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages('mplot', dependencies = TRUE, repos='http://cran.us.r-project.org') ... View more

@vbhoomireddy @Neeraj Sabharwal I have now made it work. Zeppelin 0.6.0 works now on Spark 1.5.2. If this is still of someone's interest let me know, I can describe what I did. ... View more

@Neeraj Sabharwal I needed some more time to test it on spark 1.4.1 and 1.5.2. I also put it in a nice form. Here is the process Ive taken in order to get sparkR running: https://markobigdata.wordpress.com/2016/02/22/installing-r-on-hadoop-cluster-to-run-sparkr/ ... View more

@Neeraj Sabharwal, @Artem Ervits Im going to give it a try on Ubuntu 14.04 today. Neeraj, you mentioned some other people are having the same problems. Can you give more information. How did they solve it? ... View more

@Neeraj Sabharwal I have now installed 1.5.2 on one of the datanodes, I can run spark-shell, when I run sparkR I get this issue: Launching java with spark-submit command /usr/hdp/2.3.4.0-3485/spark/bin/spark-submit "sparkr-shell" /tmp/Rtmp6iFIT6/backend_port15af7623f5dd 16/02/17 14:35:18 WARN internal.ThreadLocalRandom: Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy? Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:44867 cannot be opened Pretty much the same problem. ... View more

@Neeraj Sabharwal Ive manually installed 1.6.0 on the same node where Im running spark 1.5.2. Now I ll try doing it on one datanode - I ll install 1.5.2 manually. By the way, Neeraj, have you seen the CentOS 7.2 dialog Im having with Artem. Let me know if you have a take on that one. ... View more

@Artem Ervits, @Neeraj Sabharwal Well Here is my version: CentOS Linux release 7.2.1511 (Core) The cloud services I use have, in their menu, CentOS 7.1, I think they upgraded the systems, left 7.1 on the menu but the instances are 7.2 Now im a bit upset. Do you think this could really be it? That 7.2 version is the problem? ... View more

@Neeraj Sabharwal Nothing nice about this issue, its a show stopper if you work for researchers. 🙂 What do you mean by different node? And run it how? ... View more

@Neeraj Sabharwal No, the spark installation was done through Ambari. I installed, accepted all the defaults and ran the test examples from the TP website.Funny thing, when installing, you get to choose 1.4.2 version of spark, while 1.5.2 is installed. ... View more

@Artem ErvitsI tried that as well. Here the status command: sudo service iptables status And the result: Redirecting to /bin/systemctl status iptables.service ● iptables.service Loaded: not-found (Reason: No such file or directory) Active: inactive (dead) I tried running the iptables and firewalld commands on the datanodes and they all seem inactive (same results as above). Centos 7.2. ... View more

@Artem Ervits I ran: sudo systemctl status firewalld And the result is this: firewalld.service Loaded: not-found (Reason: No such file or directory) Active: inactive (dead) ... View more

Now I installed Spark 1.6.0 just to test if Ambari makes some changes during spark installation: Same result: Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:51604 cannot be opened Could it be YARN? ... View more

@Neeraj Sabharwal Ive ran the command you recommended and Im getting an error saying Error: Could not find or load main class log I have setup another cluster - namenode + 3 datanodes using ambari. Ive followed this link: http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/ I installed R on all the nodes. All examples worked until I came to the sparkR: Launching java with spark-submit command /usr/hdp/2.3.4.0-3485/spark/bin/spark-submit "sparkr-shell" /tmp/Rtmphs2DlM/backend_port3b7b4c9a912b 16/02/17 09:37:36 WARN SparkConf: The configuration key 'spark.yarn.applicationMaster.waitTries' has been deprecated as of Spark 1.3 and and may be removed in the future. Please use the new key 'spark.yarn.am.waitTime' instead. Error in socketConnection(port = monitorPort) : cannot open the connection In addition: Warning message: In socketConnection(port = monitorPort) : localhost:40949 cannot be opened > Ive opened all the ports (1-65535) and port 0 for the namenode and the datanodes. @Artem Ervits - do you have any idea what I am missing? ... View more