Member since
12-09-2015
61
Posts
43
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1710 | 08-05-2016 12:33 PM | |
1735 | 02-27-2016 09:19 AM | |
2539 | 02-19-2016 06:28 PM | |
754 | 02-02-2016 09:09 PM |
06-01-2017
05:28 AM
@Aravindan Vijayan yes, Ive done that part. Its been working since the upgrade. Maybe the documentation should be updated so it is not confusing. I remember running into sth similar whne upgrading to 2.4 last year. Speaking of documentation, it should be more obvious with regards to grafana when upgrading from 2.5 to 2.6, since in 2.5 Grafana is already a part of the stack, but the upgrade docs consider it as a service that must be installed and not how to upgrade it (or if there is an upgrade).
... View more
05-29-2017
11:52 AM
3 Kudos
Im using Ubuntu 14. I am upgrading ambari from 2.4 to 2.5. When upgrading ambari metrics and ambari grafana I get the following error messages: Unable to locate package ambari-metrics-collector Unable to locate package ambari-metrics-grafana Im following the upgrade documentation from here: https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-upgrade/content/upgrade_ambari_metrics.html If I run sudo apt-cache showpkg ambari-* I do get some ambari packages but not these two. What am I missing?
... View more
Labels:
- Labels:
-
Apache Ambari
03-16-2017
01:09 PM
I may be teaching wrong religion and not helping you directly here: I went away from Zeppelin in Ambari and installed it on a client manually - not a biggie. Have full control over it now.
Good luck.
... View more
03-04-2017
06:11 AM
@Simon Jespersen I am not answering to your question directly, just sharing my XML experience: I am using Spark to read XML files and DataFrames are saved to Hive tables for further analysis. Thanks for the Nifi idea, I will look into it.
... View more
08-05-2016
12:33 PM
1 Kudo
I made it work now. its not -X PUT but -X POST curl -iv -u admin:admin -H "X-Requested-By: ambari" -X POST -d '[{"MemberInfo/user_name":"user1", "MemberInfo/group_name":"group1"}]' http://ambari-server:8080/api/v1/groups/group1/members This adds new user to a group without deleting the existing group members.
... View more
08-05-2016
12:27 PM
So the answer to my question would be: File add_to_group.amb: {
"MemberInfo/user_name":"user1",
"MemberInfo/group_name":"group1"
}
curl -iv -u admin:admin -H "X-Requested-By: ambari" -X PUT -d @add_to_group.amb http://ambari-server:8080/api/v1/groups/group1/members This works! Thank you, @sbhat
... View more
08-05-2016
12:27 PM
@sbchat I had to unaccept the answer. What happens now is that the user is added, but the old users from the group are removed from it. Can you check if this behaviour occurs on your side as well?
... View more
08-05-2016
11:28 AM
@sbhat I ran what you recommended under adding user to a group and Im getting 405 Method not allowed. I tried with -X POST rather than PUT and Im getting this error: {
"status" : 400,
"message" : "The properties [MemberInfo/user_name, MemberInfo/group_name] specified in the request or predicate are not supported for the resource type Group."
* Connection #0 to host ambari-server left intact
} so I updated the string by adding members/ in front of MemberInfo, like this: {
"members/MemberInfo/user_name":"user1",
"members/MemberInfo/group_name":"group1"
}
and the error is the same. Im using Ambari 2.2.2.0 if this has to do with anything.
... View more
08-05-2016
09:41 AM
1 Kudo
I would like to add a new user to Ambari Views and add it to a group. Here is my file: {
"Users/user_name": "user1",
"Users/password": "user1",
"Users/groups": ["group1"],
"Users/active": true,
"Users/admin": false
}
So I run the file: curl -iv -u admin:admin -d @user1.amb -H "X-Requested-By: ambari" -X POST http://ambari-server:8080/api/v1/users The user is created, but not added to the group. Here is the GET output for this user: {
"href" : "http://ambari-server:8080/api/v1/users/user1",
"Users" : {
"active" : true,
"admin" : false,
"groups" : [ ],
"ldap_user" : false,
"user_name" : "user1"
},
"widget_layouts" : [ ],
"privileges" : [ ]
} Groups is empty. Does anyone have any idea what Im doing wrong in this user creation process?
... View more
Labels:
- Labels:
-
Apache Ambari
03-09-2016
11:20 AM
@Jorge de la peña Have you checked the log files?
Btw, if you can upgrade to Spark 1.6.0, then you should do it.
... View more
03-08-2016
10:32 AM
@Jorge de la peña
Jorge, next time you should quote the user you are talking to. Pure luck I saw your post.
Thank for posting this, btw. It motivated me to finally put together the How-I-did-it https://markobigdata.wordpress.com/2016/03/08/building-apache-zeppelin-0-6-0-on-spark-1-5-2-in-a-cluster-mode/
Hope it helps!
... View more
02-27-2016
09:19 AM
2 Kudos
@Neeraj Sabharwal Manage to fix it now. Silly problem, actually. I was installing R packages as sudo, but running my zeppelin service as zeppelin user. Now I ran zeppelin as sudo and it works.
... View more
02-27-2016
08:05 AM
1 Kudo
@Neeraj Sabharwal The way I have tested RZeppelin twice now is the following: I ran the RInterpreter and it worked fine. As soon as I try something on my own, I get this error. The second time I installed, I wanted to run your flights example and when I wanted to load the four libraries I already got this error message. btw, Im surprised your installation was successful without installing Cairo package. I had to install cairo package before running install install_github('IRkernel/repr'). If I run the following command in R: #list all installed packages and versions: ip <- as.data.frame(installed.packages()[,c(1,3:4)]) rownames(ip) <- NULL ip <- ip[is.na(ip$Priority),1:2,drop=FALSE] print(ip, row.names=FALSE) Packages and versions you mention in your git (plus Cairo): base64enc 0.1-3 Cairo 1.5-9 caret 6.0-64 devtools 1.10.0 dplyr 0.4.3 evaluate 0.8 repr 0.4 This is the list of all installed packages: r-installed-pkgs.txt My R version: > version _ platform x86_64-pc-linux-gnu arch x86_64 os linux-gnu system x86_64, linux-gnu status major 3 minor 2.3 year 2015 month 12 day 10 svn rev 69752 language R version.string R version 3.2.3 (2015-12-10) nickname Wooden Christmas-Tree
... View more
02-26-2016
03:38 PM
1 Kudo
@Neeraj Sabharwal Ive set up Spark 1.6.0, installed sparkR environment, and built and started zeppelin with R interpreter. I managed to run the RInterpreter notebook. Then I tried to run your example with flights data, I got the same erro message. Now it seems as if I cannot run the zeppelin example either. Same error message: unable to start device X11cairo
... View more
02-25-2016
10:16 AM
@Neeraj Sabharwal Just to help you help me, here is what Ive installed to make it work before the error message: sudo apt-get install libxml2-dev -y
sudo apt-get install libcurl4-openssl-dev -y
sudo apt-get install libssl-dev -y
sudo apt-get install libcurl4-gnutls-dev -y
sudo apt-get install libcairo2-dev -y in R (as root):
install.packages("evaluate", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("base64enc", dependencies = TRUE, repos='http://cran.us.r-project.org')
install.packages("devtools", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages("dplyr", dependencies = TRUE, repos='http://cran.us.r-project.org')
install.packages("caret", dependencies = TRUE, repos='http://cran.us.r-project.org')
install.packages("Cairo", dependencies = TRUE, repos='http://cran.us.r-project.org') install.packages('googleVis', dependencies = TRUE, repos='http://cran.us.r-project.org')
install.packages('mplot', dependencies = TRUE, repos='http://cran.us.r-project.org')
... View more
02-25-2016
07:06 AM
2 Kudos
@Neeraj Sabharwal - seeing your post on linkedin I decided to write here, hopefully someone is doing sparkR on Zeppelin I have installed Zeppelin with R interpreter and at the beginning everything worked well. I could run spark.r and spark.knitr. Then all of a sudden, I started getting the following error message: unable to start device X11cairo when running spark.r. Cairo library is installed, I could run all RInterpreter examples, but after a while this happened.
My zeppelin out file gives me this: ERROR RContext: R Error .zreplout <- rzeppelin:::.z.valuate(.zreplin) unable to start device X11cairo Zeppelin is installed on ubuntu 14.04, Im using spark 1.5.2, sparkR from the CLI runs properly. Ive used this git: https://github.com/elbamos/Zeppelin-With-R What am I missing?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache Zeppelin
02-23-2016
09:12 AM
4 Kudos
According to the Databricks people at the Spark Summit 2016 in New York, Spark 2.0 is coming out in April/May this year. This means a "jump" from 1.6.0 to 2.0. Spark 1.5.2 is installed with the latest Hortonworks distribution. Spark 1.6.0 is in technical preview, Spark 2.0 is around the corner. Im wondering what plans Hortonworks has around this matter?
... View more
Labels:
- Labels:
-
Apache Spark
02-23-2016
08:12 AM
1 Kudo
@vbhoomireddy @Neeraj Sabharwal I have now made it work. Zeppelin 0.6.0 works now on Spark 1.5.2. If this is still of someone's interest let me know, I can describe what I did.
... View more
02-22-2016
09:45 PM
@Neeraj Sabharwal I needed some more time to test it on spark 1.4.1 and 1.5.2. I also put it in a nice form. Here is the process Ive taken in order to get sparkR running: https://markobigdata.wordpress.com/2016/02/22/installing-r-on-hadoop-cluster-to-run-sparkr/
... View more
02-19-2016
06:28 PM
2 Kudos
@Neeraj Sabharwal, @Artem Ervits I made it work now! Ive used Ubuntu 14.04 Trusty, installed manually Spark 1.4.1 and set up sparkR.
Now, I dont know if the problem was in centos 7.2, but the installment of R was different than what Ive done earlier and from what it says here: http://www.jason-french.com/blog/2013/03/11/installing-r-in-linux/ If you guys want, I can try the same on centos 7.2 and report. If you want, I can describe the process of preparing the environment for using sparkR. I will also try on other spark versions. We depend on R because of the research. Let me know if there is interest.
... View more
02-19-2016
07:46 AM
2 Kudos
@Neeraj Sabharwal, @Artem Ervits Im going to give it a try on Ubuntu 14.04 today. Neeraj, you mentioned some other people are having the same problems. Can you give more information. How did they solve it?
... View more
02-17-2016
01:36 PM
1 Kudo
@Neeraj Sabharwal I have now installed 1.5.2 on one of the datanodes, I can run spark-shell, when I run sparkR I get this issue: Launching java with spark-submit command /usr/hdp/2.3.4.0-3485/spark/bin/spark-submit "sparkr-shell" /tmp/Rtmp6iFIT6/backend_port15af7623f5dd 16/02/17 14:35:18 WARN internal.ThreadLocalRandom: Failed to generate a seed from SecureRandom within 3 seconds. Not enough entrophy? Error in socketConnection(port = monitorPort) :
cannot open the connection In addition: Warning message:
In socketConnection(port = monitorPort) : localhost:44867 cannot be opened
Pretty much the same problem.
... View more
02-17-2016
01:20 PM
1 Kudo
@Neeraj Sabharwal Ive manually installed 1.6.0 on the same node where Im running spark 1.5.2.
Now I ll try doing it on one datanode - I ll install 1.5.2 manually.
By the way, Neeraj, have you seen the CentOS 7.2 dialog Im having with Artem. Let me know if you have a take on that one.
... View more
02-17-2016
12:58 PM
@Artem Ervits, @Neeraj Sabharwal
Well
Here is my version:
CentOS Linux release 7.2.1511 (Core)
The cloud services I use have, in their menu, CentOS 7.1, I think they upgraded the systems, left 7.1 on the menu but the instances are 7.2
Now im a bit upset. Do you think this could really be it? That 7.2 version is the problem?
... View more
02-17-2016
12:47 PM
1 Kudo
@Neeraj Sabharwal Nothing nice about this issue, its a show stopper if you work for researchers. 🙂 What do you mean by different node? And run it how?
... View more
02-17-2016
12:38 PM
1 Kudo
@Neeraj Sabharwal
No, the spark installation was done through Ambari.
I installed, accepted all the defaults and ran the test examples from the TP website.Funny thing, when installing, you get to choose 1.4.2 version of spark, while 1.5.2 is installed.
... View more
02-17-2016
12:01 PM
1 Kudo
@Artem ErvitsI tried that as well. Here the status command: sudo service iptables status And the result: Redirecting to /bin/systemctl status iptables.service
● iptables.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
I tried running the iptables and firewalld commands on the datanodes and they all seem inactive (same results as above). Centos 7.2.
... View more
02-17-2016
11:24 AM
1 Kudo
@Artem Ervits I ran: sudo systemctl status firewalld And the result is this: firewalld.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
... View more
02-17-2016
09:27 AM
Now I installed Spark 1.6.0 just to test if Ambari makes some changes during spark installation:
Same result: Error in socketConnection(port = monitorPort) :
cannot open the connection
In addition: Warning message:
In socketConnection(port = monitorPort) : localhost:51604 cannot be opened
Could it be YARN?
... View more
02-17-2016
08:45 AM
@Neeraj Sabharwal Ive ran the command you recommended and Im getting an error saying Error: Could not find or load main class log I have setup another cluster - namenode + 3 datanodes using ambari. Ive followed this link: http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/ I installed R on all the nodes. All examples worked until I came to the sparkR: Launching java with spark-submit command /usr/hdp/2.3.4.0-3485/spark/bin/spark-submit "sparkr-shell" /tmp/Rtmphs2DlM/backend_port3b7b4c9a912b
16/02/17 09:37:36 WARN SparkConf: The configuration key 'spark.yarn.applicationMaster.waitTries' has been deprecated as of Spark 1.3 and and may be removed in the future. Please use the new key 'spark.yarn.am.waitTime' instead. Error in socketConnection(port = monitorPort) : cannot open the connection
In addition: Warning message: In socketConnection(port = monitorPort) : localhost:40949 cannot be opened > Ive opened all the ports (1-65535) and port 0 for the namenode and the datanodes. @Artem Ervits - do you have any idea what I am missing?
... View more