Member since
04-06-2018
16
Posts
1
Kudos Received
0
Solutions
07-20-2018
04:28 PM
@rguruvannagari Do you have an idea about that issue and how to workaround it? I think it may be related to the fact there is no entry for myuser in user_groups variable defined in /var/lib/ambari-agent/data/command-3409.json. However, I am not sure how this one is built. Any pointers very much appreciated. Thanks!
... View more
07-20-2018
05:33 AM
@Jay Kumar SenSharma, @Geoffrey Shelton Okot Are you able to help on the above? Many thanks!
... View more
07-20-2018
05:14 AM
I reverted back to HDP 2.6.4, removed spark and did the upgrade
... View more
07-19-2018
04:02 PM
1 Kudo
Hi All, I have been trying to add a customer service through Ambari 2.7.0 (NOTE: that works fine with the previous versions). After selecting the service from Choose Serivces, the system tries to add a client first. But this comes back with the following error: Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 35, in <module>
BeforeAnyHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
method(env)
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 29, in hook
setup_users()
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 50, in setup_users
groups = params.user_to_groups_dict[user],
KeyError: 'myuser'
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-3409.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-3409.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', ''] Should myuser be part of the Service Users and Groups - see the below table: screen-shot-2018-07-19-at-45504-pm.png If so, how can I add one in there?
... View more
Labels:
- Labels:
-
Apache Ambari
07-16-2018
04:17 PM
@Geoffrey Shelton Okot My installation is not a fresh one. It is an upgrade from HDP 2.6.4 to 3.0 and Ambari from 2.6.1 to 2.7.0. Any idea?
... View more
07-16-2018
11:40 AM
@Jay Kumar SenSharma Can you help out on this? Many thanks!
... View more
07-16-2018
07:37 AM
I have fixed my problem by restoring my cluster from the backup.
... View more
07-16-2018
07:35 AM
Installing Livy for Spark2 returns Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 35, in <module>
BeforeAnyHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
method(env)
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 29, in hook
setup_users()
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 50, in setup_users
groups = params.user_to_groups_dict[user],
KeyError: u'livy'
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-3302.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-3302.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']
Checked the apache JIRA and found that it should have already been fixed: https://issues.apache.org/jira/browse/AMBARI-24021
... View more
Labels:
- Labels:
-
Apache Spark
07-15-2018
09:11 PM
@Geoffrey Shelton Okot I did check the upgrade table. Here is what I have:
... View more
07-15-2018
09:09 PM
@rguruvannagari Checked all the tables, there are no entries. Is there any way I can start the process from scratch? I am currently stack at the "Update Target Repositories" step with the error described initially.
... View more
07-15-2018
09:14 AM
Hi @rguruvannagari Thank you for your feedback! I did remove the SPARK services using Ambari. Carry on with the upgrade process still fails at the same stage. I also checked some postgress DB tables hostcomponentdesiredstate, hostcomponentstate, servicecomponentdesiredstate, clusterservices. The service is not there. Any other pointers please? Or maybe there is a way to start the process from the beginning? Thanks
... View more
07-15-2018
06:50 AM
Hi all, I have successfully upgraded Ambari to 2.7.0.0 During the HDP upgrade the process stopped at "Update Target Repositories". Updating the desired repository version to 3.0.0.0-1634 for all cluster services. with the full error org.apache.ambari.server.ServiceNotFoundException: Service not found, clusterName=HDP26, serviceName=SPARK
at org.apache.ambari.server.state.cluster.ClusterImpl.getService(ClusterImpl.java:883)
at org.apache.ambari.server.state.UpgradeHelper.setDesiredRepositories(UpgradeHelper.java:922)
at org.apache.ambari.server.state.UpgradeHelper.updateDesiredRepositoriesAndConfigs(UpgradeHelper.java:894)
at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:118)
at org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction.updateDesiredRepositoryVersion(UpdateDesiredRepositoryAction.java:160)
at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
at org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction.execute(UpdateDesiredRepositoryAction.java:96)
at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:550)
at org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:466)
at java.lang.Thread.run(Thread.java:745) Any idea how to get around it? Many thanks!
... View more
Labels:
- Labels:
-
Hortonworks Data Platform (HDP)
06-02-2018
08:42 AM
I have now got it working! Review and reformat your HCC https://community.hortonworks.com/content/supportkb/48793/configuring-httpfs-to-support-kerberos.html. It points out httpfs-site.xml to i.e. incorrect location for keytabs.
... View more
06-01-2018
04:20 PM
@Robert Levas Thanks for coming back. Unfortunately, this does not help. All works fine in a cluster apart from HTTPFS with Kerberos enabled. Running curl command I am getting: [hdfs@ip-10-10-0-137 ~]$ curl http://ip-10-10-0-137.ec2.internal:14000/webhdfs/v1/user?user.name=hdfs&op=LISTSTATUS
[1] 27070
[hdfs@ip-10-10-0-137 ~]$ <html><head><title>Apache Tomcat/6.0.48 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The requested resource is not available.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.48</h3></body></html>
NOTE: I changed my httpfs from .93 to .137. Any other clue?
... View more
05-17-2018
12:24 AM
Hi All, I have recently built HDP 2.6.4.0-91 with Ambari 2.6.1.0 on CentOS 7.4. Installed HTTPFS and enabled SSL for it. This worked fine. Running http://10.10.0.93:14000/webhdfs/v1/user/sample_user/?op=LISTSTATUS returned the correct output. Now, I kerberized my cluster and followed up the steps to kerberize HTTPFS: https://community.hortonworks.com/content/supportkb/48793/configuring-httpfs-to-support-kerberos.html Running:
WebBrowser: http://10.10.0.93:14000/webhdfs/v1/user/sample_user/?op=LISTSTATUS Terminal: curl -v --negotiate -X GET 'http://10.10.0.93:14000/webhdfs/v1/user/sample_user/?op=LISTSTATUS' I am getting the 404 error. [root@ip-10-10-0-93 ~]# curl -v --negotiate -X GET 'http://10.10.0.93:14000/webhdfs/v1/user/sample_user/?op=LISTSTATUS' * About to connect() to 10.10.0.93 port 14000 (#0)
* Trying 10.10.0.93...
* Connected to 10.10.0.93 (10.10.0.93) port 14000 (#0)
> GET /webhdfs/v1/user/sample_user/?op=LISTSTATUS HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.10.0.93:14000
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: Apache-Coyote/1.1
< Content-Type: text/html;charset=utf-8
< Content-Length: 949
< Date: Wed, 16 May 2018 15:26:42 GMT
<
* Connection #0 to host 10.10.0.93 left intact
<html><head><title>Apache Tomcat/6.0.48 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The requested resource is not available.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.48</h3></body></html> Here is my httpfs-site.xml <configuration>
<!-- HUE proxy user setting -->
<property>
<name>httpfs.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>httpfs.proxyuser.hue.groups</name>
<value>*</value>
</property>
<property>
<name>httpfs.hadoop.config.dir</name>
<value>/etc/hadoop/conf</value>
</property>
<property>
<name>httpfs.authentication.type</name>
<value>kerberos</value>
</property>
<property>
<name>httpfs.hadoop.authentication.type</name>
<value>kerberos</value>
</property>
<property>
<name>httpfs.authentication.kerberos.principal</name>
<value>HTTP/ip-10-10-0-93.ec2.internal@SUPPORT.COM</value>
</property>
<property>
<name>httpfs.authentication.kerberos.keytab</name>
<value>/etc/hadoop-httpfs/conf/httpfs-http.keytab</value>
</property>
<property>
<name>httpfs.hadoop.authentication.kerberos.principal</name>
<value>httpfs/ip-10-10-0-93.ec2.internal@SUPPORT.COM</value>
</property>
<property>
<name>httpfs.hadoop.authentication.kerberos.keytab</name>
<value>/etc/security/keytabs/httpfs-http.keytab</value>
</property>
<property>
<name>httpfs.authentication.kerberos.name.rules</name>
<value>RULE:[1:$1@$0](ambari-qa-hdp26@SUPPORT.COM)s/.*/ambari-qa/
RULE:[1:$1@$0](hdfs-hdp26@SUPPORT.COM)s/.*/hdfs/
RULE:[1:$1@$0](spark-hdp26@SUPPORT.COM)s/.*/spark/
RULE:[1:$1@$0](.*@SUPPORT.COM)s/@.*//
RULE:[2:$1@$0](beacon@SUPPORT.COM)s/.*/beacon/
RULE:[2:$1@$0](dn@SUPPORT.COM)s/.*/hdfs/
RULE:[2:$1@$0](hive@SUPPORT.COM)s/.*/hive/
RULE:[2:$1@$0](jhs@SUPPORT.COM)s/.*/mapred/
RULE:[2:$1@$0](knox@SUPPORT.COM)s/.*/knox/
RULE:[2:$1@$0](nm@SUPPORT.COM)s/.*/yarn/
RULE:[2:$1@$0](nn@SUPPORT.COM)s/.*/hdfs/
RULE:[2:$1@$0](rm@SUPPORT.COM)s/.*/yarn/
RULE:[2:$1@$0](yarn@SUPPORT.COM)s/.*/yarn/
DEFAULT</value>
</property>
</configuration> Any help highly appreciated!
... View more
Labels:
- Labels:
-
Apache Hadoop