Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2720 | 04-27-2020 03:48 AM | |
| 5279 | 04-26-2020 06:18 PM | |
| 4445 | 04-26-2020 06:05 PM | |
| 3567 | 04-13-2020 08:53 PM | |
| 5377 | 03-31-2020 02:10 AM |
06-27-2017
05:35 AM
@ed day The HDFS root directory allows only the hdfs:hdfs (username:groupname) user to write on "/" Please check the permission to know who can write in the ROOT (/) directory of hdfs file system. # su - hdfs -c "hdfs dfs -ls -d /"
drwxr-xr-x - hdfs hdfs 0 2017-06-21 09:59 / . In order to create a directory "Write" permission is needed to the user. So if you want to create the directory inside "/" then you should create the directory as hdfs user and then you can use chown and chmod commands properly to give read/write/execute access on this directory to your desired users. .
... View more
06-26-2017
02:27 PM
@ed day As we see that the write access is there only for the "hdfs" user on the directory "/ml-in-a-nutshell" So if you want to write in that directory as "ed" user then the "ed" user should have write permission on that mentioned HDFS directory. Example: So try changing the permission as followong:
# su - hdfs -c "hdfs dfs -chown ed:hadoop /ml-in-a-nutshell . Or else you will need to use the "hdfs dfs -chmod" command to allow write access to the "ed" user on the mentioned directory. . Similarly if you want the "admin" user to be able to access to write in that directory then you will need to change th permissions of that HDFS directory as mentioned above. .
... View more
06-26-2017
01:25 PM
@Arjun kumar Which version of ambari are you using?
There are some issues reported to address similar behavior:
1. Optionally force username from LDAP authentication data to be lowercase in Ambari
https://issues.apache.org/jira/browse/AMBARI-18813
According to this fix ambari should force username from LDAP authentication data to be lowercase in Ambari based on LDAP import configuration. Please see the discussion about adding "authentication.ldap.username.forceLowercase" flag in ambari.properties in the above JIRA.
2. Ambari usernames should not be converted to lowercase before storing in the DB.
https://issues.apache.org/jira/browse/AMBARI-18520 .
... View more
06-26-2017
04:02 AM
@white wartih When you run the ambari-server setup command (Without using -j /path/to/your/installed/jdk) option then ambari provides you two options to choose your JDK JDK1.8 or JDK1.7 as following # ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)?
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1):
. When you choose any of those options then ambari will download the JDK from the following locations. # grep "jdk1.7.url" /etc/ambari-server/conf/ambari.properties
jdk1.7.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz
# grep "jdk1.8.url" /etc/ambari-server/conf/ambari.properties
jdk1.8.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz
. On the ambari server the desired JDK TAR ball is placed inside the following location: # ls -l /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz
-rw-r--r--. 1 root root 181238643 Jun 22 16:26 /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz . And the same JDK is pushed to all the ambari agent machines in the following location. # ls -l /var/lib/ambari-agent/tmp/jdk-8u112-linux-x64.tar.gz
-rwxr-xr-x. 1 root root 183212596 Jun 9 11:12 /var/lib/ambari-agent/tmp/jdk-8u112-linux-x64.tar.gz . This is how the agents gets the JDK binary from ambari server. . So if you want to choose your own JDK then as per the Current Design and Implementation of Ambari you will need to Make sure that the JDK is manually installed on all the host machine by you and on the asme path that you mentioned in the "ambari-server setup -j /path/to/your/installed/jdk" command. And this is what Ambari Server Documentation says: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.0/bk_ambari-installation/content/setup_options.html .
... View more
06-25-2017
06:19 PM
@Mahendra Malpute'
In your command the path is missing. Your command: # su - hdfs -c "hdfs dfs -ls" . Correct command: # su - hdfs -c "hdfs dfs -ls /user/maria_dev" .
... View more
06-25-2017
11:35 AM
@Mahendra Malpute
Are you doing ssh to the sandbox on the correct port 2222 ? # ssh root@localhost -p 2222 When you see those HDFS files in ambari UI, (I guess you mean FileView). Which user is logged in to Ambari FileView? ("admin" user or "maria_dev") user? .
Can you please list the permission of directory: Can you please share the output of the following directory?
# su - hdfs -c "hdfs dfs -ls /user"
# su - hdfs -c "hdfs dfs -ls /user/maria_dev"
. As you mentioned that "i can not able see those file SSH console like /user/maria_dev/ ", Do you see any error/exception or no output? Can you please share the screenshot of the Ambari UI where you see the contents (so that we can check the logged in user and the output of the directory). Also the terminal output of the hdfs command shell.
.
... View more
06-24-2017
08:35 AM
@white wartih When you use the following option to choose your own desired JDK to run your cluster, then you will need to make sure that the mentioned JDK is alredy installed on all the host of the cluster and also it should be present on exactly the same path "/path/to/your/installed/jdk" # ambari-server setup -j /path/to/your/installed/jdk . Please refer to the following note from the doc snippet: This option specifies the JAVA_HOME path to use on the Ambari Server and all hosts in the cluster. By default when you do not specify this option, Ambari Server setup downloads the Oracle JDK 1.8 binary and accompanying Java Cryptography Extension (JCE) Policy Files to /var/lib/ambari-server/resources. Ambari Server then installs the JDK to /usr/jdk64. Use this option when you plan to use a JDK other than the default Oracle JDK 1.8. See JDK Requirements for more information on the supported JDKs. If you are using an alternate JDK, you must manually install the JDK on all hosts and specify the Java Home path during Ambari Server setup. If you plan to use Kerberos, you must also install the JCE on all hosts. This path must be valid on all hosts. For example:
ambari-server setup –j /usr/java/default . Reference: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.0.0/bk_ambari-installation/content/setup_options.html
... View more
06-23-2017
12:11 PM
1 Kudo
@Zack M There looks like a type mistake in your PATH Please check "haddop" .. it should be "hadoop" (The Jar name is not correct).
Your PATH:
/usr/hdp/current/haddop-mapreduce-client/hadoop-streaming.jar
The path should be /usr/hdp/current/hadoop-mapreduce-client/hadoop-streaming.jar .
... View more
06-23-2017
08:01 AM
@qiancai zhou Please refer to the following piece of code to understand the logic: https://github.com/apache/ambari/blob/release-2.5.1/ambari-agent/src/main/python/ambari_agent/RecoveryManager.py#L226-L257 def requires_recovery(self, component):
"""
Recovery is allowed for:
INISTALLED --> STARTED
INIT --> INSTALLED --> STARTED
RE-INSTALLED (if configs do not match)
"""
if not self.enabled():
return False
if not self.configured_for_recovery(component):
return False
if component not in self.statuses:
return False
status = self.statuses[component]
if self.auto_start_only or self.auto_install_start:
if status["current"] == status["desired"]:
return False
if status["desired"] not in self.allowed_desired_states:
return False
else:
if status["current"] == status["desired"] and status['stale_config'] == False:
return False
if status["desired"] not in self.allowed_desired_states or status["current"] not in self.allowed_current_states:
return False
logger.info("%s needs recovery, desired = %s, and current = %s.", component, status["desired"], status["current"])
return True
pass .
... View more
06-23-2017
07:45 AM
@qiancai zhou Its a crash recovery feature. So if you gracefully stop the zookeeper process ambari should not attempt to restart it. Try this to test this feature. 1. Start the Zookeeper Server Process. 2. Now suddenly kill the Zookeeper Server ( this is just to simulate an abroup zookeeper termination, in real time process can get crash due to various other reasons as well). # kill -9 `cat /var/run/zookeeper/zookeeper_server.pid` 3. Now check the ambari agent logs. you will notice that within 2-3 minutes your Zookeeper process will be restarted.
... View more