Member since
01-20-2014
578
Posts
102
Kudos Received
94
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 6695 | 10-28-2015 10:28 PM | |
| 3573 | 10-10-2015 08:30 PM | |
| 5649 | 10-10-2015 08:02 PM | |
| 4106 | 10-07-2015 02:38 PM | |
| 2891 | 10-06-2015 01:24 AM |
05-13-2015
05:09 AM
Thanks for sharing your solution!
... View more
05-12-2015
11:19 PM
1 Kudo
The list of users who are allowed to run this command are dictated by the list in the property dfs.cluster.administrators. In clusters managed by Cloudera Manager 4.x, the property is not set by default. In clusters managed by Cloudera Manager 5.x, the property is set to "hdfs" by default. So if this command is run as the hdfs user on any node on the cluster, the command will succeed. You can try setting dfs.cluster.administrators to the list of users and groups who are allowed to perform this operation and then try again. It will require a restart of the Namenode(s) to take effect. <property> <name>dfs.cluster.administrators</name> <value>user1,user2,user3 group1,group2,group3</value> </property>
... View more
05-12-2015
07:57 PM
The link to the source is here https://github.com/cloudera/hadoop-common/blob/cdh5.4.0-release/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/FsUsage.java#L120
... View more
05-12-2015
07:55 PM
I checked on CDH 5.2 and it was the same as CDH4. The output you see was introduced in CDH 5.3, the jira being HADOOP-6857. The second column is supposed to calculate the raw disk usage of the file/directory. So if a file is 42 bytes, it correctly shows (file size) * (replication factor of file) $ hadoop fs -du /hbase/hbase.id 42 126 /hbase/hbase.id However it seems incorrect for directories as column 2 shows (theoretical block size) * (replication factor of file) * (number of files in directory) $ hadoop fs -du -h /hbase/WALs 0 0 /hbase/WALs/hregion-67213513 83 384 M /hbase/WALs/server54-2.abc.cloudera.com,22101,1431434548533 166 768 M /hbase/WALs/server54-3.abc.cloudera.com,22101,1431434548804 83 384 M /hbase/WALs/server54-4.abc.cloudera.com,22101,1431434547067 $ hadoop fs -ls -R /hbase/WALs/server54-2.abc.cloudera.com,22101,1431434548533 -rw-r--r-- 3 hbase hbase 83 2015-05-12 18:42 /hbase/WALs/server54-2.abc.cloudera.com,22101,1431434548533/server54-2.abc.cloudera.com%2C22101%2C1431434548533.default.1431481363257 $ hadoop fs -ls -R /hbase/WALs/server54-3.abc.cloudera.com,22101,1431434548804 -rw-r--r-- 3 hbase hbase 83 2015-05-12 19:42 /hbase/WALs/server54-3.abc.cloudera.com,22101,1431434548804/server54-3.abc.cloudera.com%2C22101%2C1431434548804..meta.1431484974652.meta -rw-r--r-- 3 hbase hbase 83 2015-05-12 19:42 /hbase/WALs/server54-3.abc.cloudera.com,22101,1431434548804/server54-3.abc.cloudera.com%2C22101%2C1431434548804.default.1431484963607
... View more
05-12-2015
05:08 PM
Have you checked if the RPC Call Queue length is spiking correlates with the alerts? If it does, maybe you can check the Namenode logs to see why there's a sudden burst of activity once in a while, maybe one of your client applications can be tuned.
... View more
05-10-2015
04:45 PM
My output looks like this on both CDH4 and CDH5. # sudo -u hdfs hdfs dfs -du / 20143 /accumulo 3230 /hbase 0 /solr 4 /tmp 287725904 /user Please paste your command and output here so we can have a look.
... View more
03-23-2015
10:44 PM
Postgres is part of the "base" repo on CentOS. See this URL http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os for a list of CentOS repos where you can see the packages. If you are using Red Hat (not CentOS) and have a Red Hat subscription, you should ask them how to add the "os" repository to your hosts.
... View more
03-16-2015
09:49 PM
Are you facing this error with the Path A (.bin installer) or Path B installation? Is this a new cluster or is this failure when adding a new host to the cluster? Thanks.
... View more
03-15-2015
11:43 PM
Are you able to provide the command you used and the exact error message it printed? This will allow us to help you a bit quicker.
... View more
03-15-2015
09:37 PM
1 Kudo
Please also refer to this URL for a list of all ports used within Cloudera Manager and CDH http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_ports.html
... View more