Member since
06-26-2013
416
Posts
104
Kudos Received
49
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 15676 | 10-12-2015 01:56 PM | |
| 6274 | 03-27-2014 07:24 AM | |
| 24852 | 10-30-2013 08:25 AM |
09-27-2018
06:55 AM
@ramarov Thank you for the suggestion! We'll keep it in mind for our future sprint updates.
... View more
05-12-2016
10:46 AM
1 Kudo
If you happen to be in the Austin area, you should consider participating in our Hackathon this coming Sunday May 15. Hosted at the Cloudera office and sponsored by Cloudera Cares.
The hackathon will focus on reducing mosquito borne virus infections by analyzing data on water, mapping mosquito travel, and historical virus analysis.
515 Congress Ave., Suite 1212, Austin, TX
10:30am to 8:00pm
RSVP and further details here.
... View more
01-12-2016
09:42 AM
3 Kudos
Symptoms
"Permission denied" errors can present in a variety of use cases and from nearly any application that utilizes CDH.
For example, when attempting to start the jobtracker using this command
service hadoop-0.20-mapreduce-jobtracker start
You may see this error, or one similar
org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:149)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4891)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4873)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4847)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3192)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3156)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3137)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:669)
While the steps to reproduce this error can vary widely, the root causes are very well defined and you'll know you're suffering from this issue by finding the following line either on stdout or in the relevant log files:
org.apache.hadoop.security.AccessControlException: Permission denied: user=XXX, access=WRITE, inode="/someDirectory":hdfs:supergroup:drwxr-xr-x
Applies To
CDH (all versions), Mapreduce, HDFS, other services that rely on reading from or writing to HDFS
Cause
Access to the HDFS filesystem and/or permissions on certain directories are not correctly configured.
Troubleshooting Steps
There are several solutions to attempt:
1) The /user/ directory is owned by "hdfs" with 755 permissions. As a result only hdfs can write to that directory. Unlike unix/linux, hdfs is the superuser and not root. So you would need to do this: sudo -u hdfs hadoop fs -mkdir /user/,,myfile,, sudo -u hdfs hadoop fs -put myfile.txt /user/,,/,, If you want to create a home directory for root so you can store files in his directory, do: sudo -u hdfs hadoop fs -mkdir /user/root sudo -u hdfs hadoop fs -chown root /user/root Then as root you can do "hadoop fs -put file /user/root/". 2) You may also be getting denied on the network port where the NameNode is supposed to be listening:
Fix this by changing the address that the service is listening on in /etc/hadoop/conf/core-site.xml. By default your NameNode may be listening on "localhost:8020." (127.0.0.1)
So to be clear, implement this value for the following property:
<property> <name>fs.defaultFS</name> <value>hdfs://0.0.0.0:8020</value> </property>
then bounce the service with hadoop-hdfs-namenode restart optional: validate with netstat -tupln | grep '8020'
References
... View more
08-07-2015
01:37 PM
Here at Cloudera, community is part of our DNA. Building a strong, vibrant, and engaged community between our employees, customers, partners, open source developers, users, admins and industry experts is critical to our success. That's where YOU come in. We want you to find value, build relationships, gain skills, and advance your projects and careers thanks to the community you are part of here.
This discussion board is an effort to further advance those goals. We want to get to know the highly talented and unique individuals who make up our community. This board provides a place for you introduce yourself, talk about your background, goals, interesting projects, and what makes you tick. It is also a place for off-topic discussion to occur. Enjoy a welcome diversion, talk about your hobbies, have fun.
PLEASE FOLLOW THESE GUIDELINES:
1) Keep it clean and professional, no offensive or divisive topics (including, but not limited to: sex, illegal activity, politics, or religion)
2) Keep it friendly and courteous, no flaming.
3) Our regular Terms of Service and User Guidelines still apply and we reserve the right to remove any content.
... View more
03-26-2015
01:14 PM
Sorry for the delayed response here, but this is fixed now. Thanks again for reporting the issue.
... View more
02-26-2015
01:14 PM
@javang thank you for closing the loop with us and posting your findings!
... View more
01-06-2015
11:56 PM
I looked at the URL which you gave me and its still very thin for Service providers. it just says Service providers deliver Apache Hadoop-based solutions and services to end users interested in leveraging Cloudera’s training resources, support and brand. * How is the support offered to a partner any different from normal cludera support which I get if I purchase Data Hub Edition? * Are there regular training events? or just 1 training which you give at the time of signing up the partner agreement? If there are perioidic trainings then is there a schedule for these partner trainings? * Can you eloborate on what "brand" means above?
... View more
12-03-2014
09:43 AM
I do apologize for the lack of responses to some of your queries, we do try to assist community members in finding solutions to their questions, but unfortunately there are no guarantees in the forums, as responses are purely voluntary. There are no service level agreements in the community, that is only available for paid support customers and through our actual support portal. We will continue to assist to the best of our ability, but please understand that you may not always get immediate responses, or a solution to a particular thread.
... View more
11-24-2014
12:37 PM
Hi Yuxi, Can you please send me a note to dmichaels@cloudera.com? I just need to check on a few things before I can grant you access. Please let me know if you have any other questons. Thanks, Devon Michaels dmichaels@coudera.com Training Coordinator
... View more