Member since
10-01-2015
3933
Posts
1150
Kudos Received
374
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3650 | 05-03-2017 05:13 PM | |
| 3007 | 05-02-2017 08:38 AM | |
| 3263 | 05-02-2017 08:13 AM | |
| 3216 | 04-10-2017 10:51 PM | |
| 1680 | 03-28-2017 02:27 AM |
03-09-2016
01:47 AM
4 Kudos
you can skip trash hdfs dfs -rm -r -skipTrash and need not worry about file size limit. http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html
... View more
03-09-2016
01:44 AM
1 Kudo
The model also differentiates between an “access ACL”, which defines the rules to enforce during permission checks, and a “default ACL”, which defines the ACL entries that new child files or sub-directories receive automatically during creation. For example: user::rwx
group::r-x
other::r-x
default:user::rwx
default:user:bruce:rwx #effective:r-x
default:group::r-x
default:group:sales:rwx #effective:r-x
default:mask::r-x
default:other::r-x
Only directories may have a default ACL. When a new file or sub-directory is created, it automatically copies the default ACL of its parent into its own access ACL. A new sub-directory also copies it to its own default ACL. In this way, the default ACL will be copied down through arbitrarily deep levels of the file system tree as new sub-directories get created.
... View more
03-09-2016
01:20 AM
1 Kudo
Please review this document https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
... View more
03-08-2016
10:38 PM
1 Kudo
Please check your datanode logs in the log directory you specified in ambari UI. Please post the logs here.
... View more
03-08-2016
10:36 PM
1 Kudo
Can you look at yarn container level, drill down to each job's task and see the information you get. If this causes you a lot of trouble, consider opening a ticket with support.
... View more
03-08-2016
10:27 PM
this information is available in Ambari UI under Hive service or you could look up /etc/hive/conf/hive-site.xml on edge node and view the file, port info is in the XML.
... View more
03-08-2016
06:18 PM
@Ram D keep your passwords safe
... View more
03-08-2016
03:10 PM
2 Kudos
@azeltov pyspark required numpy in my case, didn't try it in Zepplin but it wouldn't work in pyspark either if you don't have it on your machine https://github.com/dbist/datamunging
... View more
03-08-2016
01:37 PM
1 Kudo
Create mount point /hadoop/ pointing to your large disk
... View more