Member since
04-03-2019
89
Posts
5
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3313 | 01-21-2022 04:31 PM | |
5756 | 02-25-2020 10:02 AM | |
3420 | 02-19-2020 01:29 PM | |
2522 | 09-17-2019 06:33 AM | |
5504 | 08-26-2019 01:35 PM |
01-31-2020
08:22 AM
I plan to get a Cloudera certification and need help on following questions: Question #1. I reviewed the page https://www.cloudera.com/about/training/certification.html, It looks like that CCP Data Engineer is the only certification that has not been suspended or retired. Am I right on this? Question #2. To prepare DE575, the only recommended Cloudera course is "Spark and Hadoop Developer" training course. according to this page. https://www.cloudera.com/about/training/certification/ccp-data-engineer.html. Should I consider other courses? Questions #3. My workplace uses HDP. Do I need to get familiar with products like CDH before taking the exam?
... View more
Labels:
- Labels:
-
Certification
01-15-2020
10:02 AM
@Shelton @EricL Thank you both. the correct ACL spec is group::r-x Now the following command works. sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1 From what I just ran into, I feel that, by design, Hive takes extra effort to prevent users from accessing managed table files directly. I will follow that design and access Hive managed table only through Hive.
... View more
01-14-2020
05:09 PM
I tried the following command # sudo -u hdfs hadoop fs -setfacl -m g::rx /warehouse/tablespace/managed/hive/test1 But I got the error -setfacl: Invalid type of acl in <aclSpec> :g::rx The acl spec is to modify the owning group permission to rx. Any suggestion?
... View more
01-14-2020
09:49 AM
I might have found the reason. I ran the following command as hdfs, which is the superuser of hdfs. $ hadoop fs -getfacl /warehouse/tablespace/managed/hive/test1 # file: /warehouse/tablespace/managed/hive/test1 # owner: hive # group: hadoop user::rwx user:hive:rwx group::--- mask::rwx other::--- default:user::rwx default:user:hive:rwx default:group::--- default:mask::rwx default:other::--- The output, as I understand, shows that the group owner has no permission on the folder. My guess is that, HDP Hive uses ACL to limit direct access to files behind managed tables. HDP Hive tries to force accessing to managed tables only through Hive.
... View more
01-14-2020
08:05 AM
# hdfs groups zeppelin zeppelin : hadoop zeppelin On the name node, # id zeppelin uid=1018(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)
... View more
01-13-2020
09:05 PM
I am using HDP. The inode in the following code is a managed hive table. # id zeppelin uid=1017(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin) # sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1 ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive/test1":hive:hadoop:drwxrwx--- The user zeppelin is in hadoop group, which has full permisison on the hdfs folder. So why do I get the permission error?
... View more
Labels:
11-20-2019
09:58 AM
@ManuelCalvo Thanks. Your solution works. My environment does not have Kerberos and Ranger enabled. So, I skipped step 2 For step 4, hbase.coprocessor.master.classes = org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor Also, as I found out, Atlas currently only stores HBase metadata and does not store HBase lineage data as it does for Hive.
... View more
11-06-2019
08:25 AM
1 Kudo
@sundar_gampa Try a different browser or a bigger monitor. Apparently, the page design makes the page adaptive to the viewing device.
... View more
09-17-2019
08:39 AM
I am reading "Enable the Atlas Book & Bridge for Apache HBase" https://atlas.apache.org/Hook-HBase.html, and get stuck on the step below Link Atlas hook jars in HBase classpath - 'ln -s <atlas package>/hook/hbase/* <hbase-home>/lib/' Where is the HBase classpath in HDP? I checked the file hbase_site.xml but still got lost. From the file, # Extra Java CLASSPATH elements. Optional.
export HBASE_CLASSPATH=${HBASE_CLASSPATH} Is the classpath env variable specified somewhere else?
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache HBase
09-17-2019
08:22 AM
Finally, I am able to reproduce the problem. It is caused by the monitor display. The problem only occurs on my laptop, which has less screen real estate.
... View more