I am running an HDP cluster without Knox, Ranger, or Keberos enabled. All Ambari users need to log in with a password and each node has local user accounts with a password. Question #1: what level of security does this cluster have, - no security or basic security? Question #2: Because there is no centralized security management, how does HDP manage the HDFS permissions?  The user/group accounts used in HDFS permissions are actually local users and authenticated by the local Linux OS, right? Question #3: I used the command below to change the owner of an HDFS folder to the ambari user admin. I understand that hadoop is a group name, but it is not an ambari group. So how does this work - a combination of ambari user account and an linux group?   hdfs dfs -chown admin:hadoop /user/admin     ... View more

I create two tables with the same name, one in Hive and the other in HBase. In Atlas, I found the hive table using the search: Search by Type: hive_table Search by Text: the table name However, I could not find the hbase table using the same search criteria Search by Type: hbase_table Search by Text: the table name If I just search by type hbase_table, the hbase table is in the result. What do I miss? ... View more

I am using HDP 3.1. I deleted a directory of an Hbase table by mistake. The deleted hdfs directory is /apps/hbase/data/data/testing/testtable1. The table has 12 regions. After the deletion, I rebooted hbase and the Ambari Hbase UI shows 12 regions in transition.  In Hbase shell, testing:testtable1 does not exist anymore. I tried to recreate it but Hbase shell just hanged, most likely due to those regions in transition. I ran the hbck -details and got the following report         sudo -u hbase hbase hbck -details Number of regions in transition: 12 testing:testtable1,,1566972509039.ab8696ff1a7dc14cd90d138844a71bf6. state=OPENING, ts=Wed Aug 28 08:49:37 PDT 2019 (PT36M51.518S ago), server=null testing:testtable1,1242054,1566972509039.8b11666a10241ec267692e9f09a85c3e. state=OPENING, ts=Wed Aug 28 08:49:37 PDT 2019 (PT36M51.524S ago), server=null           I tried the hbck repair option but got the warning that the repair option has been deprecated.   Options available for medadata repairs are: Metadata Repair options supported as of version 2.0: (expert features, use with caution!) -fixVersionFile Try to fix missing hbase.version file in hdfs. -fixReferenceFiles Try to offline lingering reference store files -fixHFileLinks Try to offline lingering HFileLinks -noHdfsChecking Don't load/check region info from HDFS. Assumes hbase:meta region info is good. Won't check/fix any HDFS issue, e.g. hole, orphan, or overlap -ignorePreCheckPermission ignore filesystem permission pre-check   So which option should I use?   ... View more