Hello everybody, For ex. I have 1 TB of HDFS disk usage . I want to restrict the access to all the other disk drives /data/ /apps/ /user/ /tmp/ /path1 /path2 In the above directories , I want my development team to use only 30 GB in /data where they can create/delete/modify the folders . If they want to use any other folder than /data . they should ask the operation team for permission to access the folders if not they should get permission denied error message.. I can exclude /tmp/,/user with 10Gb and 5GB respectiverly . What tools are good for this and how can we achieve this scenario. Sorry for the basic question . @Geoffrey Shelton Okot Sir you could also help me in this regard. @Jay Kumar SenSharma ... View more

Hello guys, What are the things we can get to do to maximize the security via Apache ranger. HDFS,HIVE,Nifi,HBASE & YARN.. Could anyone helpme in this with any supporting documents or step by step approach . I'm very new to this security area. Also need to integrate Ranger with LDAP . ... View more

I want to setup Ranger with Policies on HDFS,Hive,HBase on HDP 2.6.3. i have never done this before. Can anyone share the step by step commands/procedure on how to go about it? i am totally new to Ranger. i am using Centos. ... View more

Hello everybody, My disk space is consuming more and more in HDFS and when I dig , I can see HBASE took 400GB of Old Wals .However If I look at Wals . it is almost empty . So Can I remove the complete OLDWals from the machine . Does it affect any data. I have read WALs are used only when there is crash in memory before writing it to disk . Still do not want to take the risk . $ hdfs dfs -du -h /apps/hbase/data 0/apps/hbase/data/.tmp ©37 0/apps/hbase/data/MasterProcWALs 12.6 K /apps/hbase/data/WALs ---- ONLY KBs 7.5 G /apps/hbase/data/archive 0/apps/hbase/data/corrupt 52.9 T /apps/hbase/data/data 42/apps/hbase/data/hbase.id 7/apps/hbase/data/hbase.version 379.5 G /apps/hbase/data/oldWALs -- Large data My Question is if I set any parameter in Ambari to auto purge on regular interval and how can I check the replication is enabled or not . hbase.master.logcleaner.ttl where to set if in case i do not want to delete it . ... View more

Hi All, Nifi is writing the data into PUTHDFS as JSON files. The file permissions and owner of the files are as follows File_name1 rwx-- nifi:hadoop There are two things which we don't understand . We do not have Nifi LDAP integrated so far . it runs on default user Nifi. I cannot ask nifi to change the permission while writing the data or change the owner which is accessible to hive . We are getting permission denied errors while reading the data from hive . We cannot change the permissions in nifi as it is not added in superusers group . We did tried to add nifi as superusers group . but failed to change the owner information . How can I write the files which is accessible for both hive and Nifi or how can I change the owner/permissions in nifi after writing the data into HDFS . ... View more

Hello everybody, I want to use beeline in Shell script and it not working properly . can someone help me to resolve this . main_script.sh source /file_path/configs.sh -- with 777 permission beeline -u ${hive_connection_string} -n $USER_NAME -p $PASSWORD -f /usr/hql/query.hql configs.sh current_dir="/data/landing/" hive_connection_string='jdbc:hive2://host1:2181,host2:2181,host3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;#currentdir='${current_dir} query.hql create external table ( ....) location /user/hdfs_path/landingdir/ --> this directory is also not picking in the main script . What is the main thing am I missing or how can I achieve this . When we try to run the main_script.sh the values -n $USER_NAME/password is not picking in the main script . What am I doing wrong . I think it is taking me to beeline session . so the shell is not taking the right . Have tried this step alone for many hours , tried all the possibilites of single quotes , double quotes , flower braces without braces. It didn't seem to work . what is right way to approach this ... View more

Hello All, Can someone help me to steps/links for KAFKA TWO WAY SSL . My environment is not kerberized . Just wanted to enable two way SSL I have already referred google top links https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_security/content/ch_wire-kafka.html . didnt help me . Can someone kindly send me the steps to do it please . ... View more