1.Prerequisites Enable HDFS location access for the HIve table Enable HDFS Policy in Ranger Restrict POSIX access in HDFS File system Grant Hive table access in Ranger 2. Enable HDFS location access for the user Login to Ranger and select HDFS Policy Make sure to mention the Hive table location (In below example “Resource Path” points to default Hive Warehouse location) Validate the HDFS location access is restricted to ensure Ranger policy is working as expected 3. Create table policy in Hive Access if not exists Select the Hive Policy Add a new policy if not exists Grant table access to required users and validate the access by querying the table 4. Enable table policy in Hive Column Masking Select “Masking” under Hive policy Select “Add New Policy” Provide the required information in the Policy Details. Select the Masking option as per the requirement Below example restricts access for “Sales1” user on column “ip” by masking it using Hash option Before applying Masking on column “ip” in table omniture   Same query triggered by user Sales1 after applying Masking on column “ip” in table Omniture using Ranger for user Sales1 ... View more

1. Prerequisites Install Ranger Install Knox Test in lower environments Inform stakeholders or plan for short outage as few services requires a restart Identify the IP addresses and users to allow access 2. Enable Knox-Ranger Plugin in Ambari Login to Ambari > select Ranger > configs > Ranger Plugin Enable Knox Plugin by clicking the “off” button, and restart the required services Ambari suggests 3. Create a Knox Policy for Hive After enabling the Knox Plugin for Ranger in Ambari, Knox policy should be automatically displayed in Ranger. Select the Knox Policy Add a new policy, if not exists Grant groups/users access to required IP address 4. Connections Test Below are the IPs I considered for testing 172.25.39.156 172.25.40.41 Test Cases: Allow connections through any IPs for the group “sales” Allow only connection access through IP range for group “sales” Case 1: Allow connections through Knox → Hive for any IPs for the group “sales” Test connection from both the IPs for user Sales1 in group “Sales” and connections are successful Case 2: Allow connections through Knox → Hive only for IPs range 172.25.40.*for the group “sales” Note: * works as a wild card in Ranger Few points to consider: Specific IPs can be mentioned Use wildcard (*) for ranges Users part of the group, but not connecting from that IP range mentioned will face Authorization error Users outside of the group, but connecting from that IP range mentioned will face Authorization error 5. Access Granularity suggestions Hortonworks recommends applying group-level restrictions instead of individual users For table-level restrictions, please follow the instructions in the link We can customize Ranger Polices with Dynamic Context; the following article explains in detail the steps: Customizing Ranger Policies with Dynamic Context ... View more

Perform below steps on the servers chosen for Datanodes having different OS version 1. Prerequisites Minimum Software Requirements: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_support-matrices/content/ch_matrices-ambari.html Check and set the maximum Open File descriptors on new host https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_support-matrices/content/ch_matrices-ambari.html Check DNS and NSCD https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-installation/content/check_dns.html Disable THP Disable SELinux Enable NTP Swappiness Install same Python version Install same JDK Version   2. Manage Stack Version Go to Ambari Versions page: http://<ambariHost>:<ambariPort>/views/ADMIN_VIEW/<ambariVersion>/INSTANCE/#/stackVersions Replace <ambariHost>, <ambariPort>,<ambariVersion> with values from your cluster. Select current version. Add repository details for RHEL7   3.Manually download and Install Ambari agent 3.1 Download Ambari repo wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/<version>/ambari.repo -O /etc/yum.repos.d/ambari.repo Below command downloads the latest Ambari version 2.6.1.5 wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.5/ambari.repo -O /etc/yum.repos.d/ambari.repo 3.2 Install Ambari Agent   4. Manually register the hosts in Ambari   5. Validate the OS Family for stack and versions in Ambari   6. Complete the installation and validate the host in the hosts list 7.Test with a Job 7.1 Testing with a Terasort job /usr/hdp/current/hadoop-client/bin/hadoop \ jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-*.jar \ terasort /tmp/teragenout /tmp/terasortout 7.2 Validate by checking with a Terasort job When you run a new job, the recently added Datanode will have most resources available and some containers from your new job will land on this new host. From Yarn RM UI, validate if any container landed on this new host and confirm that they finish their execution successfully. ... View more