Short Description: How to configure KNOX for Hive1 and Hive2(LLAP) in parallel. Article By default KNOX is configured for Hive1, this article will help you to configure KNOX for Hive2(LLAP). Step 1. Before configuring KNOX for Hive2, you have to configure Hive2 for http mode as, Go to Ambari-> Services -> Hive -> configs -> custom hive-interactive-site.xml -> add below properties, hive.server2.thrift.http.path=cliservice hive.server2.transport.mode=http Restart Hive service. Step 2. Now configure KNOX for Hive2(LLAP) as, 1) Go to the below location in your KNOX server machine:- # cd /usr/hdp/<HDP VERSION>/knox/data/services 2) Copy the hive directory present in the location and rename it as llap # cp -rp hive llap 3) Edit the services.xml and rewrite.xml as below:- # cd llap/0.13.0/ # vim service.xml ------------ <service role="LLAP" name="llap" version="0.13.0"> <routes> <route path="/llap"/> </routes> <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/> </service> # vim rewrite.xml ------------ <rules> <rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap"> <rewrite template="{$serviceUrl[LLAP]}"/> </rule> </rules> 4) Go to Ambari -> KNOX -> configs -> Edit the Advanced topology of your KNOX service and add LLAP service as, <service> <role>LLAP</role> <url>http://<LLAP server hostname>:<HTTP PORT NUMBER>/{{hive_http_path}}</url> </service> Example: <url>http://abcd.example.com:10501/cliservice</url> 5) Restart Knox service. ... View more

ISSUE:- /var/log/ambari-server/ambari-server.log 18 May 2017 07:56:33,754 WARN [ambari-client-thread-26] PermissionHelper:78 - Error occurred when cluster or view is searched based on resource id java.lang.NullPointerException at org.apache.ambari.server.security.authorization.PermissionHelper.getPermissionLabels(PermissionHelper.java:74) 18 May 2017 07:56:33,975 ERROR [ambari-client-thread-26] ContainerResponse:419 - The RuntimeException could not be mapped to a response, re-throwing to the HTTP container com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key 58. at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280) 18 May 2017 07:56:33,977 WARN [ambari-client-thread-26] ServletHandler:561 - Error Processing URI: /api/v1/users/admin - (com.google.common.cache.CacheLoader$InvalidCacheLoadException) CacheLoader returned null for key 58. 18 May 2017 07:56:33,977 WARN [ambari-client-thread-26] ServletHandler:561 - Error Processing URI: /api/v1/users/admin - (com.google.common.cache.CacheLoader$InvalidCacheLoadException) CacheLoader returned null for key 58. ROOT CAUSE:- Zeppelin View is removed from ambari 2.5 but looks like a reference stays there. SOLUTION:- Check this by running the following query on ambari DB. # psql -U ambari ambari Password for user ambari: bigdata psql => command -U ambari => is the username ambari => is the DB name It will ask for password, Where we can try using the ambari's default password as "bigdata" If the password is changed then we can check the file. # grep 'server.jdbc.user.passwd' /etc/ambari-server/conf/ambari.properties server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat# cat /etc/ambari-server/conf/password.datbigdata So please get the "resource_id" of Zeppelin View if present?(In my case it is 58) # select * from adminresourcetype; # select * from adminresourcetype; where resource_type_id IN (select resource_type_id from adminresource where resource_id =58); If that is the case then please Take Ambari DB Dump and then run the following commands to clean the Zeppelin View reference: # psql -U ambari ambari Password for user ambari: bigdata # DELETE FROM adminprivilege where resource_id in(58); # DELETE FROM adminresource where resource_id in(58); Then restart Ambari Server # ambari-server restart ... View more

Change Log directory for Ranger Admin logs On Ranger Admin nodes: ADMIN LOGS (Change the symbolic link for in ews directory): mkdir /opt/ranger/admin chmod -R 775/opt/ranger/admin chown -R ranger:ranger /opt/ranger/admin cd /usr/hdp/current/ranger-admin/ews unlink log ln -s /opt/ranger/admin logs Change log directory for USERSYNC LOGS On Ranger Usersync node: mkdir /opt/ranger/usersync chmod -R 775/opt/ranger/usersync chown -R ranger:ranger /opt/ranger/usersync cd /usr/hdp/current/ranger-usersync/ unlink logs ln -s /opt/ranger/usersync logs cd /usr/hdp/current/ranger-usersync/ cp ranger-usersync-services.sh{,.backup01122015} vim ranger-usersync-services.sh #logdir=/var/log/ranger/usersync (If it is present) logdir=/opt/ranger/usersync Change log directory for TAGSYNC LOGS: On Ranger Tagsync nodes: mkdir /opt/ranger/tagsync chmod -R 775/opt/ranger/tagsync chown -R ranger:ranger /opt/ranger/tagsync cd /usr/hdp/current/ranger-tagsync/ unlink logs ln -s /opt/ranger/tagsync logs cp ranger-tagsync-services.sh{,.backup01122015} vim ranger-tagsync-services.sh #logdir=/var/log/ranger/tagsync logdir=/opt/ranger/tagsync From Ambari Web UI --> Change Log directory for Ranger Usersync, Ranger Admin and Ranger Tagsync, Restart Ranger Service. Note: If you are not able to change the directories in Ambari UI use below mathod for this, Go to Ranger Service -> Configs -> Advanced -> Advanced ”Advanced ranger-env”. Right click on the value of property “ranger_admin_log_dir” -> Inspect element -> Change the value in element section(in my case value=”/opt/ranger/admin”)-> Hit Enter. 3. In the same tab Right click on the value of property “ranger_usersync_log_dir” -> Inspect element -> Change the value in element section(in my case value=”/opt/ranger/usersync”)-> Hit Enter. 4. Go to ”Advanced ranger-tagsync-site” -> Right click on the value of property “ranger_tagsync_logdir” -> Inspect element -> Change the value in element section(in my case value=”/opt/ranger/tagsync”)-> Hit Enter. 5. Click on save changes and restart the Ranger Whole service and check at the backend in those directories(In my case it is in /opt/ranger/ directory). ... View more