yes you would need to configure user sync with ldap/ad in the ranger ui. Alternatively use UNIX user sync in Ranger to sync with the local operating system. ( Works as well ) ... View more

I ssume also tha Hive with PAM authentication will also be a viable option on Azure. https://community.hortonworks.com/articles/591/using-hive-with-pam-authentication.html ... View more

@Joseph Niemiec How can I do this command " select * from table where date <= '2017-12-08' " in nest partitions form? In case the table is partitioned by day,month,year ... View more

I have tried with the following parms hive.tez.auto.reducer.parallelism=true; hive.tez.min.partition.factor=0.25 hive.tez.max.partition.factor=2.0 set hive.exec.reducers.bytes.per.reducer = 134217728; My output is of size 2.5 GB (2684354560 bytes) and based on the formula given above, i was expecting max(1,min(1099,2684354560/ 134217728))*2 = max(1,min(1099,20))*2 = max(1,20)*2 = 40 reducers. but my query was assigned only 5 reducers, i was curious why? and are there any other parameters that can reflect the no. of reducers. below is the query that i am using : truncate table target_tab ; INSERT INTO TABLE target_tab SELECT * FROM src_tab WHERE 1=1 ORDER BY a, b,c ... View more

Do you have latest recommendations? Most of our hadoop processing is on Hive/Tez and Spark. ... View more

Hello @Ancil McBarnett , I checked that ranger-admin-site.xml, it was not updated there. I was wondering, since I am using HDP2.2.4, why is that file under .../HDP/2.3/... ?!?! Anyways, I set the port to 6182, but unfortunately after restarting Ranger the alert was still there. So I searched for other RANGER configs, because of the HDP2.2.4 vs. HDP2.3 and at the end it turned out that I had to update the port in file /var/lib/ambari-server/resources/common-services/RANGER/0.4.0/configuration/ranger-site.xml After restarting Ranger the alert disappeared. Many thanks for this hint... ... View more

Hello Satish, this error is not related to the procedure you followed, you need to kinit as kafka's user. ... View more

How do you do this? I have no idea how to delete the entry in the ambari database ... View more

@Robin Dong As mentioned by Ancil, you might want to have a script to do the sqoop download in parallel. And you need to control quite well how big is your parallelism. Above all if you want to avoid the typical "No more spool space in...". Here's a script to do that: https://community.hortonworks.com/articles/23602/sqoop-fetching-lot-of-tables-in-parallel.html Another problem I saw in Teradata, is that it is some data types are not supported when you try to directly insert the data into Hive from Sqoop. So the solution I took was the traditional one: 1) Sqoop to HDFS. 2) Build external tables on top of them 3) Create ORC file and then insert the data or the external tables ... View more

Hi, I have hive.server2.enable.doAs set to true. I have created a hive policy for qaadmin user to have access to only initcap() UDF which is builtin Hive UDF. But qaadmin user can access other built in UDFs too which user should not have access as per Hive Policy. Note- qaadmin is only part of this policy. Please help me how to make it work ? ... View more