Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

[#1]

While trying out the sandbox 2.5 TP for some new features in Atlas, I encountered an error when running sqoop cmd to move data from mySQL to Hive (this should trigger a linage generated inside Atlas and that is my goal)

created a simple table inside mysql and then sqoop the data under ambari-qa credential

mysql --host=sandbox.hortonworks.com --user=root --database=hive 
use test; 
create table source_fact (id int, name varchar(100), description varchar(100));
exit;

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --table source_fact --hive-import --hive-overwrite --hive-table atlasdemo.source_fact --username root -m 1

Then got the following error:

FAILED: RuntimeException java.io.FileNotFoundException: /etc/hive/2.5.0.0-817/0/xasecure-audit.xml (No such file or directory)

this only happens when running sqoop. Hive, Atals and Ranger all looks fine and function as expected.

Thanks,

Note: move this to security/Ranger, look like this is related to Ranger config after looking at some old thread. And based on the old thread, xasecure-audit.xml should not be part of the new version anyway. Looking for some Ranger guru to provide some feedbacks.

[#2]

A quick workaround is to disable ranger authorization from hive->authorization in ambari UI and restart hive. However, if ranger is part of your tests and want to keep it enabled, this is not the solution.

[#3]

Ravi, I want to take a look of Atlas/Ranger integration in HDP2.5. So disable Ranger is not really an option.

[#4]

Hi @Qi Wang

Which user is running the sqoop command? Can you verify file /etc/hive/2.5.0.0-817/0/xasecure-audit.xml exists? Does the user running sqoop import has read access to this file? Also, check the following link. It might be your issue.

https://community.hortonworks.com/questions/369/installed-ranger-in-a-cluster-and-running-into-the.h...

[#5]

I tried different users running sqoop cmd and none worked, including root, ambari-qa, sqoop, hive...

the file /etc/hive/2.5.0.0-817/0/xasecure-audit.xml does not exist. And I did a search of all folders and it doesn't exist on the sandbox VM

cd /
find -name xasecure-audit.xml

[#6]

Check the link I just added to my answer.

[#7]

I tried the suggested fix in the old thread and it is not working. Still got the same error with sqoop. Move this to ranger section and hoping some ranger guru jump in.

Thanks for your help

[#8 · Accepted solution]

Hi @Qi Wang, there is a bug from Ambari perspective, it is not generating hiveserver2-site.xml. So any changes made in Advanced Hiveserver2 site section from Ambari are not getting reflected, (we make changes in hiveserver2-site.xml for Ranger) so if you disable authorization from the general settings as mentioned above, your Ranger policies will not work as expected. This issue has been raised up and will be resolved soon in the upcoming releases of Sandbox.

[#9]

this is a misguided exception , can you please check /etc/hive/conf/conf.server , whether ranger-audit.xml is present there ?

[#10]

under /etc/hive/conf/conf.server, following ranger files are there, but not ranger-audit.xml

-rwxr--r-- 1 hive hadoop 1208 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-hive-audit.xml
-rwxr--r-- 1 hive hadoop 1072 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-hive-security.xml
-rwxr--r-- 1 hive hadoop 1030 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-policymgr-ssl.xml
-rw-r--r-- 1 hive hadoop   64 2016-06-26 21:39 /etc/hive/conf/conf.server/ranger-security.xml