question Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found in Support Questions

Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

qiwang — Wed, 06 Jul 2016 23:05:41 GMT

While trying out the sandbox 2.5 TP for some new features in Atlas, I encountered an error when running sqoop cmd to move data from mySQL to Hive (this should trigger a linage generated inside Atlas and that is my goal)

created a simple table inside mysql and then sqoop the data under ambari-qa credential

mysql --host=sandbox.hortonworks.com --user=root --database=hive 
use test; 
create table source_fact (id int, name varchar(100), description varchar(100));
exit;

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --table source_fact --hive-import --hive-overwrite --hive-table atlasdemo.source_fact --username root -m 1

Then got the following error:

FAILED: RuntimeException java.io.FileNotFoundException: /etc/hive/2.5.0.0-817/0/xasecure-audit.xml (No such file or directory)

this only happens when running sqoop. Hive, Atals and Ranger all looks fine and function as expected.

Thanks,

Note: move this to security/Ranger, look like this is related to Ranger config after looking at some old thread. And based on the old thread, xasecure-audit.xml should not be part of the new version anyway. Looking for some Ranger guru to provide some feedbacks.

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

ravi1 — Thu, 07 Jul 2016 01:16:05 GMT

A quick workaround is to disable ranger authorization from hive->authorization in ambari UI and restart hive. However, if ranger is part of your tests and want to keep it enabled, this is not the solution.

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

qiwang — Thu, 07 Jul 2016 04:11:17 GMT

Ravi, I want to take a look of Atlas/Ranger integration in HDP2.5. So disable Ranger is not really an option.

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

mqureshi — Thu, 07 Jul 2016 04:58:41 GMT

Hi @Qi Wang

Which user is running the sqoop command? Can you verify file /etc/hive/2.5.0.0-817/0/xasecure-audit.xml exists? Does the user running sqoop import has read access to this file? Also, check the following link. It might be your issue.

https://community.hortonworks.com/questions/369/installed-ranger-in-a-cluster-and-running-into-the.html

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

qiwang — Thu, 07 Jul 2016 05:01:49 GMT

I tried different users running sqoop cmd and none worked, including root, ambari-qa, sqoop, hive...

the file /etc/hive/2.5.0.0-817/0/xasecure-audit.xml does not exist. And I did a search of all folders and it doesn't exist on the sandbox VM

cd /
find -name xasecure-audit.xml

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

mqureshi — Thu, 07 Jul 2016 05:02:59 GMT

Check the link I just added to my answer.

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

qiwang — Thu, 07 Jul 2016 09:12:34 GMT

I tried the suggested fix in the old thread and it is not working. Still got the same error with sqoop. Move this to ranger section and hoping some ranger guru jump in.

Thanks for your help

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

mrizvi — Sat, 09 Jul 2016 17:43:41 GMT

Hi @Qi Wang, there is a bug from Ambari perspective, it is not generating hiveserver2-site.xml. So any changes made in Advanced Hiveserver2 site section from Ambari are not getting reflected, (we make changes in hiveserver2-site.xml for Ranger) so if you disable authorization from the general settings as mentioned above, your Ranger policies will not work as expected. This issue has been raised up and will be resolved soon in the upcoming releases of Sandbox.

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

dsharma — Tue, 12 Jul 2016 19:56:18 GMT

this is a misguided exception , can you please check /etc/hive/conf/conf.server , whether ranger-audit.xml is present there ?

Re: Sandbox HDP 2.5 TP: error when running sqoop, xasecure-audit.xml not found

qiwang — Wed, 13 Jul 2016 04:25:05 GMT

under /etc/hive/conf/conf.server, following ranger files are there, but not ranger-audit.xml

-rwxr--r-- 1 hive hadoop 1208 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-hive-audit.xml
-rwxr--r-- 1 hive hadoop 1072 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-hive-security.xml
-rwxr--r-- 1 hive hadoop 1030 2016-06-25 16:25 /etc/hive/conf/conf.server/ranger-policymgr-ssl.xml
-rw-r--r-- 1 hive hadoop   64 2016-06-26 21:39 /etc/hive/conf/conf.server/ranger-security.xml