Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Atlas based based lineage / Ranger based access & COTS ETL tools

avatar
New Contributor

Do Atlas / Ranger based lineage / tag based access keep on working when a Hive table is processed with a 'external' ETL tool like SAS DI or SAP BODS and then written back to a (different) Hive table?

I would guess this is only possible when the metadata from these ETL tools can sync up with Atlas / Ranger

http://hortonworks.com/apache/atlas/#section_1 tells me that:

Atlas, at its core, is designed to exchange metadata with other tools and processes within and outside of the Hadoop stack, thereby enabling platform-agnostic governance controls that effectively address compliance requirements

...so if I am reading this correctly, the functionallity I seek is at least 'on the roadmap'...

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
New Contributor

Hi @Dave Russell, thank you for your answer.

I have a followup question to your response: do you know whether there is more 'native hadoop-like' ETl tooling available at this moment that generates pure map-reduce / hive / pig scripts 'under the hood' so this kind of lineage & security inheritance is not lost?

I am thinking about tooling from companies like Talend or Syncsort DMX-h here...

avatar
Contributor

Hello all,

I am in the same question as @Dave Russell expecting that companies like Talend make this available as soon as possible.

Thanks