Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How Apache atlas captures lineage for sqoop,kafka and storm(Version 0.5)?

How Apache atlas captures lineage for sqoop,kafka and storm(Version 0.5)?

Expert Contributor

As we everyone know that,Apache atlas(Version 0.5) has capability to capture lineage/metadata for Hive by default.But In Apache Atlas documentation they have explained a concept such as bridge/Hook.Again their,I didn't find much explanation about it.I have few questions over this as follow:

What is bridge/Hook?

What is use of it and why?

Does it require for integrating new hadoop component with Apache atlas?

Can we built bridge/Hook,if yes then what is it's process?

I think Apache atlas comes with inbuilt Bridge/Hook for hive.

2 REPLIES 2
Highlighted

Re: How Apache atlas captures lineage for sqoop,kafka and storm(Version 0.5)?

Hi @Manoj Dhake. Good to see you again!

If you've watched the video clips that I pointed you towards on my last Atlas response to you then you know almost as much as anyone here :o)

The hooks are used to provide the metadata information to Atlas after each and every action has completed within the component (Hive for example).

If you wanted to integrate a new componenet you would indeed be required to write a set of hooks for it.

The process is somewhat complex, I'd suggest starting to review the code on github and even becoming a member of the developer mailing lists.

https://github.com/apache/incubator-atlas

Highlighted

Re: How Apache atlas captures lineage for sqoop,kafka and storm(Version 0.5)?

Expert Contributor

Thank you Drussell.

Don't have an account?
Coming from Hortonworks? Activate your account here