Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we connect Atlas with Sql table outside hadoop cluster

Solved Go to solution

Can we connect Atlas with Sql table outside hadoop cluster

Rising Star

@Ayub Khan

Hello Sir,

Am confused, Can we connect Atlas to sql tables outside the hadoop cluster, if Yes then how ?

It will be great help If i get some example.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can we connect Atlas with Sql table outside hadoop cluster

@Dinesh Das

Atlas does not currently provide out-of-box integration with components outside of the HDP stack.

You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples:

http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf

Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation.

3 REPLIES 3

Re: Can we connect Atlas with Sql table outside hadoop cluster

@Dinesh Das

Atlas does not currently provide out-of-box integration with components outside of the HDP stack.

You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples:

http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf

Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation.

Re: Can we connect Atlas with Sql table outside hadoop cluster

@Dinesh Das

As @Eyad Garelnabi pointed out, there is no out-of-box integration with mysql. But you can come up with your mysql model in atlas and create your own entities.

Highlighted

Re: Can we connect Atlas with Sql table outside hadoop cluster

Rising Star

You can also use the atlas-client artifact (Java) to import the xls metadata into Atlas. I'd recommend going through atlas.incubator.apache.org/api/v2/index.html for an in-depth REST documentation for Atlas (0.8 onwards)

Don't forget to upvote/accept the answer if you find it useful.