Support Questions

Find answers, ask questions, and share your expertise

Can we connect Atlas with Sql table outside hadoop cluster

avatar
Expert Contributor

@Ayub Khan

Hello Sir,

Am confused, Can we connect Atlas to sql tables outside the hadoop cluster, if Yes then how ?

It will be great help If i get some example.

1 ACCEPTED SOLUTION

avatar

@Dinesh Das

Atlas does not currently provide out-of-box integration with components outside of the HDP stack.

You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples:

http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf

Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation.

View solution in original post

3 REPLIES 3

avatar

@Dinesh Das

Atlas does not currently provide out-of-box integration with components outside of the HDP stack.

You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples:

http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf

Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation.

avatar

@Dinesh Das

As @Eyad Garelnabi pointed out, there is no out-of-box integration with mysql. But you can come up with your mysql model in atlas and create your own entities.

avatar
Expert Contributor

You can also use the atlas-client artifact (Java) to import the xls metadata into Atlas. I'd recommend going through atlas.incubator.apache.org/api/v2/index.html for an in-depth REST documentation for Atlas (0.8 onwards)

Don't forget to upvote/accept the answer if you find it useful.