Can we connect Atlas with Sql table outside hadoop cluster

[#1]

@Ayub Khan

Hello Sir,

Am confused, Can we connect Atlas to sql tables outside the hadoop cluster, if Yes then how ?

It will be great help If i get some example.

[#2 · Accepted solution]

@Dinesh Das

Atlas does not currently provide out-of-box integration with components outside of the HDP stack.

You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples:

http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf

Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation.

[#3]

@Dinesh Das

As @Eyad Garelnabi pointed out, there is no out-of-box integration with mysql. But you can come up with your mysql model in atlas and create your own entities.

[#4]

You can also use the atlas-client artifact (Java) to import the xls metadata into Atlas. I'd recommend going through atlas.incubator.apache.org/api/v2/index.html for an in-depth REST documentation for Atlas (0.8 onwards)

Don't forget to upvote/accept the answer if you find it useful.