@Dinesh Das Atlas does not currently provide out-of-box integration with components outside of the HDP stack. You can, however, create your own entities and use the REST API (or Kafka messages) to populate them. This, of course, will require some scripting/coding from you to push the data via REST. Here is some documentation and examples: http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in the latest version (HDP 2.6) and will be removed in future ones. Still, it's good to get you started with your implementation. ... View more

@vnandigam You are correct, Atlas does not currently provide lineage for Spark. This is something engineering/community is working on. You can, however, create your own entities and use the REST API to populate them. Here is some documentation and examples: http://atlas.apache.org/0.7.0-incubating/AtlasTechnicalUserGuide.pdf Please note that while this documentation also applies to Atlas 0.7-0.8 (in HDP 2.5-2.6), it does use APIs that have been deprecated in that version and will be removed n future ones. Still, it's good to get you started with your implementation. As always, if you find any responses here useful, don't forget to "accept" an answer. ... View more

@regie canada You can use the ExtractText processor and use regex within it to pull the first 4 lines. Your regex would be: (.*)\n(.*)\n(.*)\n(.*) After that you can use the SplitText processor if you want each line to be an individual flowfile or you can use the UpdateAttribute processor to make any kind of transformations on the 4 lines. ... View more

@Robin Dong Hi Robin, Can you please close/delete this question since it's a duplicate of https://community.hortonworks.com/questions/103733/anyone-have-info-on-how-mongodb-do-the-sharding-wi.html Thanks ... View more

@Robin Dong What repo did you use to install MongoDB? Have you seen this one:https://github.com/geniuszhe/ambari-mongodb-cluster ... View more