Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hi Neeraj can you please help me with apache falcon . my data is in hive how can i tag it and where should i save the metadata in falcon?not the data but metadata

avatar
Rising Star
1 ACCEPTED SOLUTION

avatar
Master Mentor

@khushi kalra

It's a really good question.

See this https://blogs.apache.org/falcon/tags/hive

If you meant that you want to store hive tables definitions , columns and data types etc then you can leverage atlas

https://community.hortonworks.com/questions/2081/atlas-tutorials-and-examples.html

Or you can design your own system using search like solr , drop hive table information in solr index , integrate with banana or Kibana and start searching but you have to be very creative about this process

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@khushi kalra

It's a really good question.

See this https://blogs.apache.org/falcon/tags/hive

If you meant that you want to store hive tables definitions , columns and data types etc then you can leverage atlas

https://community.hortonworks.com/questions/2081/atlas-tutorials-and-examples.html

Or you can design your own system using search like solr , drop hive table information in solr index , integrate with banana or Kibana and start searching but you have to be very creative about this process

avatar
Rising Star

but where does falcon save the metadata and who all can access it? atlas is not a full fleged tool yet right?@Neeraj Sabharwal

avatar
Master Mentor

@khushi kalra Atlas and Hive hook works. Take a look on this tutorial please...you can see that all the tags are defined in xml http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-end-data-pipeline-apache-falcon/