question Re: Add metadata automatically in Archives of Support Questions (Read Only)

Add metadata automatically

arthur_grevin — Thu, 25 Feb 2016 21:03:03 GMT

Hello,

I am searching a way to do this :

On HDFS, new datas arrived on a certain folder, and in the datas, there could be metadatas like names, emails, that are defined, and I would like a way to use these metadas to class my datas.

I know that you can define tags on Falcon, but it is not automatically.

Is there a way to automatically use the metadatas that are in a file, as to transfer them to falcon or another tools ? The goal is to use a Search tool (like there is in falcon).

Re: Add metadata automatically

nsabharwal — Fri, 26 Feb 2016 01:09:50 GMT

@Arthur GREVIN

Take a look on this https://community.hortonworks.com/content/repo/5240/hadoop-data-pipeline-using-falcon.html

Atlas is the solution for this but as of now, there is hook for hive only. HDFS work is in progress.

Re: Add metadata automatically

arthur_grevin — Fri, 26 Feb 2016 18:10:43 GMT

Thank you for your answer.