Member since
01-22-2016
59
Posts
17
Kudos Received
0
Solutions
07-20-2016
09:45 PM
How would you modify this code to connect to the hortonworks HDP sandbox? I know you have to modify your variables url.dbc and conn, but not quite sure how that would work. Could you please help?
... View more
03-31-2016
03:43 PM
But what are the basic requirements for kafka 0.9 for ssl so that i can only upgrade that?
... View more
02-11-2016
03:34 PM
@khushi kalra Atlas and Hive hook works. Take a look on this tutorial please...you can see that all the tags are defined in xml http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-end-data-pipeline-apache-falcon/
... View more
02-10-2016
11:35 PM
@khushi kalra Please accept one of the answers to close this thread
... View more
02-05-2016
05:54 PM
3 Kudos
You can refer this doc Atlas Falcon bridge.
... View more
02-03-2016
06:39 PM
1 Kudo
@khushi kalra The answer in short is it depends what you are looking for. In Hortonworks platform we have Apache Atlas and Apache Falcon. The 2 tools though under governance has different use case. For Metadata Management with HDP you should use Apache Atlas. The verison 0.5 is the first release of the product and it gets much slicker with the upcoming release. Waterline integrates with Atlas. Waterline will give you metadata discovery, but does not completely integrate with HDP. They run a map reduce job, which will allow you to see patterns in data and say what kind of data it is. Now if you have to take that file metadata and use in conjunction with Hive for any policy work, it will be via Atlas. Atlas is part of the DGI framework. The idea of DGI is to be able to provide an metadata exchange were a community of companies can work in one platform. As Neeraj mentioned, Dataguise is one of them. We have Collibra, Allation and others that are also there. Now the question, I have for you is what are you trying to achieve? Governance is little bit fuzzy in people's mind. Look at the presentation here http://hortonworks.com/partners/learn/#dgi I hope this helps.
... View more
03-30-2016
04:19 PM
Just out of curiosity, how did you get this working, I am working through the same problem on sandbox 2.4, log attachedfalcon-app-log.txt [falcon@sandbox logs]$ hadoop fs -ls -R /apps/falcon
drwxrwxrwx - falcon hdfs 0 2016-03-30 14:56 /apps/falcon/backupCluster
drwxrwxrwx - falcon hdfs 0 2016-03-30 14:54 /apps/falcon/backupCluster/staging
drwxr-xr-x - falcon hdfs 0 2016-03-30 14:56 /apps/falcon/backupCluster/working
drwxrwxrwx - falcon hdfs 0 2016-03-30 14:55 /apps/falcon/primaryCluster
drwxrwxrwx - falcon hdfs 0 2016-03-30 14:53 /apps/falcon/primaryCluster/staging
drwxr-xr-x - falcon hdfs 0 2016-03-30 14:55 /apps/falcon/primaryCluster/working
... View more