@Antony Shajin Lucas Yes, this is the expected behavior. Tags are independent of entities. You can create a tag and apply it to multiple entities. If an entity is removed the tag will still remain. ... View more

Are you only looking to transfer tags or everything (entities, tags, lineage)? ... View more

@robert cheung Deny policies take precedence over allow policies. So, in your scenario above, User 1 should not have access to data tagged as both, "International" and "PII". Take a look at the flow chart below on the sequence of policy evaluation. ... View more

@vikas aggarwal Yes, the community is working on a replacement Taxonomy/Business Catalog feature. However, we do not have a release timeframe yet. ... View more

@Alp Alp This is currently not possible. Atlas-Hive tracking is all or nothing. What is the business driver behind this requirement? If you can explain the business need then I'll be able to raise it with our product managers and the community for consideration in future releases. ... View more

No, tags are associated to entities. If you want something associated with a type then just add an attribute to it. ... View more

You can create a Tag that accepts attributes. For example, an "Expiry_Date" tag that contains an attribute named "date". You can associate that tag directly to your table or column. At the time of association, the user will be asked to fill out the "date" value for that particular entity/instance. So, yes, this could be a workaround. ... View more

I'm afraid not at this point. The community has stopped development on this code-base and is looking at alternate paths. I'll update this post once I have more information. ... View more

@Zakir Hossain The way to approach this isn't through Atlas, but rather through the way you ingest the data into Hive. Rather than dropping/deleting and recreating the Hive table you can either use "Truncate" or "Insert Overwrite" to replace the data only. This way, your metadata and tags stay intact and only your data is refreshed. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-TruncateTable https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-InsertingdataintoHiveTablesfromqueries *Keep in mind that if you have ACID enabled then you will not be able to use "Insert Overwrite", and your only option is "Truncate" As of Hive 0.14, if a table has an OutputFormat that implements AcidOutputFormat and the system is configured to use a transaction manager that implements ACID, then INSERT OVERWRITE will be disabled for that table. This is to avoid users unintentionally overwriting transaction history. The same functionality can be achieved by using TRUNCATE TABLE (for non-partitioned tables) or DROP PARTITION followed by INSERT INTO. ... View more

@Riddhi Sam For an explanation of the evolution of Hadoop 1 to Hadoop 2 take a look at this article/blog-post: http://www.tomsitpro.com/articles/hadoop-2-vs-1,2-718.html For an explanation of what to expect with Hadoop 3 read the below: https://hortonworks.com/blog/data-lake-3-0-deploy-minutes-cut-tco-half/ https://hortonworks.com/blog/data-lake-3-0-part-2-multi-colored-yarn/ Also, take a look at the below presentation from Dataworks Summit that covers YARN past, present and future: https://www.youtube.com/watch?v=PRsr1hgidQI&index=137&list=PLQ-KRsI-e9bBoDjQV_pe7L9171r2Sddsr ... View more

duplicated here: https://community.hortonworks.com/questions/115277/hdp-sandbox-not-able-to-connect-1270018888-or-1270-1.html ... View more

duplicated here: https://community.hortonworks.com/questions/115277/hdp-sandbox-not-able-to-connect-1270018888-or-1270-1.html ... View more

Thanks Matt, Interesting approach and makes a lot of sense to do things that way. I'll give it a try. Thanks for your help. ... View more

@heta desai One option is to use Nifi/HDF to get the tweets from twitter and then post them to Kafka. Take a look at the demo in the link below to give you an idea ho that might work. The destination will be Kafka though rather than HDFS or Solr (which is what the example uses). https://community.hortonworks.com/articles/1282/sample-hdfnifi-flow-to-push-tweets-into-solrbanana.html Alternatively, you can write a Java producer using the Twitter API to get the tweets and put them in Kafka as per the examples below. https://acadgild.com/blog/streaming-twitter-data-using-kafka/ https://www.tutorialspoint.com/apache_kafka/apache_kafka_real_time_application.htm ... View more

No, Atlas only works with Titan, not Janusgraph. So you cannot use DynamoDB as a datastore for Atlas. ... View more

When you follow the links to the Github repo you'll see that AWS has built custom adapters to allow integration with JanusGraph not Titan. JanusGraph is a fork of the Titan project and has some differences. ... View more

How are you trying to use the metadata? In most of our implementations we use the Atlas REST API (http://atlas.apache.org/api/v2/index.html ) for metadata/lineage import and export. Have you considered using that? Please note that I have linked above to the new API, the legacy API ( http://atlas.apache.org/api/rest.html ) has been deprecated with HDP 2.6 (Atlas 0.8) and will be removed in a future version. ... View more

Please share a link to the source. There might be some confusion. ... View more

@Doug Lennings You can use the ExecuteSQL processor in Nifi and configure it to use Impala's JDBC. ... View more

@rohith ak HDPCD Spark Certification uses the following versions: HDP 2.4.0 Spark 1.6 Scala 2.10.5 Python 2.7.6 (pyspark) Details in link below: https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/05/HDCD_Spark_Data_Sheet.pdf While it's ok to practice on a newer version, just keep in mind that there's functionality in 2.x that is not available in 1.6. I strongly suggest that you study and practice on HDP 2.4 (Spark 1.6) to avoid confusing yourself. You can download the HDP 2.4 sandbox that contains Spark 1.6 from the link below. https://hortonworks.com/downloads/#sandbox Scroll down to where it says "Hortonworks Data Platform Archive" and click "Expand" to get the archived versions and the download link for HDP 2.4. ... View more

For a comparison between compression formats take a look at this link: http://comphadoop.weebly.com/ ... View more

Please share a link to the tutorial you're using ... View more