@Antony Shajin Lucas Yes, this is the expected behavior. Tags are independent of entities. You can create a tag and apply it to multiple entities. If an entity is removed the tag will still remain. ... View more

@robert cheung Deny policies take precedence over allow policies. So, in your scenario above, User 1 should not have access to data tagged as both, "International" and "PII". Take a look at the flow chart below on the sequence of policy evaluation. ... View more

@Alp Alp This is currently not possible. Atlas-Hive tracking is all or nothing. What is the business driver behind this requirement? If you can explain the business need then I'll be able to raise it with our product managers and the community for consideration in future releases. ... View more

@Zakir Hossain The way to approach this isn't through Atlas, but rather through the way you ingest the data into Hive. Rather than dropping/deleting and recreating the Hive table you can either use "Truncate" or "Insert Overwrite" to replace the data only. This way, your metadata and tags stay intact and only your data is refreshed. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-TruncateTable https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-InsertingdataintoHiveTablesfromqueries *Keep in mind that if you have ACID enabled then you will not be able to use "Insert Overwrite", and your only option is "Truncate" As of Hive 0.14, if a table has an OutputFormat that implements AcidOutputFormat and the system is configured to use a transaction manager that implements ACID, then INSERT OVERWRITE will be disabled for that table. This is to avoid users unintentionally overwriting transaction history. The same functionality can be achieved by using TRUNCATE TABLE (for non-partitioned tables) or DROP PARTITION followed by INSERT INTO. ... View more

@heta desai One option is to use Nifi/HDF to get the tweets from twitter and then post them to Kafka. Take a look at the demo in the link below to give you an idea ho that might work. The destination will be Kafka though rather than HDFS or Solr (which is what the example uses). https://community.hortonworks.com/articles/1282/sample-hdfnifi-flow-to-push-tweets-into-solrbanana.html Alternatively, you can write a Java producer using the Twitter API to get the tweets and put them in Kafka as per the examples below. https://acadgild.com/blog/streaming-twitter-data-using-kafka/ https://www.tutorialspoint.com/apache_kafka/apache_kafka_real_time_application.htm ... View more

@Doug Lennings You can use the ExecuteSQL processor in Nifi and configure it to use Impala's JDBC. ... View more

@rohith ak HDPCD Spark Certification uses the following versions: HDP 2.4.0 Spark 1.6 Scala 2.10.5 Python 2.7.6 (pyspark) Details in link below: https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/05/HDCD_Spark_Data_Sheet.pdf While it's ok to practice on a newer version, just keep in mind that there's functionality in 2.x that is not available in 1.6. I strongly suggest that you study and practice on HDP 2.4 (Spark 1.6) to avoid confusing yourself. You can download the HDP 2.4 sandbox that contains Spark 1.6 from the link below. https://hortonworks.com/downloads/#sandbox Scroll down to where it says "Hortonworks Data Platform Archive" and click "Expand" to get the archived versions and the download link for HDP 2.4. ... View more