@heta desai Take a look at the below demo using Nifi to fetch the tweets and Spark and Zeppelin for the analysis. https://community.hortonworks.com/articles/30213/us-presidential-election-tweet-analysis-using-hdfn.html ... View more

@mel mendoza Kafka is a message broker so it only receives files/events from publishers and makes them available for consumption by consumers. It does not do any processing. Spark streaming would dictate how files/events are read. Since Spark Streaming does micro-batching it will read several files/events from Kafka and process them together in a micro-batch. I believe this will achieve what you are asking to do, it'll be on the Spark side though, not Kafka. As always, if you find this post helpful, don't forget to "accept" answer. ... View more

@Deepak Rawat In VMware, right click on your sandbox VM, choose settings and then modify the number of CPUs and RAM that are configured to 1 CPU, also set the RAM to half of what your machine has. Now you should be able to start your sandbox VM. Please note that the Sandbox is configured to run with 4 CPUs and 8GB RAM. Running a configuration less than that will cause the sandbox to run very slow and possibly some services/jobs fail. As always, if you find this post helpful, don't forget to "accept" answer. ... View more

@Gaurav Vats You can reset the Ambari admin password by using the below steps. 1. Start your sandbox, ssh using a terminal or the web browser link 2. Run the following commands: ambari-admin-password-reset The following text will appear Please set the password for admin: Please retype the password for admin: 3. After setting and retyping your new password, type the command: ambari-agent restart 4. Ambari Admin password should be reset 5. Open Ambari login page. Verify your new password allows you to login as admin user. As always, if you find this post helpful, don't forget to "accept" answer. ... View more

@Dinesh Das Prohibition against dataset combinations: https://community.hortonworks.com/content/kbentry/63664/how-to-create-a-ranger-policy-that-prohibits-combi.html Data Expiry-based access policy: https://community.hortonworks.com/articles/92083/using-atlas-and-ranger-to-enforce-data-expiration.html Location-specific access policies: https://community.hortonworks.com/articles/57314/customizing-ranger-policies-with-dynamic-context.html As always, if you find this post helpful, don't forget to "accept" answer. ... View more

@Qinglin Xia You are correct. When you look at the lineage graph, anything leading up to the entity in consideration (which is highlighted in red) is called "lineage". "Impact" is how that particular entity influences others down the graph. ... View more

@Prakhar Agrawal Take a look at the DistributedMapCache. It allows you to save flowfile data for consumption by processors across nodes and processors. There are two controller components, the DistributesMapCacheServer (runs on one node) and the DistricutedMapCacheClient (runs on all nodes if you've got a cluster). There are also the processors that interact with the DistributedMapCache, namely FetchDistributedMapCache and PutDistributedMapCache Take a look here for some elaboration/documentation: https://nifi.apache.org/docs.html https://community.hortonworks.com/questions/35223/distributedmapcacheclientservice-nifi-wecrawlerxml.html Here for examples using DistributedMapCache: https://github.com/hortonworks-gallery/nifi-templates/tree/master/templates (webcrawler.xml template) http://funnifi.blogspot.ca/2016/04/ ... View more