Hi @Sarah Maadawy. Did you run ./tpcds-setup.sh 100? That's 100 GB of data. Are you sure you wanted that much data? You might be running out of space. ... View more

Hi @Vasilis Vagias. Looks like you downloaded the vmdk for vmware. Make you download the ova for VirtualBox instead. ... View more

Changing "choose authorization" to NONE fixed the problem. 2016-08-23-09-34-47.png ... View more

@ripunjay godhani 1. YARN provides resource isolation for most data access. The exception is streaming processes in which you will want to size and dedicate your hardware appropriately. You can use Capacity Scheduler for fine-grained resource allocations. There are also node labeling if you want to run certain jobs on certain nodes based on their hardware architecture. You will still want to do your due diligence around service co-location and proactively monitor and maintain your environment for proper performance. SmartSense is vital to helping you in this regard. 2. You cannot have multiple HDP versions under a single Ambari server. HDP 2.5 will allow for multiple Spark versions but the core HDP will need to be the same. You use development and/or test environments to test upgrades or varying tech preview components. 3. Again, YARN Capacity Scheduler can fence off application resources so no one application will consume all your cluster resources. Security is always a concern but following best practices around encryption, authentication, authorization, auditing, RBAC policies, etc. will help alleviate most scenarios. If you think about it, we've been using shared storage (SAN) for over a decade. HDFS is similar but much more versatile than SAN. Still, the same basic centralized storage concept applies. ... View more

@Vincent Romeo. Your use case makes a lot of sense. I don't know for sure but you might be able to override the setting. Adding Wei to the conversation. + @Wei Zheng ... View more

Hi @Vincent Romeo I think I know what's going on here. The issue isn't that there is something technically different between external and internal tables but, instead there is a design expectation between the two functionalities. A user will use external tables because they expect the data to not change. In this way, you could have multiple schemas applied to the same data set without fear of any one user deleting or changing the data and, if they decide to drop a table, the data isn't removed. The sole purpose of ACID is to insert, update, and delete data so this goes against the basic premise of why you would use external tables. To adhere to this expectation, the developers essentially disable the ability to run ACID on external tables, i.e. disable compaction which is the change mechanism for Hive ACID. Hope this helps! ... View more

Hi @devers, ACID does require some performance considerations especially if you have a high number of inserts and deletes. The performance will slowly degrade over time until a compaction is executed. After compaction your performance will normalize. Based on your use case you'll want to adjust the compaction frequency to find your unique performance sweet spot. https://community.hortonworks.com/questions/15095/hive-compaction-for-acid-transactions.html ... View more