@Freakabhi   You can consider few more points before choose one of the approach, like...   1. Number of records: approach 1 is fine for very huge records and approach 2 is ok for the less records 2. How to handle the issue if something goes wrong? : The 4th step in approach 2 deletes base table and recreate with new data. Consider you have noticed an issue with data after couple of days, how do you get deleted base_table? if you have answer then go for approach 2 3. Approach 3: You are choosing approach 1 because Hbase supports updates but hive does not support updates (I guess this is your understanding). Yes your understand was correct with old hive version. But Update is available in starting Hive 0.14   https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Update   ... View more

@VincentSF   oh ok got it...    Go to CM -> Yarn -> Configuration -> search for "yarn.nodemanager.resource.memory-mb"   it will show you the memory restriction that you set for each node (it will get configuration from yarn-site.xml)   You can tweak this 'little'.    Note: 1. The memory is common for all the services. so you cannot use all the memory for Yarn alone. Also don't increase the memroy for the above setting too much because it may create memory overlap issue accross the services. So may be you can set aprox 50% of total memory but again it is depends upon the memory utilization by other services. Since you have 183 nodes, the 50% is not common for all the nodes, it will change case by case   2. Also when you increase your memory on each node, it is not recommended to increase more than  yarn.scheduler.maximum-allocation-mb   Hope this will give some idea   ... View more

@SatishS   Hiverserver2 Introduced in Hive version 0.11   https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients ... View more

I was able to solve this problem. I was getting the concept wrong. So, what I did was --   1. I created keytab file for my current user i.e "cloudera" 2. The Jaas.conf should have that keytab, not the hdfs keytab. 3. I had to add this user, hdfs user in the yarn allowed hosts through CM. Also, I added solr user too there. 4. kinit the cloudera user 5. Put the required files in the /user/cloudera directory in hdfs. 6. Run the job. ... View more