---> Large file can be input to your Map Reduce program ----> Small file can be passed in distributed cache and can be loaded in List ----> Inside your mapper function, you can do comparisons(Input file vs List) or any other operation which you want. Let me know if you need help in MR job. Please post sample data for files and operation you want to perform. ... View more

There are no error details in post. https://community.hortonworks.com/content/supportkb/48985/how-to-setup-oozie-shell-action.html This is article which may help. Can you post error from yarn logs? Check the map logs under application master. ... View more

Hi @Koundinya Ravulapati, I could think of two ways. - Renewal of ticket via code i.e running background thread to renew ticket. You can refer below mentioned link- https://github.com/joshelser/phoenix-kerberos/blob/master/src/main/java/com/github/joshelser/PhoenixWithRenewal.java - Running kinit in cron job and using ticket from cache. Thanks Shubham ... View more

Not sure about direct command. But you can use some shell script. Script: ----------------------------------------------------------------------------------------------------------------------------------------------------------------- for a in `yarn applicationattempt -list application_1522234336469_0002|grep appattempt|awk '{print $1}'` do echo "Attempt $a nodes" for b in `yarn container -list $a|grep 'container_'| awk '{print $15}'` do echo "Node: $b" done done ----------------------------------------------------------------------------------------------------------------------------------------------------------------- ... View more

---->How region server will use heap and what are the parameters will occupy heap memory Region server heap memory consumption is mainly dependent on below three. - Block cache (Buffer maintained in heap for read) - Memstore Size (Buffer maintained in heap for write and flush) - Other Objects created within region server while during various operations. Below two are the parameter which controls the max % of heap block cache and block cache consume. hfile.block.cache.size hbase.regionserver.global.memstore.size =>Below links will help in understading region server configuration for block cache/memstore and region server sizing. https://hbase.apache.org/book.html#perf.rs.memstore.size https://hbase.apache.org/book.html#ops.capacity.regions.count https://hbase.apache.org/book.html#block.cache.usage -----> Is region server will use only 16 GB heap or even other space of RAM Region server will comsume maximum = 16GB of heap + XX:MaxDirectMemorySize which you have configured. MaxDirectMemorySize—The JVM has a kind of memory called direct memory, which is distinct from normal JVM heap memory, that can run out. You can increase the direct buffer memory either by increasing the maximum heap size (see previous JVM Heap Size), which increases both the maximum heap and the maximum direct memory, or by only increasing the maximum direct memory using -XX:MaxDirectMemorySize. The following parameter added to the Java application startup increases the maximum direct memory size to 256 megabytes: ... View more

Hi @Dhiraj Sardana Can you try deleting STATS for table and check count again? delete from SYSTEM.STATS where physical_name='<TABLE_NAME>' -Shubham ... View more