Member since
09-25-2015
13
Posts
8
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
927 | 06-12-2018 09:26 PM |
07-28-2020
11:31 PM
"I highly recommend skimming quickly over following slides, specially starting from slide 7. http://www.slideshare.net/Hadoop_Summit/w-235phall1pandey" This slide is not there at the path
... View more
08-28-2018
06:42 PM
@snukavarapu Thanks for the article, this worked great for me. Is this something you keep continuously updated? If so, what's your strategy for keeping the table updated?
... View more
11-18-2015
04:30 PM
smaller blocks take up more space in the namenode tables, so in a large cluster, small blocks come at a price. What small block sizes can do is allow for more workers to get at the data (half the blocksize == twice the bandwidth), but it also means that code that works with > 128MB of data isn't going to get all the data local to a machine, so more network traffic may occur. And, for apps that spin up fast, you may find that 128 MB blocks are streamed through fast enough that the overhead of scheduling containers and starting up the JVMs outweighs the extra bandwidth opportunities. So the notion of "optimal size" isn't really so clear cut. If you've got a big cluster and you are running out of NN heap space, you're going to want to have a bigger block size whether or not your code likes it. Otherwise, it may depend on your data and the uses made of it. As an experiment, try to save copies of the same data with different block sizes. Then see which is faster to query
... View more
06-30-2016
07:55 AM
I have the same problem, as users of Hive (configured with doAs=false and security in Ranger) create lots of external tables to map their data. But hive is unable to access this data by default, we have to give explicit permissions for hive user to read the hdfs data of external table. That is very cumbersome. I don't see any best practice regarding external tables in the document you referenced. Do you guys have any advice how to handle external tables in such case? Thanks!
... View more
10-12-2015
01:56 PM
Joe, please dont' mention customer names as we dicussed. Please reword your response to remove it.
... View more