@Ash Pad I personally like @Ted Yu answer. Until that is release I don't recommend storing these files on hbase. Instead a common practice to have the file save on HDFS and have the "pointer" stored in hbase. ... View more

@Michael Sobelman That DNS is not detectable by the node you are trying to access from. You can be fancy on aws and configure through routing tables by setting up a proper vpn between the EMR and NiFi nodes. Another option I used is route53 which will give you DNS publicly available. Lastly you can put a ELB infront of your EMR HBase master node. You may have to script it up (via boot scripts) to configure your ELB to point to new internal IP. ... View more

@Laurent Edel thanks. I thought so but needed another to confirm. ... View more

@Samie WALA I have not seen this before. I need to do a quick dive into the code. until then @Artem Ervits have you seen this? ... View more

@Randy Gelhausen that is is awesome feedback. thanks for the insights. I more then likely would use the api for most use cases. for this specific use case a jdbc connector is required. ... View more

@Samie WALA can you post what the ambari log is spitting out when you try to restart service? ... View more

@slachterman and @Randy Gelhausen thanks for the answers. helpful ... View more

I use this blog often when I forget the data movement between map-->reduce The map outputs are copied to the reduce task JVM’s memory if they are small enough (the buffer’s size is controlled by mapred.job.shuffle.input.buffer.percent, which specifies the proportion of the heap to use for this purpose); otherwise, they are copied to disk. When the in-memory buffer reaches a threshold size (controlled by mapred.job.shuffle.merge.percent) or reaches a threshold number of map outputs(mapred.inmem.merge.threshold), it is merged and spilled to disk. ... View more