@Gaurab D Can you please share logs for the job? There should be more info you can find either in Ambari or just look under /var/log/sqoop folder or may be /var/log/yarn ... View more

@Alex Raj I am a little confused by the following statement: "We have HBase tables where the data is in in Binary Avro format." HBase stores data in HFiles and it's HBase's own format and not Avro. May be what you mean is you are exporting data from HBase into Avro and using Hive to read that data. If this is true, you can continue to do that as there are some advantages to this approach but if you want to keep data in HBase without moving it, then you can simply use Phoenix on top of HBase to read that data without moving it. In fact you can use Hive to read data in HBase. It's slow compared to Phoenix but it will do the job. May be that's what you are doing right now. On the other hand, if you want to use Phoenix on top of HBase, you can read HBase tables from Phoenix using SQL. Again, you don't have to export data. Here is a link to quick start Phoenix. The point is Avro doesn't come into play here and it's a little confusing why you are asking for Avro format. Between Phoenix and Drill, I would recommend using Phoenix because it's solely created for HBase and has better features and support compared to Drill. ... View more

@Zack Riesland Yes, it is safe to remove these folders and do a clean up. There are already actually cleanup scripts for this. Basically when a client runs a query with HiveServer2, Hive first creates these temporary folders to store intermediate/temporary data. For most queries, this is cleaned up at the end of query but sometimes due to issues with the query, these files are still hanging and you have to do a manual cleanup. Check this link for more details. https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration#AdminManualConfiguration-TemporaryFolders Following link might also give you some ideas on how to cleanup. https://community.hortonworks.com/questions/19204/do-we-have-any-script-which-we-can-use-to-clean-tm.html ... View more

@Kyle Travis To start with please check the following link for some basics which I assume you already know, but I think it's still a good start. https://community.hortonworks.com/questions/55201/number-of-zookeepers-in-a-3-rack-cluster-with-data.html Now, to the question whether you should have separate zookeeper quorums. You mention "a few hundred nodes". I am assuming it's a HBase cluster and Storm is writing to HBase. Let me make some assumptions about your application. 1. High read/write throughput. 2. Time sensitive. HBase latency is important. 3. Other hadoop components also running, like Hive or Phoenix may be. 4. Kafka is not being used. If my assumptions are reasonable and close to what you really have, I think having a separate Zookeeper for HBase might give you some benefits. Zookeeper is very sensitive with timeouts. Serving multiple components at this scale, it might make sense to give HBase its own Zookeeper. This would ensure better HBase operation/stability as compared to when you share Zookeeper across. Although, someone might argue that, you shouldn't run into any issue with just one zookeeper either. Have you seen any issues in your testing? For all the rest of cluster, just one zookeeper quorum is fine. However, if you are using Kafka, I will have one quorum for everything including HBase and one zookeeper for just Kafka. Kafka is very fragile with Zookeeper. In my experience, it is better Kafka has its own Zookeeper. This brings to last point of your question. Two zookeepers are currently not supported by Ambari. There is a jira open but no support yet. https://issues.apache.org/jira/browse/AMBARI-14714 ... View more

Well, this really depends on your tolerance for failure. Zookeeper requires a quorum of servers to be up at any time. It uses a majority quorum to make a decision. Zookeeper is up when ceil(N/2) servers are up where N are total number of servers in the quorum. For 3 node zookeeper, you can tolerate one failure. For 5 node zookeeper, you can tolerate up to 2 failures. the reason I would recommend 5 zookeeper nodes in your case is because you have a 100 node cluster. To make sure your business continuity and be confidently tolerate couple of failures, it's better to go with 5 zookeepers. Also, think about planned maintenance. With five zookeepers, you can take one out for maintenance and still have a tolerance of one more failure. With three zookeepers, maintenance is also a challenge. That being said, now that you know the implications of going with 3 vs 5 zookeepers, you can decide to go with three zookeepers knowing that in case of one zookeeper failure, you have limited window to bring the failed zookeeper up because one more zookeeper failure means risk to business. ... View more