Member since
01-27-2017
6
Posts
1
Kudos Received
0
Solutions
10-01-2017
04:46 PM
Greetings, I am seeing the following problem, and hope somebody might be able to provide some guidance. I have a rudimentary Hadoop installation: Three virtual hosts; one HDFS NameNode and three HDFS DataNodes (one per host); one Yarn NodeManager and one Yarn ResourceManager per host; one HBase Master and three HBase Regionservers (one per host); three Zoo Keeper servers (one per host). All these components are connected and operating seamlessly as a cluster. For example, the ZooKeeper servers have achieved quorum and elected a leader; HBase sees a single "global state", and tables have been created and data ingested, etc. Everything runs fine. However, when I reboot one of my worker hosts (to simulate a host crash, in order to test resiliency of my cluster), although ZooKeeper instantly recovers and elects a new leader from the remaining two ZooKeeper servers, HBase seems to lose track of where things are. For example, from the HBase Master host, I can still list tables (using hbase shell), but cannot scan or get them; from the other, remaining HBase worker host (again, using hbase shell), I can't even list tables any tables (it appears as if zero tables have been defined). If I (re-)start the Hadoop components on the rebooted worker host, the newly started ZooKeeper server connects with the other two, as I'd expect. But the global state for the HBase remains the same -- strangely disconnected, as I'd described. The only solution is to shut the entire cluster down and then bring it back up; then, everything is perfect once again. I suspect that that momentary disruption of ZooKeeper (even though ZooKeeper instantly heals itself following it), is somehow affecting HBase in a manner that HBase can't recover from, but haven't been able to figure why, or what to do to prevent that. I am not running any HQuorumPeer processes, nor a backup HBase master at this time, just in case anybody was wondering.... Thanks in advance for any possible suggestions! ~ Joseph
... View more
Labels:
- Labels:
-
Apache HBase
05-17-2017
07:07 PM
Thanks very much, @mqureshi....will give that a try! 🙂
... View more
05-17-2017
04:56 PM
Greetings! When manually installing lucidworks-hdpsearch (solr) using yum, is there any way to direct the installation toward some location other than /opt/lucidworks-hdpsearch? For example, if I wanted to install it in some other file system/mount point, but leave everything else as is; e.g., after running yum, I'd like to find solr in /myFS/lucidworks-hdpsearch Thanks very much! ~ Joseph
... View more
Labels:
- Labels:
-
Apache Solr
03-21-2017
02:49 AM
Greeings, all. Can anybody tell me if a File URI is mandatory for each component of the value of the dfs.datanode.data.dir element in the hdfs-site.xml file? For example, am I required to specify "file:///grid/hadoop/hdfs/dn, file:///grid1/hadoop/hdfs/dn"? Or, could I use "/grid/hadoop/hdfs/dn, /grid1/hadoop/hdfs/dn", instead, and have effectively the same metadata? I'm using HDP 2.4.3. Thanks very much! ~ Joseph
... View more
Labels:
- Labels:
-
Apache Hadoop
03-02-2017
02:54 AM
Thanks...helped me out right away!
... View more
02-17-2017
03:28 PM
1 Kudo
Greetings, all. I have a very simple configuration: 3 node cluster -- one master, two workers; HDFS NN on master, HDFS SNN on one of the workers, and HDFS DN on each node. ZooKeeper successfully started on all three nodes with an established quorum and elected leader, etc. When I attempt to start HBase Master on the master node, it waits while HDFS NN works in "safe mode", doing its block replication thing. (While using the HBase shell to issue a status command, you get "HBase Initializing", etc.). Eventually, HBase Master goes away, because it times out waiting for HDFS NN to finish and exist safe mode. I have three questions: 1) Does my (simplest possible attempt) configuration, as described above, seem reasonable? 2) I'd prefer not to force HDFS out of safe mode; but as this is just an initial startup of a freshly provisioned Hadoop cluster, I assume that might not cause any issues. Do you agree? 3) Is there a threshold setting somewhere in the config. files that I can raise to ensure HBase Master waits longer for HDFS? I've been searching, but haven't found anything yet. Attached is an abbreviated screen cap of log output...some messages just before the initial warning, and then the exception indicating that HBase Master has finally shutdown. Thanks in advance for your help on this one! ~ Joseph
... View more
Labels:
- Labels:
-
Apache HBase