hbase shell missing class name ('org.apache.log4j.level')

bigdata_dev01 — Tue, 24 Sep 2019 03:26:27 GMT

I am getting the below error while trying to start hbase 2.2.0 shell.
I have downloaded the `hbase-2.2.0 binary tar.gz` and extracted into a Windows x64 environment 16GB RAM.

As the below error message suggests, it is arising out of `Line 81` of `%HBASE_HOME%\bin\hirb.rb` due to the `org.apache.log4j.Level` class being not found

I set the HBase environment variables in `%HBASE_HOME%\conf\hbase-env.cmd` as well as added all the `%HBASE_HOME%\lib` jars into `HBASE_CLASSPATH` (as shown below) but getting the same error

> %HBASE_HOME%\conf\hbase-env.cmd

set JAVA_HOME=c:\Program Files\Java\jdk1.8.0_101 set HBASE_CLASSPATH=%HBASE_HOME%\lib;%HBASE_HOME%\lib\client-facing-thirdparty;%HBASE_HOME%\lib\ruby;%HBASE_HOME%\lib\zkcli;%HBASE_HOME%\lib\shaded-clients set HBASE_HEAPSIZE=8000 set HBASE_OPTS="-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stack=true" set SERVER_GC_OPTS="-verbose:gc" "-XX:+PrintGCDetails" "-XX:+PrintGCDateStamps" %HBASE_GC_OPTS% set CLIENT_GC_OPTS="-verbose:gc" "-XX:+PrintGCDetails" "-XX:+PrintGCDateStamps" %HBASE_GC_OPTS% set HBASE_USE_GC_LOGFILE=true set HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false" "-Dcom.sun.management.jmxremote.authenticate=false" set HBASE_MASTER_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10101" set HBASE_REGIONSERVER_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10102" set HBASE_THRIFT_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10103" set HBASE_ZOOKEEPER_OPTS=%HBASE_JMX_BASE% -Dcom.sun.management.jmxremote.port=10104" set HBASE_REGIONSERVERS=%HBASE_HOME%\conf\regionservers set HBASE_IDENT_STRING=%USERNAME% set HBASE_MANAGES_ZK=true

Customized the HBase and Zookeeper data directories (which is different from HBASE_HOME) and specified distributed mode to false, to run in standalone mode.

> %HBASE_HOME%\conf\hbase-site.xml

<configuration> <property> <name>hbase.rootdir</name> <value>file:///D:/HBase/data</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>file:///D:/HBase/zookeeper</value> </property> <property> <name>hbase.unsafe.stream.capability.enforce</name> <value>false</value> <description> Controls whether HBase will check for stream capabilities (hflush/hsync). Disable this if you intend to run on LocalFileSystem, denoted by a rootdir with the 'file://' scheme, but be mindful of the NOTE below. WARNING: Setting this to false blinds you to potential data loss and inconsistent system state in the event of process and/or node failures. If HBase is complaining of an inability to use hsync or hflush it's most likely not a false positive. </description> </property> <property> <name>hbase.cluster.distributed</name> <value>false</value> </property> </configuration>

Below is Line 81 of hirb.rb script

Line 81: log_level = org.apache.log4j.Level::ERROR

The error:

NameError: missing class name (`org.apache.log4j.Level') method_missing at org/jruby/javasupport/JavaPackage.java:259 <main> at D:\HBase\hbase-2.2.0-bin.tar\hbase-2.2.0-bin\hbase-2.2.0\bin\hirb.rb:81

> Console log

D:\HBase\hbase-2.2.0-bin.tar\hbase-2.2.0-bin\hbase-2.2.0>bin\hbase shell 2019-09-23T17:54:25.354+0530: [GC (Allocation Failure) 2019-09-23T17:54:25.354+0530: [ParNew: 69952K->8703K(78656K), 0.0227532 secs] 69952K->15376K(253440K), 0.0232659 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 2019-09-23T17:54:25.377+0530: [GC (CMS Initial Mark) [1 CMS-initial-mark: 6672K(174784K)] 15401K(253440K), 0.0024235 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2019-09-23T17:54:25.380+0530: [CMS-concurrent-mark-start] 2019-09-23T17:54:25.504+0530: [CMS-concurrent-mark: 0.124/0.124 secs] [Times: user=0.42 sys=0.02, real=0.12 secs] 2019-09-23T17:54:25.505+0530: [CMS-concurrent-preclean-start] 2019-09-23T17:54:25.507+0530: [CMS-concurrent-preclean: 0.002/0.002 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2019-09-23T17:54:25.508+0530: [CMS-concurrent-abortable-preclean-start] 2019-09-23T17:54:26.668+0530: [CMS-concurrent-abortable-preclean: 0.694/1.160 secs] [Times: user=3.84 sys=0.22, real=1.16 secs] 2019-09-23T17:54:26.669+0530: [GC (CMS Final Remark) [YG occupancy: 49337 K (78656 K)]2019-09-23T17:54:26.669+0530: [Rescan (parallel) , 0.0086939 secs]2019-09-23T17:54:26.678+0530: [weak refs processing, 0.0001806 secs]2019-09-23T17:54:26.678+0530: [class unloading, 0.0045989 secs]2019-09-23T17:54:26.683+0530: [scrub symbol table, 0.0034678 secs]2019-09-23T17:54:26.686+0530: [scrub string table, 0.0008139 secs][1 CMS-remark: 6672K(174784K)] 56010K(253440K), 0.0193630 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 2019-09-23T17:54:26.693+0530: [CMS-concurrent-sweep-start] 2019-09-23T17:54:26.694+0530: [CMS-concurrent-sweep: 0.001/0.001 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2019-09-23T17:54:26.694+0530: [CMS-concurrent-reset-start] 2019-09-23T17:54:26.704+0530: [CMS-concurrent-reset: 0.009/0.010 secs] [Times: user=0.05 sys=0.00, real=0.01 secs] 2019-09-23T17:54:26.941+0530: [GC (Allocation Failure) 2019-09-23T17:54:26.941+0530: [ParNew: 78655K->8704K(78656K), 0.0437466 secs] 85328K->26040K(253440K), 0.0442171 secs] [Times: user=0.08 sys=0.02, real=0.04 secs] 2019-09-23T17:54:27.150+0530: [GC (Allocation Failure) 2019-09-23T17:54:27.150+0530: [ParNew: 78656K->8704K(78656K), 0.0176586 secs] 95992K->31897K(253440K), 0.0188060 secs] [Times: user=0.05 sys=0.00, real=0.02 secs] 2019-09-23T17:54:27.408+0530: [GC (Allocation Failure) 2019-09-23T17:54:27.408+0530: [ParNew: 78656K->8548K(78656K), 0.0068995 secs] 101849K->32196K(253440K), 0.0081199 secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 2019-09-23T17:54:27.649+0530: [GC (Allocation Failure) 2019-09-23T17:54:27.650+0530: [ParNew: 78500K->8677K(78656K), 0.0069747 secs] 102148K->32515K(253440K), 0.0081496 secs] [Times: user=0.02 sys=0.00, real=0.01 secs] 2019-09-23T17:54:27.807+0530: [GC (Allocation Failure) 2019-09-23T17:54:27.807+0530: [ParNew: 78629K->7285K(78656K), 0.0074532 secs] 102467K->32051K(253440K), 0.0085688 secs] [Times: user=0.00 sys=0.00, real=0.01 secs] ... 2019-09-23T17:54:44.632+0530: [GC (Allocation Failure) 2019-09-23T17:54:44.633+0530: [ParNew: 77107K->8704K(78656K), 0.0066891 secs] 159919K->91658K(253440K), 0.0080000 secs] [Times: user=0.06 sys=0.00, real=0.01 secs] 2019-09-23T17:54:45.217+0530: [GC (Allocation Failure) 2019-09-23T17:54:45.218+0530: [ParNew: 78656K->8704K(78656K), 0.0125041 secs] 161610K->93529K(253440K), 0.0137068 secs] [Times: user=0.06 sys=0.00, real=0.01 secs] NameError: missing class name (`org.apache.log4j.Level') method_missing at org/jruby/javasupport/JavaPackage.java:259 <main> at D:\HBase\hbase-2.2.0-bin.tar\hbase-2.2.0-bin\hbase-2.2.0\bin\hirb.rb:81 Heap par new generation total 78656K, used 32489K [0x00000005cc000000, 0x00000005d1550000, 0x00000005e0cc0000) eden space 69952K, 34% used [0x00000005cc000000, 0x00000005cd73a7e0, 0x00000005d0450000) from space 8704K, 100% used [0x00000005d0450000, 0x00000005d0cd0000, 0x00000005d0cd0000) to space 8704K, 0% used [0x00000005d0cd0000, 0x00000005d0cd0000, 0x00000005d1550000) concurrent mark-sweep generation total 174784K, used 84825K [0x00000005e0cc0000, 0x00000005eb770000, 0x00000007c0000000) Metaspace used 35739K, capacity 36202K, committed 36360K, reserved 1079296K class space used 6361K, capacity 6500K, committed 6524K, reserved 1048576K

Running `start-hbase` is leading to below error:

D:\HBase\hbase-2.2.0-bin.tar\hbase-2.2.0-bin\hbase-2.2.0>bin\start-hbase Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:187) at org.apache.hadoop.hbase.util.HBaseConfTool.main(HBaseConfTool.java:39) Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 2 more ERROR: Could not determine the startup mode.

Re: hbase shell missing class name ('org.apache.log4j.level')

bigdata_dev01 — Tue, 24 Sep 2019 17:54:25 GMT

I have resolved the issue.

Made the following environment variable settings in %HBASE_HOME%\conf\hbase-env.cmd

Provided JAVA_HOME is set beforehead, in this case JAVA_HOME=C:\Program Files\Java\jdk1.8.0_101 set in Windows environment variable

        set JAVA_HOME=%JAVA_HOME% 
        set HBASE_CLASSPATH=%HBASE_HOME%\lib\client-facing-thirdparty\*
        set HBASE_HEAPSIZE=8000
        set HBASE_OPTS="-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stack=true"
        set SERVER_GC_OPTS="-verbose:gc" "-XX:+PrintGCDetails" "-XX:+PrintGCDateStamps" %HBASE_GC_OPTS%
        set HBASE_USE_GC_LOGFILE=true
        set HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false" "-Dcom.sun.management.jmxremote.authenticate=false"
        set HBASE_MASTER_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10101"
        set HBASE_REGIONSERVER_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10102"
        set HBASE_THRIFT_OPTS=%HBASE_JMX_BASE% "-Dcom.sun.management.jmxremote.port=10103"
        set HBASE_ZOOKEEPER_OPTS=%HBASE_JMX_BASE% -Dcom.sun.management.jmxremote.port=10104"
        set HBASE_REGIONSERVERS=%HBASE_HOME%\conf\regionservers
        set HBASE_LOG_DIR=%HBASE_HOME%\logs
        set HBASE_IDENT_STRING=%USERNAME%
        set HBASE_MANAGES_ZK=true

Since running in standalone mode, add only the following property in %HBASE_HOME%\conf\hbase-site.xml

IMP: No need to add hbase.rootdir or hbase.zookeeper.property.dataDir

<configuration>
        <property>
            <name>hbase.cluster.distributed</name>
            <value>false</value>
        </property>
</configuration>

question Re: hbase shell missing class name ('org.apache.log4j.level') in Support Questions

hbase shell missing class name ('org.apache.log4j.level')

Re: hbase shell missing class name ('org.apache.log4j.level')