Cloudera Community
AlinaGHERMAN
user rank
Champion Alumni

Re: Minimum number of nodes, and specs for a real ...

by Champion Alumni in Support Questions
‎01-22-2016 01:54 AM
1 Kudo
‎01-22-2016 01:54 AM
1 Kudo
Our test cluster (on amazon): - 5 workers m4.xlarge, 250 GB disk magnetic (we increased the disk to 1T afterwards)            * we used one of the 5 machine just for flume(kafka)  - 2 masters m4.2xlarge, 125 GB SSD (we decreased the memory and CPU afterwards ==> m4.xlarge)   This was perfect for us for testing purposes.   ... View more

Missing configurations in Cloudera Manager

by Champion Alumni in Support Questions
‎01-22-2016 01:44 AM
‎01-22-2016 01:44 AM
Hello,   I wanted to folow the slides in order to optimize HBase :http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok    However, there are some configurations that I didn't find in Cloudera Manager configurations (in their respective services):   namenode.avoid.read.stale.datanode = true namenode.avoid.write.stale.datanode = true namenode.stale.datanode.interval = 30000 client.read.shortcircuit.buffer.size = 131072 regionserver.checksum.verify = true server.tcpnodelay = true client.tcpnodelay = true hregion.majorcompaction.jitter = 0.5 (½ week, default) hstore.min.locality.to.skip.major.compact master.wait.on.regionservers.timeout ipc.client.tcpnodelay zookeeper.useMulti   Why aren't these available?    Thank you!         ... View more

Re: HBase regions are moved on the same machine

by Champion Alumni in Support Questions
‎01-08-2016 06:55 AM
‎01-08-2016 06:55 AM
I found a partial answer to my question:   http://hadoop-hbase.blogspot.fr/2013/07/hbase-and-data-locality.html    Since I am in DEV, I start and Stop the machines every day. Also, we do restart the service from time to time.      ... View more

Re: HBase regions are moved on the same machine

by Champion Alumni in Support Questions
‎01-08-2016 06:38 AM
‎01-08-2016 06:38 AM
Hello, I also have a quick question. On the HBase interface http://<hbase ip>:60010/table.jsp?name=table_name we can see where is each region located. However, normally I have a replication of 3. How is this done? For a region X, is the hole region duplicated on 2 other servers? How can I found out the 2 other servers on wich I can find this X region? ... View more

HBase - Get region start from one shell command

by Champion Alumni in Support Questions
‎01-07-2016 07:48 AM
‎01-07-2016 07:48 AM
Hello,   I want to do a script that decides where to move each region. For that I want to use the command:   move ERROR: wrong number of arguments (0 for 1) Here is some help for this command: Move a region. Optionally specify target regionserver else we choose one at random. NOTE: You pass the encoded region name, not the region name so this command is a little different to the others. The encoded region name is the hash suffix on region names: e.g. if the region name were TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396. then the encoded region name portion is 527db22f95c8a9e0116f0cc13c680396 A server name is its host, port plus startcode. For example: host187.example.com,60020,1289493121758 Examples: hbase> move 'ENCODED_REGIONNAME' hbase> move 'ENCODED_REGIONNAME', 'SERVER_NAME'     For that I started to use the   hdfs dfs -du "/hbase/data/default_table_name/" in order to find the region names.      However, I do not know, if there si a shell command to find out the startcode for each region (?).   Thank you!   ... View more
Labels:

HBase regions are moved on the same machine

by Champion Alumni in Support Questions
‎01-07-2016 05:36 AM
‎01-07-2016 05:36 AM
Hello,     I have several HBase tables (each with a bunch of regions), and I do not understand why, but Hbase tends to put all the regions of the same table on the same machine.   However, I already moved the regions in order to have regions from each table on all workers (with the move command), and the day after I saw that the regions were again on either less machines, either on one machine.    I tried to do 'balance_switch false' but with no success (the regions were still moved on one machine..).     I do not want to have them on less machines, because if all the regions are on one machine, then only one machine is working...   Do I get it wrong? Should I let Hbase move the regions on one machine/less machines?     Thank you!       ... View more
Labels:

can we match HBase partitions with Impala Partitio...

by Champion Alumni in Support Questions
‎01-06-2016 06:01 AM
‎01-06-2016 06:01 AM
Hello,   Impala/ is having better performances on partitioned tables (if they are big enough). However, I use Impala on an external HBase table. My HBase table is partitioned.   Therefore, I wonder if I can match the HBase partitions with the Impala/Hive ones? If yes, How can I do this?     Thank you!   ... View more

Re: oozie- attach the job logs to email (email act...

by Champion Alumni in Support Questions
‎12-17-2015 01:38 AM
‎12-17-2015 01:38 AM
Hello,   Yes I can see my application_id logs . However, frstly it shows the message : Logs not available at /tmp/logs/hdfs/logs/application_1449728267224_0138 Log aggregation has not completed or is not enabled. and then it recharge the page and it shows the good logs: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.3.5-1.cdh5.3.5.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/disk2/yarn/nm/filecache/7107/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Dec 15, 2015 10:16:44 AM com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider get WARNING: You are attempting to use a deprecated API (specifically, attempting to @Inject ServletContext inside an eagerly created singleton. While we allow this for backwards compatibility, be warned that this MAY have unexpected behavior if you have more than one injector (with ServletModule) running in the same JVM. Please consult the Guice documentation at http://code.google.com/p/google-guice/wiki/Servlets for more information. Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Dec 15, 2015 10:16:44 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering ....... (What I'm searching for..) Thank you!   Alina ... View more

Re: Flume - This role's process is starting. This ...

by Champion Alumni in Support Questions
‎12-16-2015 01:12 AM
‎12-16-2015 01:12 AM
I didn't found any OutOfMemory error in the indicated logs (I did a grep).  However, changing the heap helped. So it really was a heap problem 🙂   Thank you!   Alina  ... View more
Contact Me
Online
Offline
Last Visited
‎09-04-2017 03:49 AM
Community Statistics
Member Since ‎11-18-2014 07:55 AM
Last Visited ‎09-04-2017 03:49 AM
Posts 196
Kudos received 16