Member since
04-12-2019
105
Posts
3
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2561 | 05-28-2019 07:41 AM | |
1205 | 05-28-2019 06:49 AM | |
987 | 12-20-2018 10:54 AM | |
780 | 06-27-2018 09:05 AM | |
4442 | 06-27-2018 09:02 AM |
10-06-2019
06:56 AM
Hi, I'm facing same issue. Have there any luck?
... View more
09-24-2019
06:35 AM
Hi @Shelton Thanks for give your precious time. As you have mentioned, already value is set: #cat /proc/sys/fs/file-max 11534502 #ulimit unlimited #ulimit -Sn 128000 #ulimit -Hn 128000 Each node in Hbase Cluster are already running with these settings. Noticed: vm.swappiness value is 60, also sometime swap uses is more than 80%. #free -m Swap: 7167 6066 1101 Looking for other solution. Regards, Vinay K
... View more
09-23-2019
06:59 AM
Hi Folks,
We are using HDP 2.6.5 version with HBase, oozie. We are getting "java.lang.OutOfMemoryError: unable to create new native thread" error in our logs:
2019-09-22 05:56:37,297 INFO container.ContainerImpl - Container container_e56_1565687270063_197657_01_001489 transitioned from LOCALIZING to LOCALIZED
2019-09-22 05:56:37,299 FATAL event.AsyncDispatcher - Error in dispatcher thread
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:118)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:55)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:110)
at java.lang.Thread.run(Thread.java:748)
2019-09-22 05:56:37,300 INFO ipc.Server - Auth successful for appattempt_1565687270063_197657_000001 (auth:SIMPLE)
2019-09-22 05:56:37,303 FATAL yarn.YarnUncaughtExceptionHandler - Thread Thread[AsyncDispatcher event handler,5,main] threw an Error. Shutting down now...
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:110)
at java.lang.Thread.run(Thread.java:748)
2019-09-22 05:56:37,308 INFO util.ExitUtil - Halt with status -1 Message: HaltException2019-09-22 05:56:37,308 INFO util.ExitUtil - Halt with status -1 Message: HaltException
I have gone through multiple links, But i didn't get valuable output.
Ouput of max thread: cat /proc/sys/kernel/threads-max 902309
ulimit output: ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 451154 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 128000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 451154 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
Could some help me to resolve the issue?
Regards,
Vinay K
... View more
Labels:
08-08-2019
03:36 PM
Hey Harsh Thanks for responding. As multiple client are requesting data to hbase, at some point, sometimes user don’t get data, EOF exception or connection interruptions occur. We are not able to track the record of requested data and size of input and output data sending to end user. Regards Vinay K
... View more
05-28-2019
10:24 AM
@PJ you can use * in database column while define policy in ranger's hive plugin. * will apply restriction policy for all hive DB.
... View more
05-28-2019
07:41 AM
1 Kudo
Hi @Haijin Li The input file you are using is incorrect stock2.json {"myid":"0001","mytype":"donut"} command is correct CREATE EXTERNAL TABLE stock4_json (myjson struct<myid:string, mytype:string>)ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe' LOCATION '/warehouse/tablespace/external/hive/haijintest.db/stock2'; myjson struct define it is having further list and map. You have to place correct input file stock3.json {"myjson":{"myid":"0001","mytype":"donut"}} and create table with same command and check. Let me know if still you are facing issue. Regards, Vinay K
... View more
05-28-2019
06:49 AM
Hi @PJ It depends on you, how you define policies for hive DB. You may restrict by multiple ranger policies. What is your exactly question can you explain it?
... View more
04-12-2019
07:32 AM
HI Have you find any method?
... View more
04-12-2019
05:34 AM
Hi I'm planning to use hbase, where i've written code in hbase java api and running query. Have there any way to check the executed query in logs as well the response from query in logs?
... View more
Labels:
- Labels:
-
Apache HBase
-
Cloudera Search
01-10-2019
08:47 AM
Almost, we had done. Thanks again @Geoffrey Shelton Okot
... View more