Nik94
Explorer

access hbase tables using rest api

by Explorer in Support Questions
‎07-18-2019 10:08 AM
‎07-18-2019 10:08 AM
developer needs acess to hbase tables using rest api. which things need to do on ambari cluster to access hbase tables using rest api. ... View more
Labels:

Delete host which is down permenantally

by Explorer in Support Questions
‎07-07-2019 06:33 AM
‎07-07-2019 06:33 AM
In hdp cluster one of the host is down permanently. What are the prerequisites to remove that host from cluster. ... View more

spark config changes

by Explorer in Support Questions
‎07-05-2019 06:15 AM
‎07-05-2019 06:15 AM
can we pass config parameter while running spark query instead of doing changes from ambari. how to do? ... View more
Labels:

Regarding hdfs missing block

by Explorer in Support Questions
‎06-20-2019 06:57 AM
‎06-20-2019 06:57 AM
in ambari there is alert regarding 1 missing block from 6 days. but when i check using " hdfs fsck / " its showing there is not any missing block. is this bug in ambari ? ... View more
Labels:

need to control spark2-history size

by Explorer in Support Questions
‎06-12-2019 06:09 AM
‎06-12-2019 06:09 AM
in spark2-history ,long running application generating logs with 30GB+ size. how to control spark2-history size for each application . ... View more
Labels:

ambari-metrics-collector not showing any data on d...

by Explorer in Support Questions
‎06-10-2019 08:05 AM
‎06-10-2019 08:05 AM
ambari-metrics collector running but in log its showing that no live collector to collect data. hbase-ams-master.log: No live collector to send metrics to. Metrics to be sent will be discarded. This message will be skipped for the next 20 times. ambari-metrics-collector.log: 2019-06-10 13:29:52,653 INFO org.apache.hadoop.hbase.client.RpcRetryingCaller: Call exception, tries=15, retries=35, started=629189 ms ago, cancelled=false, msg=java.io.IOException: Table Namespace Manager not fully initialized, try again later at org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:2753) at org.apache.hadoop.hbase.master.HMaster.ensureNamespaceExists(HMaster.java:2976) at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1746) at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:491) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:59956) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167) ambari alert: metrics-collector process alert connection refused to ambari-metrics collector:6188 ... View more
Labels:

need to compress & rotate hdfs audit log

by Explorer in Support Questions
‎06-06-2019 09:45 AM
‎06-06-2019 09:45 AM
i did logrotate using linux,but i am not getting where current logs are getting dumped.in that todays logs are also in compressed format till 3:00 am.after that there are no logs. everyday hdfs audit logs generated aroun 7 gb . I need keep 7 days log and compress logs older than a day. which config changes i need to do. ... View more
Labels:

oozie not starting,giving below alert

by Explorer in Support Questions
‎06-04-2019 06:36 AM
‎06-04-2019 06:36 AM
while starting oozie service getting below error: oozie server status:Execution of 'source /usr/hdp/current/oozie-server/conf/oozie-env.sh ; oozie admin -oozie http://master1:11000/oozie -status' returned 255. Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. Trying after 1 sec. Retry count = 1 Connection exception has occurred [ java.net.ConnectException Connection refused (Connection refused) ]. oozie server web ui:Connection failed to master1:11000/oozie/?user.name=oozie (<urlopen error [Errno 111] Connection refused>) oozie log: Jun 03, 2019 8:34:40 PM org.apache.coyote.http11.Http11Protocol pause INFO: Pausing Coyote HTTP/1.1 on http-11000 Jun 03, 2019 8:34:41 PM org.apache.catalina.core.StandardService stop INFO: Stopping service Catalina Jun 03, 2019 8:34:41 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc SEVERE: The web application [/oozie] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Jun 03, 2019 8:34:41 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/oozie] appears to have started a thread named [FileWatchdog] but has failed to stop it. This is very likely to create a memory leak. ... View more
Labels:

Re: Could not find any valid local directory for o...

by Explorer in Support Questions
‎05-03-2019 03:51 PM
‎05-03-2019 03:51 PM
i am also getting this error.we are processing 500 gb data and nodemanager local-dir size is 100 gb.at the time of job execution it was 91 % full.1100 mappers and 1000 reducers are there .mapper job completed ,some reducer job failed and killed.Please help me to solve that. error will be as: Status: Failed Vertex re-running, vertexName=Map 1, vertexId=vertex_1556753562511_0049_1_00 Vertex failed, vertexName=Reducer 2, vertexId=vertex_1556753562511_0049_1_01, diagnostics=[Task failed, taskId=task_1556753562511_0049_1_01_000470, diagnostics=[TaskAttempt 0 failed, info=[Error: exceptionThrown=org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$ShuffleError: error in shuffle in DiskToDiskMerger [Map_1] at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:357) at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:334) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for attempt_1556753562511_0049_1_01_000470_0_10014_src_1187_spill_-1 at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:441) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:151) at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:132) at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager$OnDiskMerger.merge(MergeManager.java:841) at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeThread.run(MergeThread.java:89) , errorMessage=Shuffle Runner Failed:org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$ShuffleError: error in shuffle in DiskToDiskMerger [Map_1] at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:357) at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle$RunShuffleCallable.callInternal(Shuffle.java:334) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ... View more

ambari metrics collector getting down

by Explorer in Support Questions
‎04-27-2019 03:25 PM
‎04-27-2019 03:25 PM
ambari metrics collector getting down frequentally with error: WARN org.apache.hadoop.hbase.io.util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.size ... View more
Labels:

Re: How to rotate and archive hdfs-audit log file

by Explorer in Support Questions
‎04-16-2019 03:54 PM
‎04-16-2019 03:54 PM
i did following changes,but hdfs-audit logs are not rotating, hdfs.audit.logger=INFO,console log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger} log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false #log4j.appender.DRFAAUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.DRFAAUDIT=org.apache.log4j.RollingFileAppender log4j.appender.DRFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log log4j.appender.DRFAAUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.DRFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n log4j.appender.DRFAAUDIT.DatePattern=.yyyy-MM-dd log4j.appender.DRFAAUDIT.MaxFileSize=100MB log4j.appender.DRFAAUDIT.MaxBackupIndex=5 ... View more

Ambari-metrics-collector getting down

by Explorer in Support Questions
‎04-12-2019 11:04 AM
‎04-12-2019 11:04 AM
Ambari metricss collector getting down with following error and we are not using hbase: WARN org.apache.hadoop.hbase.io.util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.size ... View more
Labels:

jobs failing on particular Nodemanager

by Explorer in Support Questions
‎04-10-2019 06:54 AM
‎04-10-2019 06:54 AM
on one of the datanode jobs are failing .stopping nodemanager is helpful or not. Please help. ... View more
Labels:

at the time of spark job submission getting error

by Explorer in Support Questions
‎04-04-2019 01:21 PM
‎04-04-2019 01:21 PM
Getting this error at the time of job submission,please help me regarding whats the issue: 19/04/02 21:39:09 WARN DFSClient: DataStreamer Exception java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[DatanodeInfoWithStorage[10.0.1.7:50010,DS-a5960a3c-6e53-45a7-a692-5374b4d0646b,DISK], DatanodeInfoWithStorage[10.0.1.6:50010,DS-0c1ff599-7179-4566-baea-e26249589633,DISK]], original=[DatanodeInfoWithStorage[10.0.1.7:50010,DS-a5960a3c-6e53-45a7-a692-5374b4d0646b,DISK], DatanodeInfoWithStorage[10.0.1.6:50010,DS-0c1ff599-7179-4566-baea-e26249589633,DISK]]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration. at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1059) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1122) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1280) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1005) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:512) 19/04/02 21:39:22 INFO RetryInvocationHandler: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category WRITE is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:87) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:2006) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1404) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renewLease(FSNamesystem.java:4785) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.renewLease(NameNodeRpcServer.java:1117) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.renewLease(ClientNamenodeProtocolServerSideTranslatorPB.java:675) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) , while invoking ClientNamenodeProtocolTranslatorPB.renewLease over 10.0.1.5:8020. Trying to failover immediately. 19/04/02 21:39:22 INFO RetryInvocationHandler: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category WRITE is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:87) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:2006) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1404) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renewLease(FSNamesystem.java:4785) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.renewLease(NameNodeRpcServer.java:1117) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.renewLease(ClientNamenodeProtocolServerSideTranslatorPB.java:675) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) …while invoking ClientNamenodeProtocolTranslatorPB.renewLease over 10.0.1.17:8020 after 1 failover ... View more
Labels:

how to improve performance of copyfromlocal

by Explorer in Support Questions
‎04-03-2019 04:46 PM
‎04-03-2019 04:46 PM
We have to load 1 month data from local file system to hdfs. to load 1 days data its taking 30 min,to load 1 month data its taking 15 hrs. so how to improve speed of loading data from local file system to hdfs. ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎12-18-2019 09:19 AM
Public Statistics
Date Registered ‎04-19-2018 02:30 AM
Last Visited ‎12-18-2019 09:19 AM
Total Messages Posted 21