Re: HDP 3.0.1 is not working

Ayush · ‎02-02-2019

@Deepak Malviya Let us know if you issue solved?, you can also check local system firewall status by using below command. systemctl status firewall.services If it shows "running" then stop that using below command. systemctl stop firewall.services

Ayush · ‎02-01-2019

Hi Dears, After rebooting the HDP physical server hive metastore and hive server2 is starting but after couple of seconds throwing error "DB account locked" & going down. HDP Version: 2.6.3 Ambari version: 2.6.0 Data base: Oracle 12c Any help will be highly appreciated.

Ayush · ‎01-30-2019

Hi @Olufemi Babalola Let me know if you seeing any corrupted block in the cluster? also Hbase region server is running fine? It would be better if you share hbase-master & Hbase-region server logs. Regards Ayush

Ayush · ‎01-30-2019

Hi @Niraj Saparia, You need to first understand how site to site tunnel configured in your network for accessing the AWS instances .The IP which you are using to connect the AWS instance from your LAN that only you can use in host file while configuring the cluster. Let me know if I clear your doubt. Regards Ayush

Ayush · ‎01-29-2019

Hi Neha, As per the log it shows that 'emp _data' table doesn't exist. ""2019-01-29 17:35:41,252 ERROR [main] mapreduce.ImportTsv: Table 'emp_data' does not exist."" But still Can you provide below info: 1. Version of Ambari 2. All the components of HBase and Zookeeper are running without any error? 3. Is this a kerberos enabled cluster?

Ayush · ‎01-29-2019

Dears, I am getting below error while running "list" command in hbase shell. ""ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet"" Below is the complete error, ""hbase(main):007:0> list TABLE ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2725) at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:974) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:59928) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167) Here is some help for this command: List all tables in hbase. Optional regular expression parameter could be used to filter the output. Examples: hbase> list hbase> list 'abc.*' hbase> list 'ns:abc.*' hbase> list 'ns:.*' hbase(main):008:0> Note: All the service & component in Ambari is up & running without any alert. HDP Version: 2.6.3.0 Ambari version: 2.6.2.0 Any help will be highly appreciated Regards Ayush

Ayush · ‎11-14-2018

Hi Anarasimham, Thank you for your reply but I am getting below error while running the command. [root@azu130218r72-1 ~]# hdfs dfs -chown -R kafka:hdfs /user/kafka 18/11/14 10:42:28 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] chown: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "azu130218r72-1/172.175.0.59"; destination host is: "azu130218r72-1":8020; [root@azu130218r72-1 ~]# Thanks in advance. Ayush

Ayush · ‎11-13-2018

Hi, Have installed HDP 2.6.5 with ambari 2.7.1 but after that not able to start hive CLI. Please let me know what is wrong here. below is the error logs. any help would be highly appreciated. "[root@azu130218r72-1 resource]# hive log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender. Logging initialized using configuration in file:/etc/hive/2.6.5.0-292/0/hive-log4j.properties Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=kafka, access=WRITE, inode="/user/kafka":hdfs:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:325) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:246) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:277) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1917) at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4185) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1109) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:645) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=kafka, access=WRITE, inode="/user/kafka":hdfs:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:325) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:246) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:277) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1917) at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4185) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1109) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:645) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73) at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3101) at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:3069) at org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1181) at org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1177) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1195) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1169) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1925) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.getDefaultDestDir(DagUtils.java:787) at org.apache.hadoop.hive.ql.exec.tez.DagUtils.getHiveJarDirectory(DagUtils.java:897) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.createJarLocalResource(TezSessionState.java:366) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:161) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:116) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:579) ... 8 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=kafka, access=WRITE, inode="/user/kafka":hdfs:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:325) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:246) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:277) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1917) at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4185) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1109) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:645) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554) at org.apache.hadoop.ipc.Client.call(Client.java:1498) at org.apache.hadoop.ipc.Client.call(Client.java:1398) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) at com.sun.proxy.$Proxy12.mkdirs(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:610) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:290) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:202) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:184) at com.sun.proxy.$Proxy13.mkdirs(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3099) ... 21 more"

Ayush · ‎11-13-2018

I am also getting the same error

Ayush · ‎11-13-2018

I am also getting the same problem, any help would be highly appreciated

Ayush · ‎11-13-2018

Thanks a lot Sandeep for your response. I communicated the same to user.

Ayush · ‎10-13-2018

Hi Harsh, Thanks alot for your support. Really appreciate. I was able to make hbase stable by adding the line mentioned by you but the only one change was reuiqred. -Dzookeeper.skipACL=yes we need to give "yes" not true. It worked for me. Thanks for making my cluster happpy. Regards Ayush

Ayush · ‎10-13-2018

Hi jmspaggi, Thanks alot for your reply. I was able to resolve the issue by reinstalling Hbase alone. Regards Ayush

Ayush · ‎10-12-2018

I have installed HDP 2.6.5_292 and wanted to add Spark2 2.3.1 service but it is not available. I can see only spark 2.3.0. Can you please help me know if by any other way we can add spark2 2.3.1 in HDP 2.6.5_292. HDP verison: 2.6.5_292 Ambari Version: 2.7.1.0 Anyone's help will be highly appreciated. Regards Ayush

Ayush · ‎09-26-2018

Hi, Can anyone help me with this, getting this error and HBase master is getting after every successfull restart. Could not obtain block: BP-892504517-172.31.16.10-1537889422648:blk_1073741826_1002 file=/hbase/hbase.version No live nodes contain current block Block locations: Dead nodes: . Throwing a BlockMissingException at org.apache.hadoop.hdfs.DFSInputStream.refetchLocations(DFSInputStream.java:1040) at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1023) at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1002) at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:642) at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:895) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:954) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:201) at org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:606) at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:689) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:500) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:169) at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:144) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:704) at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:194) at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1834) at java.lang.Thread.run(Thread.java:748)

Online	Offline
Last Visited	‎09-04-2020 02:31 AM

Date Registered	‎09-26-2018 05:09 AM
Last Visited	‎09-04-2020 02:31 AM
Total Messages Posted	16
Total Kudos Received	1

Re: HDP 3.0.1 is not working

After rebooting the HDP physical server hive metas...

Re: ERROR: Can't get master address from ZooKeeper...

Re: Best Practice Architecture in AWS Cloud for Ma...

Re: Hbase shell is giving error - ""ERROR: org.ap...

Hbase shell is giving error - ""ERROR: org.apache...

Re: not able to start hive CLI after the fresh ins...

not able to start hive CLI after the fresh instill...

Re: java.lang.reflect.InvocationTargetException: :...

Re: log4j:WARN No such property [maxFileSize] in o...

Re: I have installed HDP 2.6.5_292 and wanted to a...

Re: how to remove a node in zookeeper, forcibly ?

Re: HBase Master: Failed to become active master

I have installed HDP 2.6.5_292 and wanted to add S...

Re: HBase Master: Failed to become active master