Cloudera Community
KPG1
user rank
Contributor
My Accepted Solutions
Show All

Re: Spark master shuts down when one of zookeeper ...

by Contributor in Support Questions
‎12-11-2023 02:24 AM
‎12-11-2023 02:24 AM
yes it's Apache spark 3.4.1  ... View more

Re: Spark master shuts down when one of zookeeper ...

by Contributor in Support Questions
‎11-06-2023 09:09 PM
‎11-06-2023 09:09 PM
It is the same as :-https://issues.apache.org/jira/browse/SPARK-15544 I see the ticket is Resolved but no solution is present. ... View more

Spark master shuts down when one of zookeeper dies

by Contributor in Support Questions
‎11-06-2023 04:52 AM
‎11-06-2023 04:52 AM
I am using spark-3.4.1 I have a setup with three ZooKeeper servers, Spark master shuts down when a zookeeper instance is down a new master is elected as leader and the cluster is up. But the original master that was down never comes up. can anyone help me with this issue? ... View more

Getting Warning: Maximum heap size rounded up to 5...

by Contributor in Support Questions
‎01-24-2023 09:56 AM
‎01-24-2023 09:56 AM
Even after passing the executor memory and driver memory as 1 Gb still it is getting maximum heap size rounded to 512 MB   spark-shell --conf spark.executor.memory=1g --conf spark.driver.memory=1g Warning: Maximum heap size rounded up to 512 MB   Anything that is missing? ... View more

Configure Postgress HA database for Hive Metastore

by Contributor in Support Questions
‎12-12-2022 10:22 PM
‎12-12-2022 10:22 PM
Hi All, I am using  Postgress for Hive metastore How can we configure postgress HA with Hive with true HA on the Hive Metastore without changing the connection string manually? ... View more

Migrating Hive database from Postgress to Mysql

by Contributor in Support Questions
‎12-08-2022 01:11 AM
‎12-08-2022 01:11 AM
Hi All We have our hive database on Postgress and wanted to migrate it to MySQL, Is there any easy way to do it?   Thanks ... View more

Getting Info message while reading and writing mes...

by Contributor in Support Questions
‎11-27-2022 09:51 PM
‎11-27-2022 09:51 PM
I am getting info messages like the one below while reading and writing files in hdfs. can anyone explain the meaning of this message, will it cause any issues? I have 3 Namenode(open-source Hadoop) in HA.   INFO retry.RetryInvocationHandler: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby. Visit https://s.apache.org/sbnn-error at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:108) at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:2094) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1550) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3342) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1208) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:1042) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976) , while invoking ClientNamenodeProtocolTranslatorPB.getFileInfo over hostname/ip:8020 after 1 failover attempts. Trying to failover after sleeping for 1195ms.   Thanks   ... View more
Labels:

Configure HA database for Hive Metastore

by Contributor in Support Questions
‎11-13-2022 10:35 PM
‎11-13-2022 10:35 PM
We have an MYSQL HA cluster, In case of failover, for the secondary master to take the place of the failed master, the hive metastore connection string configuration would need to be changed to point to the new master, and the services restarted. is it possible to have true HA on the Hive Metastore without changing the connection string manually ?   ... View more

Hadoop Migration from Hortonworks HDP3 to Apache h...

by Contributor in Support Questions
‎10-04-2022 05:11 AM
‎10-04-2022 05:11 AM
Is there any other way to migrate the Hortonworks HDP3  cluster to the Apache Hadoop 3.3.2  other than the distcp? is it possible to upgrade by updating the jar and pointing the older metadata to the newer stack of Hadoop?   Thanks ... View more

frequently getting stale alerts for Datanodes.

by Contributor in Support Questions
‎07-17-2022 09:48 PM
‎07-17-2022 09:48 PM
Hi everyone. I am getting stale alerts every hr, if the last point of contact between Data nodes and Namenode is more than 30 s we get these alerts. I am not able to find the root cause of this slowness, I have 32 cores system, but when this alert is generated in htop hdfs usage is more but not all cores are 100% utilized. DataNode Health Summary DataNode Health: [Live=5, Stale=1, Dead=0] Please suggest changes required to resolve this. ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎11-04-2024 12:11 AM
Community Statistics
Member Since ‎04-10-2021 04:45 AM
Last Visited ‎11-04-2024 12:11 AM
Posts 21
Kudos received 1