Member since
04-10-2021
22
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1487 | 05-26-2022 03:00 AM |
12-29-2025
01:22 AM
Hi Everyone, We are evaluating whether Apache Hadoop can run inside an Istio service mesh and would like to understand if this is a supported or practical setup. Our environment: Hadoop cluster running on physical machines (bare metal) No Kerberos Istio is mandatory Istio mTLS must be enabled Hadoop services (NameNode, DataNode, YARN, etc.) would need to communicate through Istio Our concerns: Hadoop uses long-lived TCP connections and custom RPC protocols Istio mTLS intercepts and terminates connections Unsure if Hadoop components work reliably behind Envoy proxies Uncertain about the performance and stability impact Questions: Has anyone successfully run Hadoop inside Istio with mTLS enabled? Is this officially supported or just theoretically possible? Are there known limitations or failure cases (HDFS, YARN, shuffle, etc.)? Any real-world experience or guidance would be greatly appreciated. Thanks!
... View more
Labels:
- Labels:
-
Apache YARN
-
HDFS
12-19-2023
03:29 AM
We are not sure, this kind of scenorio we are going to support. If you are cloudera customer, you can check with your account team to engage cloudera Professional Team to enagage and support this feature.
... View more
10-24-2023
09:43 PM
I think you need to verify the yarn and spark resources are configured properly. If yes then go and check from spark ui, it will show driver memory and executor memory. It is coming as expected then safely you can ignore it.
... View more
12-22-2022
05:33 PM
Hi @KPG1 , As per the description I believe you are looking for a way to have a high availability for HMS , if that is the case please find the below doc which can help - https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-cdh/topics/hive-hms-ha-configuration.html And for the backend DB to configure it for HA , it will be out of scope of Cloudera and you have to work with DB team itself to understand how they can configure.
... View more
12-09-2022
05:33 AM
@KPG1 Here is a good post i found that describes the required steps to move postgres to mysql including a php to do the postgres to mysql conversion. https://cloudera.ericlin.me/2016/10/hive-metastore-migration-from-embedded-postgres-to-mysql/ You may need to find a more modern tool for that conversion, but the process to dump, convert, and move metastores should be very similar.
... View more
11-28-2022
12:10 PM
Hello @KPG1 , If you are able to successfully read/write from HDFS and if these messages are appearing only on standby NN logs, then it should be normal. Thank you.
... View more
11-14-2022
03:44 AM
1 Kudo
@KPG1 I can think of Oracle database ha a feature i.e Oracle RAC,where we give SCAN address as the jdbc string to metastore,so the scan ip wll take care of loadbalancing. You need to check with mysql vendor(Oracle),if they have similar thing as of Oracle database. You can have HMS HA but all the HMS instance point to same database, you can not have 2 HMS instance pointing to a different database. Please let me know,if you have any querties and please "Accept As Solution", if your queries are answered
... View more
10-18-2022
01:07 PM
1 Kudo
@KPG1 We only support upgrading an existing cluster using Ambari or Cloudera Manager, instead of importing/updating the jars manually. In latest CDP Private cloud base, and our Public Cloud, we are using Hadoop version 3.1.1 at this point.
... View more
07-18-2022
02:00 AM
1 Kudo
Hello @KPG1 , The time taken to mark a datanode as stale is give by dfs.namenode.stale.datanode.interval, with a default of 30 seconds. If this is happening with a specific Datanode, you can check if there is any network issues between the Datanode and the Namenode or if the Datanode has any JVM pauses reported by checking the Datanode logs. As a bandaid, you can bump up the above parameter till the underlying problem is solved.
... View more
05-31-2022
03:04 AM
Hi @ChethanYM , Thanks for the help, gracefully shutting down the region server triggers the hbase Master to perform a bulk assignment of all regions hosted by that region server. regards, KPG1
... View more