About jagadeesan

RangaReddy · ‎08-31-2022

Hi @nvelraj Pyspark job working locally because in your local system pandas library is installed, so it is working. When you run in cluster, pandas library/module is not available so you are getting the following error. ModuleNotFoundError: No module named 'pandas' To solve the. issue, you need to install the pandal library/module in all machines or use Virtual environment.

paulo_klein · ‎08-12-2022

To solve "unable to find valid certification path to requested target" I just import the certificate to java and restart the Zeppeling Server. ### LINUX LIST CERT cd /usr/lib/jvm/java-11-openjdk-11.0.15.0.10-2.el8_6.x86_64/bin ./keytool -list -keystore /usr/lib/jvm/java-11-openjdk-11.0.15.0.10-2.el8_6.x86_64/lib/security/cacerts ### LINUX IMPORT CERT ./keytool --import --alias keystore_cloudera --file /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem -keystore /usr/lib/jvm/java-11-openjdk-11.0.15.0.10-2.el8_6.x86_64/lib/security/cacerts

jagadeesan · ‎08-02-2022

@Asim- JDBC also you need HWC for Managed tables. Here is the example for Spark2, but as mentioned earlier Spark3 we don't have any other way to connect Hive ACID tables from Apache Spark other than HWC and it is not yet a supported feature for Spark3.2 / CDS 3.2 in CDP 7.1.7. Marking this thread close, if you have any issues related to external tables kindly start a new Support-Questions thread for better tracking of the issue and documentation. Thanks

VidyaSargur · ‎08-01-2022

@paulo_klein, Have any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

jagadeesan · ‎07-29-2022

Hi @paulo_klein, Apache Spark by default request a delegation token for all 4 services: HDFS, YARN, Hive, and HBase. It is printed as a WARN message but it does no harm. It is captured due to the fact that no HBase jars are in the Spark classpath, hence, it's unable to get the HBase DelegationToken. To fix this issue you start spark-shell or pyspark or spark-submit via --conf spark.security.credentials.hbase.enabled=false Example: # spark-shell --conf spark.security.credentials.hbase.enabled=false

DianaTorres · ‎07-26-2022

@jeyaguna Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks

VidyaSargur · ‎07-17-2022

@Seedy Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

VidyaSargur · ‎07-11-2022

@naymar, Has the reply helped resolve your issue? If so, can you kindly mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?

VidyaSargur · ‎07-10-2022

@mamoune, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

Jessica_cisco · ‎07-05-2022

Hi @jagadeesan, I tried that as well, nothing worked.

Online	Offline
Last Visited	‎04-29-2026 08:47 AM

Member Since	‎11-12-2018 10:00 AM
Last Visited	‎04-29-2026 08:47 AM
Posts	218
Kudos received	179

Cloudera Community

Re: Migrating workloads from Spark 2 to Spark 3

Re: Looking for a supported version of Spark 3 for...

Re: Spark 3 Parcel Compatibility with CDP Private ...

Re: Apache Storm support in Cloudera

Re: Complete example for using spark MLlib for twi...

Re: pyspark toPandas() works locally but fails in ...

Re: CDP - Zeppeling: Spark + Livy + Hive - HWC

Re: Spark3 connection to HIVE ACID Tables

Re: CDP - Zeppelin - Livy Error

Re: CDP - spark-shell --master yarn : security.HBa...

Re: Upgrade RHEL 7.7 to 7.9

Re: org.apache.atlas.AtlasBaseClient lines in nifi...

Re: {msg":"User 'UserX' not allowed to impersonate...

Re: 2 concurrent data sources types for the same c...

Re: Guava jar giving runtime error while deploymen...