Cloudera Community
DianaTorres
user rank
user rank
Community Manager

Re: ERROR SparkContext: Error initializing SparkCo...

by Community Manager Community Manager in Support Questions
‎05-03-2023 10:39 AM
‎05-03-2023 10:39 AM
@Abhay_Kumar Welcome to the Cloudera Community! To help you get the best possible solution, I have tagged our Spark experts @Gopinath and @smdas  who may be able to assist you further. Please keep us updated on your post, and we hope you find a satisfactory solution to your query. ... View more

Re: NiFi ListS3 Maximum Object Age

by Community Manager Community Manager in Support Questions
‎05-02-2023 10:29 AM
‎05-02-2023 10:29 AM
@acasta Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more

Re: Looking to encrypt data while copying data int...

by Community Manager Community Manager in Support Questions
‎05-02-2023 10:20 AM
‎05-02-2023 10:20 AM
@Vas_R Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more

Re: NiFi using memory more than allocated

by Community Manager Community Manager in Support Questions
‎05-02-2023 10:06 AM
‎05-02-2023 10:06 AM
@Amit_barnwal Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more

Re: How do I remove the nulls with this jolt?

by Community Manager Community Manager in Support Questions
‎04-24-2023 09:37 AM
‎04-24-2023 09:37 AM
@wolfsilver00 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  ... View more

Re: Unable to connect remote Hadoop cluster using ...

by Explorer in Support Questions
‎04-24-2023 07:25 AM
‎04-24-2023 07:25 AM
Sorry for late response, I use oozie to submit a spark job ... View more

Re: how can i copy hbase data from replication set...

by New Contributor in Support Questions
‎04-21-2023 03:00 AM
‎04-21-2023 03:00 AM
i fix this issue by run command on destination cluster,i think it caused by original version is too old to support ec (hadoop-2.7.5) ... View more

Re: Hive metastore lost connection while executing...

by Frequent Visitor in Support Questions
‎04-20-2023 01:10 AM
‎04-20-2023 01:10 AM
Thank you @mszurap for you response . I tried the suggested work around already and it seems like the issue still persists .  I agree the table has lot of partitions but I am pretty sure the code times out before 5 mins .  I have also tried enforcing the hive-site.xml with the updated timeout which also did not help much.  Only thing which worked was adding spark.catalog.recoverPartitions(table) before issuing the drop partition command .  I am really not sure as why recovering the partitions in the catalog eliminated the metastore warning .  Below is the updated code which is working without any warning : spark.sql.catalog.recoverPartitions(orders) spark.sql("alter table orders drop if exists partition(year=2023)") data.write.mode('Overwrite').parquet(hdfsPath) Any help here in understanding the problem will be much appreciated . ... View more

Re: Oozie Hive Job status changes to KILLED but th...

by Guru Guru in Support Questions
‎04-19-2023 12:22 AM
‎04-19-2023 12:22 AM
Glad to hear that. 🙂 The "hive.stats.autogather" should be unrelated to this, it just controls whether Hive should gather statistics at the end of INSERT statements, disabling it just speeds up the insert query somewhat. If you see something weird or not working with that enabled, feel free to open a support case on our Support Portal. See this and all the hive configuration descriptions in the HiveConf.java source code. Cheers,  Miklos ... View more

April 2023 Community Highlights

by Community Manager Community Manager in Community Announcements
‎04-16-2023 10:05 PM
‎04-16-2023 10:05 PM
Here are some highlights from the month of March   COMING NEXT! Streaming Data Pipeline Development   Check out the FY24 Cloudera Meetup Events Calendar for upcoming & past event details!     198 new support questions 2 new community articles 592 new members Rank  Community Article Author Components/ Labels #1 Processing files within the CML EFS (better I/O performance) - Edge Case with non-tabular data Ryan Cicak  @RyanCicak   Cloudera Machine Learning (CML)   We would like to recognize the below community members and employees for their efforts over the last month to provide community solutions.    See all our top participants at Top Solution Authors leaderboard and all the other leaderboards on our Leaderboards and Badges page.     @MattWho @smdas @steven-matison @mszurap @ckumar  @cotopaul @SAMSAL @davehkd @BrianChan @jorgeoa    Share your expertise and answer some of the below open questions. Also, be sure to bookmark the unanswered question page to find additional open questions.    Unanswered Community Post  Components/ Labels Hortonworks maven repo broken? Hortonworks Data Platform (HDP) Python updates Cloudera Data Platform (CDP) Nifi - Mongo X509 auth Apache NiFi ATLAS-409-00-002: Given type xxx has references Apache Atlas value cannot be converted to a timestamp Apache NiFI NiFi Registry   ... View more
Contact Me
Online
Offline
Last Visited
‎12-02-2025 11:05 AM
Community Statistics
Member Since ‎11-17-2021 08:08 AM
Last Visited ‎12-02-2025 11:05 AM
Posts 1,117
Kudos received 233