Unanswered questions

HiI've got a CDE 1.22.0-h1-b3 cluster that fails to load the Spark UI after the job has completed. I can see the Spark UI during the running of a job but it fails to load/open once the job has completed.I see an http GET to the /jobs endpoint fails with a 404 Not Found.Have you come across this error and resolved it please?CheersCraig

Hi all,I'm setting up a CML session with 64GB of RAM and 4 CPUs, then I set up a PySpark session with these configurationsspark = SparkSession.builder \.appName("OptimizedSparkSession") \.master("local[*]") \.config("spark.sql.execution.arrow.pyspark.enabled", "false") \.config("spark.sql.shuffle.partitions", "400") \.config("spark.executor.memory", "32g") \.config('spark.executor.instance', '10') \.config("spark.executor.cores", "4") \.config("spark.driver.memory", "32g") \.config("spark.network.timeout", "800s") \.config("spark.executor.heartbeatInterval", "200s") \.config("spark.driver.maxResultSize", "64g") \.getOrCreate()The transformations I do are quite simple. Basically loading two tables, one with entities of interest and another containing the additional features and other entities, so I join the second one to the first to keep only the records that are needed.table1 = spark.sql(f"""SELECT DISTINCT key1, key2FROM db1.table1""")table2 = spark.sql(f"""SELECT key1, key2, col1, c

Is there a roadmap or plan whether and if yes, when the Cloudera will support dbt v2? Even a rough timeframe would be very helpful.

Dear support, community, As you all may have noticed there has been a lot of security issues and hacks being done these last few months. Exploiting vulnerabilities has never been so easy with the use of AI. Just last. week eMudhra CA certificates have been compromised. How does the apache nifi community cope with these vulnerabilities? Is there a policy for this? of a site where we can get patches or al least follow mitigation actions?  Thank you in advanced.Kind regards

Hi everyone,We’re setting up a few new worker nodes for our cluster and I’m stuck on memory. Our jobs are mostly Spark and Hive, and sometimes they run out of memory when a lot of them start at once.The servers we picked take  and I’m trying to decide how much to put in each node. Is it better to fill all the slots now, or start smaller and add more later?How much memory per worker node do you run, and has it been enough? I’d love to hear what worked for you.Thanks!

Can Impala and Kudu be sold as commercial products?A Chinese  company, Sensors Data, uses them as part of their commercial product.Is this okay?

Hello,I have MsSql DB and Oracle DB. I am gonna try to explain what I want to do. I am fetching data from MsSQL DB than I am inserting that rows to Oracle DB but this structure not dynamic. Because I created TEST table on both side. So I am writing to table name of processor configuration. I want to create dynamic structure so when I start to flow, I need to migrate all database tables with all rows. But at the same time all tables need to create by processor automatically and when new rows came to MsSQL DB, the flow need to insert only new rows. Any one knows how to do that on Nifi 2.0.0-M3 ?

Hello when I try to View Data Provenance of a processor then it shows multiple rows but then when I try to view its content then I get "Content is no longer available in the content repository"though the size in KB of the input and output claims are there. Also surprisingly the new entries content I can see but the same thing does not show the next day.