Member since
07-20-2022
9
Posts
0
Kudos Received
0
Solutions
09-13-2022
02:25 AM
Dear all, I am trying to load a PySpark Dataframe into an Impala table, using the jdbc connector. However, the df.write statement fails, because the "Create table" - statement that is generated contains quotation marks for the column names: Do you have any idea how to get rid of these quotation marks? If not, what would be a different approach to load a dataframe into an Impala table? I also tried spark.sql('select identifier_id as identifier from tempView').write.jdbc(...), but here I am getting the error "File /tmp/hive does not exist". Thanks a lot in advance for any help!
... View more
Labels:
- Labels:
-
Apache Impala
-
Apache Spark
08-02-2022
12:27 AM
Dear Diana, no I still couldnt solve the issue. I tried to create another CDE with one of the default environments from the trial account, but in this case the CDE is not even getting initiated. Error says: Cluster provisioning failed: error while creating instance groups, check failed events, info: 4 errors occurred: * error creating worker nodes stack, cause: [error creating aws stack] info: ValidationError: AccessDenied. User doesn't have permission to call ssm:........
... View more
07-23-2022
07:38 AM
Hi Bill, thanks for your reply. Unfortunately, syncing the users did not help. However, I generated a Diagnostics bundle and when I check the "Clod Diagnostics" it tells me: "Message": "error getting metrics diagnostics data for cluster: cluster-XXXXX, err: AccessDenied: User: arn:aws:sts::XXXXXXXXXX:assumed-role/ClouderaUser/XXXXXXXXXXXXXX is not authorized to perform: cloudwatch:GetMetricData because no identity-based policy allows the cloudwatch:GetMetricData action\n\tstatus code: 403, request id: 332d3b99-f102-475b-9466-ba24c6ef217b" (I inserted the XXX so don't be surprised). This ClouderaUser mentioned in the logs is the AWS role I created when registering the environment. I used the default values for that: Do I need to grant this role more rights so it can also access the CDE features or am I supposed to use any other role for that (e.g. the DATALAKE_ADMIN_ROLE)? Best, Moritz
... View more
07-22-2022
11:51 AM
Hi everyone, I have signed up for the 60 CDP trial and have finally managed to register a new environment based on my AWS account. Now I want to do this tutorial: https://www.cloudera.com/tutorials/cdp-getting-started-with-cloudera-data-engineering.html I have enabled a CDE Service and I can see that there is 1 cluster ("Default-Virtual-Cluster"). However, when I click on the "View Jobs"-icon right next to the cluster, the page does not load. The same happens when I want to open the Airflow UI or the Grafana charts: I also noticed that the statistics under "Charts" do not load: The logs dont indicate any problems though: Do you have any hint what I have to change or where I could start debugging? Thanks a lot in advance! Edit: I just noticed that the logs of the CDE service contain the following warning: Might this be the reason for my issue?
... View more
07-21-2022
01:38 AM
According to the docs, the support needs to assign me admin rights: The CDP account administrator user account cannot be managed within CDP. You must contact Cloudera support to add or remove an account administrator from your CDP account. Stated here: https://docs.cloudera.com/management-console/cloud/user-management/topics/mc-cdp-account-administrator.html
... View more
07-20-2022
10:54 AM
Hi everyone, I have signed up for the CDP 60-days trial verison because I will work with Cloudera on the job soon and therefore I would like to get familiar with it in advance and do some of these tutorials: https://www.cloudera.com/tutorials.html#?fq= When I go to Cloudera Data Platform I dont see any environments and I cant create/register news ones, because I am not the admin. During the registration process, there was no way to choose any specific user rights. It seems I have confused a few things, as I thought the Public Cloud trial would be some kind of sandbox where I could try out all kind of stuff like connecting to my AWS databases etc., but I don't see how this works if I lack the admin rights. Can anyone tell me if I can create an admin account by myself or if there is any other way to get free temporary access to some sandbox-like environment? Thanks a lot in advance!
... View more
Labels: