Created on 07-22-2022 11:51 AM - edited 07-22-2022 11:54 AM
I have signed up for the 60 CDP trial and have finally managed to register a new environment based on my AWS account. Now I want to do this tutorial: https://www.cloudera.com/tutorials/cdp-getting-started-with-cloudera-data-engineering.html
I have enabled a CDE Service and I can see that there is 1 cluster ("Default-Virtual-Cluster").
However, when I click on the "View Jobs"-icon right next to the cluster, the page does not load. The same happens when I want to open the Airflow UI or the Grafana charts:
I also noticed that the statistics under "Charts" do not load:
The logs dont indicate any problems though:
Do you have any hint what I have to change or where I could start debugging?
Thanks a lot in advance!
I just noticed that the logs of the CDE service contain the following warning:
Might this be the reason for my issue?
Since you're using an environment only deployed for a 60-Day trial, I don't think it will hurt anything to try manually initiating the user sync as a first attempt at a resolution. You can review the procedure for doing that here:
Hi Bill, thanks for your reply. Unfortunately, syncing the users did not help. However, I generated a Diagnostics bundle and when I check the "Clod Diagnostics" it tells me:
"Message": "error getting metrics diagnostics data for cluster: cluster-XXXXX, err: AccessDenied: User: arn:aws:sts::XXXXXXXXXX:assumed-role/ClouderaUser/XXXXXXXXXXXXXX is not authorized to perform: cloudwatch:GetMetricData because no identity-based policy allows the cloudwatch:GetMetricData action\n\tstatus code: 403, request id: 332d3b99-f102-475b-9466-ba24c6ef217b"
(I inserted the XXX so don't be surprised).
This ClouderaUser mentioned in the logs is the AWS role I created when registering the environment. I used the default values for that:
Do I need to grant this role more rights so it can also access the CDE features or am I supposed to use any other role for that (e.g. the DATALAKE_ADMIN_ROLE)?
@Ploeplse For some reason that is a mystery to me, they were being miscategorized as spam. If I were to guess, it's the partial page shots, but again I don't know. I would try cutting and pasting the text of the error messages into your posts as opposed to the page shots as images.
One of the other moderators might weigh in here and tell me why I am wrong. In any case, I took action on it; your reply should now be visible in the thread.
@Ploeplse Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks
no I still couldnt solve the issue. I tried to create another CDE with one of the default environments from the trial account, but in this case the CDE is not even getting initiated. Error says:
Cluster provisioning failed: error while creating instance groups, check failed events, info: 4 errors occurred: * error creating worker nodes stack, cause: [error creating aws stack] info: ValidationError: AccessDenied. User doesn't have permission to call ssm:........
Hi @Ploeplse , what you ran into was not a user problem. But you may ran into user problem after you fix this one.
The error "the site can't be reached" is actually a network connectivity issue instead of IAM issue. I ran into this issue while I'm using an Azure environment. I made an `nslookup` against the FQDN in the url, and turned out it came back with a private IP address in the k8s cluster node subnet, which means you actually need connectivity from your browser to the K8s network that facilitating the environment.
In my case, my browser is on a VM on Azure residing in another VNET, so what I did was to peer the network between my browser VM and the K8s VNET.
If you are using AWS, you also need build connectivity between the AWS PVC to your browser operating system. If it is your laptop, you need VPN setup; if it is a VM, you need peer the network. Cloudera documentation missed this step in setup or it is hidden somewhere.
While, after I fixed the network issue, I ran into 403 issue, which is actually an IAM issue. Still working on it. Good luck.