Member since
06-01-2020
3547
Posts
10
Kudos Received
0
Solutions
07-22-2021
12:12 AM
This video describes a way to hide the plain text passwords in Knox topology. The passwords are basically the Bind User passwords when used in ShiroProvider and LDAP/AD authentication.
... View more
04-07-2021
01:29 AM
6 Kudos
The kubectl tool provides direct administrative access to the Kubernetes cluster underlying a CDE service, which is useful for troubleshooting, among other things. This video will demonstrate how to set up kubectl access.
To enable kubectl, we will need a couple of prerequisites. We will need the kubeconfig file from the CDE service. We will need to get and authorize the IAM user, and then we will need to make sure that everything is set up correctly, both for kubectl and some other tools like k9s.
There are a couple of prerequisites for this operation, and we are not going to cover their installation in this video. You will need to follow the process relevant to your platform of choice to get the AWS CLI installed and configured with an IAM user. You will also need to get kubectl installed.
Key steps:
Download the kubeconfig file from the CDE service
Retrieve the ARN for the chosen IAM user
Grant administrative access for the IAM user
Verify kubectl access is successfully authorized
(Optional) Use the kubeconfig file with other tools, such as k9s
You need to download and install aws-iam-authenticator separately. Check this install guide.
... View more
Labels:
02-20-2021
04:16 PM
The issue is very generic and there could be multiple possible reasons for job to be in pending state in yarn. You can follow some steps as listed below to check the problem. 1. Check for the application log of the pending job. yarn logs -applicationId <application id of job> 2. Which application job like hive,hue,oozie etc. For example, If it's a hive job sometimes it may happen the job is submitted to yarn but it may take long time to write metadata to hive metastore. This is where it was stuck. In my case job was stuck for more than an hour while writing metadata to hive metastore.The issue was because some metastore db backup was running during the same time and db was in readonly mode. 3. Enable Debug logging for resource manager logs and collect the log related to your application from resource manager. cat <resource manager log> | grep <stuck job application id> 4. Check what is going on just before the application is stuck. You should probably be able to figure out the cause. Good Luck
... View more
02-01-2021
08:18 PM
4 Kudos
This video provides instructions on how to SSH keys in CDP Public Cloud data hub clusters. It touches on how to generate a new SSH key pair and steps through the process of deploying it for a workload user through the Cloudera Management Console Web UI, as well as using the CDP command-line tool. It discusses the security implications of using the Cloudbreak user for login purposes on data hub hosts and explains why workload user credentials should be used instead in most cases. It also demonstrates using the deployed SSH keys for login to data hub hosts.
... View more
Labels:
12-07-2020
10:17 PM
2 Kudos
In this video, we'll briefly discuss what is Hive locking mechanism, what are the type of locks and how they work. We will also demonstrate with the help of a query about validation on the locks and removal of obsolete Hive locks from backend.
Query used:
CREATE TABLE ABORTED_TXNS AS SELECT A.TXN_ID AS HL_TXNID FROM TXNS A , HIVE_LOCKS B WHERE A.TXN_ID=B.HL_TXNID AND A.TXN_STATE='a'; DELETE FROM HIVE_LOCKS WHERE HL_TXNID IN (SELECT HL_TXNID FROM ABORTED_TXNS);
Reference Link:
https://cwiki.apache.org/confluence/display/Hive/Locking
Open YouTube video here
... View more
Labels: