Member since
12-06-2022
29
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1365 | 06-08-2023 11:41 PM |
05-17-2023
07:57 PM
Turn out there will be two icons if you need to "redeploy client conf" If the blue icon below appears, means you have to tick the "redeploy client conf" button to restart the whole cluster If only this orange icon appears, mean you don't need to do that
... View more
05-17-2023
07:27 PM
I'm following guide on install Kerberos on Cloudera Cluster, thing went fine. There is a step that setup a superuser in the end as this link I changed the config in HDFS conf as the guide said, added a kerberos principal with the name "vega". But when I executed commands like: kinit -k -t /opt/kerberos/vega.keytab vega@BI.VEGA.COM (get tgt for "superuser" vega) hadoop fs -chmod -R 771 /user/hive/warehouse I still got permission error with user "vega", which is a "superuser" chmod: Permission denied: user=vega, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x If I add "sudo - u hdfs" before the command like some guides on the internet said, I got kerberos error: 23/05/18 09:11:57 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] chmod: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "data-node01.vega.com/172.25.0.103"; destination host is: "data-node01.vega.com":8020; I thought that "superuser" means that I can do anything. Even the Cloudera guide said superuser is the replication for "sudo -u hdfs" command when Kerberos is enabled. Why I still get bunch of permission/kerberos error after setup "superuser" When you enabled Kerberos for the HDFS service, you lost access to the default HDFS superuser account using sudo -u hdfs commands. Cloudera recommends you use a different user account as the superuser, not the default hdfs account
... View more
Labels:
- Labels:
-
HDFS
05-17-2023
03:39 AM
I encountered the same error. After a few days of trying I decided to give up. Just go to Hbase config on Cloudera Manager UI and turn "hbase.security.authentication" to "simple" and "hbase.thrift.security.qop" to "none"
... View more
05-17-2023
12:49 AM
When I make some changes to some service configurations, Cloudera asks me to "restart stale service". And there is a window pop-up asking me should I "Redeploy Client Configuration" too like below: The default option is No (not tick), so does this mean this is optional? When should I tick this? Since if I tick this, it will take a long time to restart everything, even I only make a small change. I wonder if is it necessary.
... View more
Labels:
- Labels:
-
Cloudera Manager
05-15-2023
12:42 AM
Where is the "host Resource tab" on Cloudera Manager Web UI?
... View more
05-14-2023
09:48 PM
I have a cluster of 15 nodes and I'm a bit headache when try to assign nodes. I'm following this Role distribution instruction on Cloudera. But I wonder can I just add a Gateway role to every node for some services? Even if it's name node, master node or whatever? I just want to make sure that everyone can access services on every node in the cluster. I always wonder something like "this is a hive name node, do I need to add Gateway role?, oh that is a hbase master node, do I need to add Gateway role?,....". Now I don't want to think too much. Is there any performance issue, or incompatibility issue if I add Gateway role to every node for every service just for the sake on simplicity?
... View more
Labels:
- Labels:
-
Cloudera Manager
05-14-2023
07:53 PM
I successfully installed it on 3 nodes. Normally you only need to install everything on 1 node (things like java, python you have to install on 3 nodes first of course). When go to the CM UI website, you can add another node and Cloudera will automatically install everything for you. In case you want to install things manually. Install all 3 packages "cloudera-manager-daemons", "cloudera-manager-agent", "cloudera-manager-server" on your main node, and for other nodes only install "cloudera-manager-daemons" "cloudera-manager-agent" and start these agent services. After that, you will see that two nodes are "managed" on the CM UI, meaning that you can skip the "Install Agent" step (since you've already installed "cloudera-manager-agent" and start it)
... View more
05-11-2023
06:32 AM
I have a cluster of 3 nodes (all brand new with centos7, no java, no MySQL, nothing at all). I'm following this official install guide to install CDH 6.2.0 on the first node (called node1). Everything was fine, but do I need to install everything in the guide the same way for node2 and node3? I mean do I need to run sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server and sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm and all other commands in all 3 nodes? The instruction is unclear. I read some article on the internet that said I only need to install "cloudera-manager-agent", "cloudera-manager-daemons" on all nodes, "cloudera-manager-server" only need to install on 1 node or something like that. Which step should I execute on all nodes, and which step I only need to execute on 1 node? Or do I only need to install cdh on 1 node, then I use the Cloudera Manager UI website and add a new node, it will automatically install everything on that new node?
... View more
Labels:
- Labels:
-
Cloudera Manager
04-27-2023
08:16 PM
Thank you so much, this work for me!
... View more
02-01-2023
05:49 PM
I'm using a tool in which I have to point out the master node (driver node) of the Cloudera Spark Cluster (spark :// <some-spark-master> : 7077). Also as I learned, Spark has "Master Node", "Driver Node" and "Worker Nodes". So I decided to go to the Cloudera Web Manager and checked the Configuration Tab of the Spark service, but all I found are "Gateway instance" and "History Server instance". Where are the "Driver instance" and "Worker instance"? I can't add these two instances in the "Add Role Instances" too My guess is that it's in Yarn service configuration, but I can't find anything related to "Master", "Driver" or "Worker" either. So what is the link to "Spark Master" that ends with 7077 (what is the Node)? I can't find it anywhere in the Configuration tab
... View more
Labels:
- Labels:
-
Apache Spark
- « Previous
-
- 1
- 2
- Next »