Member since
09-02-2016
523
Posts
89
Kudos Received
42
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1722 | 08-28-2018 02:00 AM | |
1341 | 07-31-2018 06:55 AM | |
3592 | 07-26-2018 03:02 AM | |
1493 | 07-19-2018 02:30 AM | |
4092 | 05-21-2018 03:42 AM |
07-28-2019
05:13 PM
@cjervis Very good news and thanks much for sharing the happy moment with us! Waiting for the update!! I have went through the FAQ about new repuation program... looks very interesting! Still I am not 100% clear about my current/existing reputation. I have contributed to Cloudera Community for 2.5 years with more than 500+ posts and received Champion award for the year 2017. But unfortunately not able to contribute in recent days (since Jan-2019) due to my new role/technologies and some additional works... Hope I can re-start my contribution in couple of months... Now my question is, Does my reputation still valid after you roll out a new reputation system? or all my effort will be voiding? Please clarify
... View more
12-11-2018
07:29 AM
1 Kudo
@orak Are you using Cloudera Enterprise by any chance? if so, you can generate report from CM -> Clusters (top menu) -> Reports -> Directory usage For more details, pls refer https://www.cloudera.com/documentation/enterprise/5-13-x/topics/cm_dg_disk_usage_reports.html#cmug_topic_12_1
... View more
11-12-2018
07:27 PM
@yunsu It means, some services are free of cost up to 100 nodes and more than 100 nodes (mostly for commercial purposes) will be charged or free version may not allow to add more than 100 nodes. But if you/your company has a valid licensed product then you don't need to worry about it
... View more
11-12-2018
07:09 PM
@Tomas79 Please increase the below parameter value as needed and try again, it may help you Java Heap Size of Catalog Server in Bytes
... View more
11-05-2018
12:21 PM
@public Are you using Cloudera Manager? if so, do you have same version of CM in your prod and qa then you can try the below 1. CM -> Backup menu -> Peers (add peer) - one time 2. CM -> Bakcup menu -> Replication schedule -> Create schedule (as many as you want)
... View more
11-02-2018
09:14 AM
@sidh Not sure it is considering different user id from beeline and spark... if so The one reason could be a keytab if it is a kerberoized env, you can maintain your keytab somewhere and call it from your spark-submit as follows. It will make sure to execute your spark job with required kerberos ticket spark-submit ..... --master yarn \ --keytab /home/auth/myauth.keytab \ also pls make sure the user/group has been assigned to the required sentry role , it has been granted to the required DB with create permission
... View more
10-26-2018
11:24 AM
@DanielWhite I had the similar issue long back and below was my findings Please check the owner of HDFS folder/files for the corresponding db that you are trying to delete. if you are the owner and trying to delete the table/db from hive/impala, it will delete both metadata and hdfs file/folder. Whereas you are not the owner of hdfs folder/file but got an access in hive/impala to manage data and trying to delete it, it will just delete the metadata but not the underlined folder/files from hdfs pls try this with a sample db/table for more understanding
... View more
10-23-2018
12:50 PM
1 Kudo
@Broche Please refer the below link, it may help you https://www.cloudera.com/documentation/enterprise/5-8-x/topics/impala_set.html
... View more
10-19-2018
08:58 AM
@archit_neema That is the reason why i've asked to use -R command -chown # you are talking about this, yes it will impact only once -chown -R # I am talking about this, this will triger the recursive
... View more
10-18-2018
12:41 PM
@archit_neema you can use -R option on the parent directory, so that it will inherit the permission recursively to the child folder. hdfs dfs -chown -R hive:hive /user/test Note: you have to be super user to change ownership of any folder in hdfs
... View more
10-17-2018
07:40 AM
@chriswalton007 According to the below link, If you have CDH 5.7 and above then you can upgrade an existing cluster to Cloudera 6. But this is for enterprise edison , are you trying it from enterprise edison? https://community.cloudera.com/t5/Community-News-Release/ANNOUNCE-Cloudera-Enterprise-6-0-Released/m-p/79235
... View more
10-10-2018
08:11 AM
@Eaks There are different approaches, usually i do follow the below ## Connect to beeline.... ## Run the following commands beeline> Create role role_name; beeline> DROP ROLE role_name; etc
... View more
10-08-2018
07:47 PM
@kush Please make sure "hive-site.xml" is present in /etc/spark/conf (or) in the corresponding spark2 config directory. You can either copy paste from hive config directory (or) create a soft link
... View more
10-04-2018
01:21 PM
@fil You can get this report from Cloudera Navigator. Search by user id and apply filter as needed
... View more
09-25-2018
07:45 PM
@mdjedaini There is nothing to do with cloudera on this request as there are so many other tools are available in the market. I am not sure how big your environment. In general, those who are using big environments with huge nodes will use some tools like Chef, Puppet, Terraform, Ansible, etc to achieve your requirement (for cloud there are another different set of tools like Cloudformation, etc) In high level, you can divide them into two category: Push and Pull based a. Tools like Puppet and Chef are pull based. Agent/Client on the server periodically checks for the configuration information from central server(master) b. Ansible is Push based. Central server pushes the configuration information on target servers. You control when the changes are made on the servers
... View more
09-06-2018
07:57 AM
@mohankumars yes, if you are going to use it for your personal use (or) practice purpose... with less loading
... View more
09-06-2018
02:39 AM
@phaothu To do via CM, Login as admin to CM -> HDFS -> Instances -> 'Federation and high availability' button -> Action -> Manual Failover
... View more
09-05-2018
03:47 AM
@Hichki In general, this issue should go off after you kinit (if you are using command line). This is just an additional check to make sure everything is good in your new cluster: Pls check whether the required kerbeors Principals are added as needed in the new cluster. You can use the commands like kadmin.local and klist to get the principal and make sure the similar principals are available in your new cluster (CM -> Administration(menu) -> Security -> Kerberos Credentials). Pls make sure all the principals are mapped to the corresponding host in the above link
... View more
09-04-2018
01:43 AM
@hadoopNoob if the command is working on the other nodes then run the netstat command again on both the nodes (for the port starting 4040) to see the difference. it is clear that it is not a spark issue as it is working form other nodes. so you have to identify the port open/availability status
... View more
09-03-2018
03:10 AM
@hadoopNoob yes, it may be due to port, pls try the below export SPARK_MAJOR_VERSION=2 pyspark --master yarn --conf spark.ui.port=12888 pyspark --master yarn --conf spark.ui.port=4041 pyspark --master yarn --conf spark.ui.port=4042 etc
... View more
08-31-2018
05:08 AM
1 Kudo
@vkrish You have to follow any of those links depends upon your requirement. 1. https://www.cloudera.com/documentation/enterprise/5-10-x/topics/cm_mc_autoconfig.html if you want to install, upgrade, add a new service, add a cluster, etc then you can follow your link. Mostly this is one time work, as it will be used for only initial setup. Also this is an optional for small clusters. May be suitable for large cluster. 2. https://www.cloudera.com/documentation/enterprise/5-10-x/topics/cdh_ig_yarn_tuning.html if you are using mapreduce, yarn, hive, spark with mr, sqoop, etc and if you want to tune those jobs then you can use the link that i've shared. Whenever you have install/configure your cluster, it will come with default setttings, the default may not suitable for all the scenarios, so this link will give you the instruction to customize your environment for better performance
... View more
08-29-2018
03:37 AM
@vkrish Not sure which document that you have referred (in case if you have referred a different link then ) The below link has an excel sheet, you have to download and fill as needed to get the recommeded values https://www.cloudera.com/documentation/enterprise/5-10-x/topics/cdh_ig_yarn_tuning.html
... View more
08-29-2018
03:29 AM
@Matt_ I can give you two easy steps , it may reduce your burden 1. To list the valid kerberos principal
$ cd /var/run/cloudera-scm-agent/process/<pid>-hdfs-DATANODE
$ klist -kt hdfs.keytab
## The klist command will list the valid kerbros principal in the following format "hdfs/<NODE_FQDN>@<OUR_REALM>"
2. to kinit with the aboev listed full path
$ kinit -kt hdfs.keytab <copy paste the any one of the hdfs principal from the above klist>
... View more
08-28-2018
10:46 AM
@Matt_ It may possible if your JAVA_HOME is not referring to the right path. export JAVA_HOME=<the right path -or- (usually /usr/java)> please check java path at node13 and set the right path and try again, it may help you
... View more
08-28-2018
02:04 AM
@tavi99 What about this? CM -> Administration (menu) -> Settings -> search for a. "Authentication Backend order" -> choose 'External then database'
... View more
08-28-2018
02:00 AM
1 Kudo
@AWT If you have your data is in hdfs and If your CM version is same in all your cluster/environment (if you are using different CM Login), then the easy way is ClouderaManager -> Backup(menu) -> Peers -> Add Peer ClouderaManager -> Backup(menu) -> Replication Schedules -> Create schedule or you can use distcp
... View more
08-28-2018
01:41 AM
@tavi99 Not sure you are getting the user info from ldap/ad or database, or some other 1. CM -> Hue-> config -> Authentication Backend a. if you are using ldap then it should be "desktop.auth.backend.LdapBackend" b. if you are using database then it should be "desktop.auth.backend.AllowFirstUserDjangoBackend" 2. CM -> Administration (menu) -> Settings -> search for ldap a. "Authentication Backend order" -> choose 'External then database' b. make sure your ldap configuration is updated as needed NOTE: Before you make any changes/overwrite your config, pls make sure to take a backup Pls try this, it may help you
... View more
08-23-2018
01:45 AM
@saikrishnamante Login to your mysql as root and create the required databases and grant the priviledge Ex:
create database hive;create user 'hive' identified by 'password'; grant all on hive.* to hive; flush privileges;
create database rman;create user 'rman' identified by 'password'; grant all on rman.* to rman; flush privileges;
... View more
08-23-2018
01:38 AM
@BTibetanMastiff When I was in the similar siutation we tried the decommission in batches (3 to 4 nodes at a time). CM/Hadoop will internelly take care of the replication (for the deleted data from decommission node), so no need to worry about the replication part. So below are my recommendations 1. Decommission the nodes by batches 2. CM -> HDFS -> Web UI -> Datanods -> Decommissioning -> Make sure there is no under replicated blocks 3. CM -> All Hosts -> Commission state(left side menu) -> Decommissioned -> Make sure your host is listed 4. If you have time, wait for 1 or 2 days or more then delete the host from CM
... View more