Member since
07-06-2018
24
Posts
4
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3093 | 11-06-2019 04:11 AM | |
2181 | 10-31-2018 04:56 AM | |
6508 | 10-14-2018 11:04 AM | |
7668 | 07-18-2018 12:26 AM |
11-06-2019
04:11 AM
After we configured these firewall rules, I was able to build the custom image: http://archive.cloudera.com/* PORT: 80 http://ppa.launchpad.net/* PORT: 80 Hint on docker company-registry: I was at the beginning a bit confused about the docker company-registry because we never set up a public docker registry. This is important because when you build and push the docker image, the commands require the company-registry. I could find it simply with the command docker images You will get a list of images registered on your machine. I found out that the company registry in my case was docker.repository.cloudera.com/cdsw/ So what I did was just building the docker image with my company-registry. In my case there was no need to push the image to docker. (see CDSW GPU Guide) docker build --network host -t docker.repository.cloudera.com/cdsw/cdsw-cuda:8 . -f cuda.Dockerfile After that, the image was built successfully. The image can be listed with the command docker images One should see the newly created docker image in the repository. For the last step the site admin. has to add this image to CDSW (see CDSW GPU Guide). The Repository:Tag in CDSW would be the same as shown above with the company-registry. In this example it would be Repository:Tag docker.repository.cloudera.com/cdsw/cdsw-cuda:8 For airgapped installations you need these firewall rules to be able to build the NVIDIA image for CDSW: (all on Port 80) http://developer.download.nvidia.com/* http://archive.ubuntu.com/ubuntu/* http://security.ubuntu.com/ubuntu/* http://archive.cloudera.com/* http://ppa.launchpad.net/* Regards.
... View more
10-17-2019
08:43 AM
Happy to hear that. Yes, we have this in Radar and soon our doc will be updated with known issue in 1.6 version. Cheers, Keep visiting and contributing Cloudera Community
... View more
10-15-2019
04:21 PM
Thank you very much for the reply and answer @Bhuv
... View more
04-05-2019
01:03 AM
1 Kudo
Thank you for the great explanation @AutoIN. This solved my problem. On our CDSW cluster we have 2 nodes with a master and a slave. As described, I was able to figure out that the available cpu and memeory on both hosts are badly distributed. As an example, I'm able to spin an engine with a lot of vcpus but with little memory and vice versa. I was just not aware that a session can't share resources across nodes. Thank you very much!
... View more
10-31-2018
04:56 AM
The problem could be solved with setting up the Wildcard DNS Subdomain for CDSW properly. (CDSW 1.4.X documentation page 51) Thanks.
... View more
10-15-2018
08:59 AM
EDIT: I did a copy/paste mistake! Please ignore my full-log given in my post above. Here is the correct Error-Log: /usr/share/cmf/bin/gen_credentials.sh failed with exit code 1 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ CMF_REALM=MYCOMPANY.REALM
+ KEYTAB_OUT=/var/run/cloudera-scm-server/cmf2548823212650177196.keytab
+ PRINC=sdc/hostname.FQDN@MYCOMPANY.REALM
+ MAX_RENEW_LIFE=604800
+ KADMIN='kadmin -k -t /var/run/cloudera-scm-server/cmf6838080336847771087.keytab -p admin/admin@MYCOMPANY.REALM -r MYCOMPANY.REALM'
+ RENEW_ARG=
+ '[' 604800 -gt 0 ']'
+ RENEW_ARG='-maxrenewlife "604800 sec"'
+ '[' -z /var/run/cloudera-scm-server/krb52847952611766397096.conf ']'
+ echo 'Using custom config path '\''/var/run/cloudera-scm-server/krb52847952611766397096.conf'\'', contents below:'
+ cat /var/run/cloudera-scm-server/krb52847952611766397096.conf
+ kadmin -k -t /var/run/cloudera-scm-server/cmf6838080336847771087.keytab -p admin/admin@MYCOMPANY.REALM -r MYCOMPANY.REALM -q 'addprinc -maxrenewlife "604800 sec" -randkey sdc/hostname.FQDN@MYCOMPANY.REALM'
WARNING: no policy specified for sdc/hostname.FQDN@MYCOMPANY.REALM; defaulting to no policy
add_principal: Operation requires ``add'' privilege while creating "sdc/hostname.FQDN@MYCOMPANY.REALM".
+ '[' 604800 -gt 0 ']'
++ kadmin -k -t /var/run/cloudera-scm-server/cmf6838080336847771087.keytab -p admin/admin@MYCOMPANY.REALM -r MYCOMPANY.REALM -q 'getprinc -terse sdc/hostname.FQDN@MYCOMPANY.REALM'
++ tail -1
++ cut -f 12
get_principal: Operation requires ``get'' privilege while retrieving "sdc/hostname.FQDN@MYCOMPANY.REALM".
+ RENEW_LIFETIME='Authenticating as principal admin/admin@MYCOMPANY.REALM with keytab /var/run/cloudera-scm-server/cmf6838080336847771087.keytab.'
+ '[' Authenticating as principal admin/admin@MYCOMPANY.REALM with keytab /var/run/cloudera-scm-server/cmf6838080336847771087.keytab. -eq 0 ']'
/usr/share/cmf/bin/gen_credentials.sh: line 35: [: too many arguments
+ kadmin -k -t /var/run/cloudera-scm-server/cmf6838080336847771087.keytab -p admin/admin@MYCOMPANY.REALM -r MYCOMPANY.REALM -q 'xst -k /var/run/cloudera-scm-server/cmf2548823212650177196.keytab sdc/hostname.FQDN@MYCOMPANY.REALM'
kadmin: Operation requires ``change-password'' privilege while changing sdc/hostname.FQDN@MYCOMPANY.REALM's key
+ chmod 600 /var/run/cloudera-scm-server/cmf2548823212650177196.keytab
chmod: cannot access ‘/var/run/cloudera-scm-server/cmf2548823212650177196.keytab’: No such file or directory
>>
... View more
10-14-2018
11:17 AM
Thanks Boris for the below information. Will try the same.
... View more
10-08-2018
03:50 PM
@Baris Glad you got it working. With Director version 2.4+ you actually don't need to use a bootstrap script as shown in my previous example. You can just specify the CSD URL on the conf file and Director will automatically download and place it in '/opt/cloudera/csd/'. See the documentation here: https://www.cloudera.com/documentation/director/latest/topics/director_non-cdh_products_custom_descriptors.html#concept_e2y_dpj_4z
... View more
07-30-2018
07:48 PM
Have you followed the solution made above? Depending on where you are trying to write into your cluster, unless you have full access to communicating with all your DataNode hosts and its ports, you will face this error.
... View more