@dturner,   Perhaps you don't have Spark_On_Yarn Gateway roles on any hosts?  Usually other roles depend on it, but I can't think of any other reasons the Deploy Client Configuration link wouldn't be available. ... View more

@krishnap,   Two things must be in place in order for your browser to performa authentication via SPNEGO to the UIs:   (1)   The browser must be able to access kerberos credentials (TGT)   (2)   The browser must be configured to allow Kerberos authentication to the hosts to which you are connecting.    It sounds as if your browser may be configured to allow authentication to one set of hosts but not another.  I recommend checking to make sure your browser is configured as described in the link that @manuroman provided in the previous response.   ... View more

@Youssef,   (1)   Make sure you have selinux disabled.  cloudera-scm-server-db will fail much like you showed if selinux is enforcing. (run sestatus to check status)   (2)   Check /var/log/cloudera-scm-server/db.log It may have more failure info   (3)   Check /var/lib/cloudera-scm-server-db/data/pg_log   It may have files containing information regarding failures of some sort. ... View more

Hi @ramin,   Here are some general instructions I found internally. Note: You can change the path to match the OS release and CDH version of the client you need.   On the external host download the CDH repo file to the /etc/yum.repos.d/ directory: curl -O https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/cloudera-cdh5.repo Edit the base URL in the cloudera-cdh5.repo file to install the CDH version (otherwise it will install the latest). For example, to install the 5.7.1 hadoop-client, update the baseurl to: baseurl=https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.7.1/ Install the hadoop-client rpm: $ yum clean all $ yum install hadoop-client        (See http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.7.1/RPMS/x86_64/) In Cloudera Manager navigate to, HDFS -> "Actions" drop down -> "Download Client Configuration" (this will download a zip file called hdfs-clientconfig.zip). Move the zip file over to the external host and unzip it. Copy all the unzipped configuration files to /etc/hadoop/conf. Example: $ cp *  /etc/hadoop/conf Run hadoop commands. Example: $ sudo -u hdfs hadoop fs -ls Note: You can also download the RPM file and install locally if desired.   ... View more

@Dacoo,   If Cloudera Manager is managing your cluster, it is needed to start your services. This is because Cloudera Manager creates all the configuration files for your services and then uses shell scripts to start the process.  Without Cloudera Manager, you will not have an up-to-date set of configuration files for the process.  In essence, there is no "easy" way to start your HDFS service.   If you lost your Cloudera Manager databse and have no means for recovery, it is possible to create a new one, but it will require a gread deal of work.  You would need to add the same services and roles that are on each host.  The agents should still be heartbeating to Cloudera Manager, so CM should allow you to select those Managed Hosts and add them to your cluster.  You would need to make sure all your HDFS disk paths are correct and add back any configuration items as you had them before.   For a frame of reference, you can look at existing process directories.  When CM starts a process it will lay down the configuration files in   /var/run/cloudera-scm-agent/process   You can look at all the configuration files for each Role (process) there to guide you.   If you are installing Cloudera Manager on a new host, you will need to update the "server_host" configuration in the /etc/cloudera-scm-agent/config.ini file for ayour agents.   If you really need to get the cluster running right away before you can start rebuilding the Cloudera Manager information, let us know.  There is is a way to do it from the command line, but it is a bit tricky and unproven.   -Ben     ... View more

@dewdrop,   No worries at all... we are just glad you got it working.  Nothing more frustrating then being stuck like you were! ... View more

@dewdrop,   So it does sound like the host is not accessible from your browser's network.  Packet sniffing is a good way to help determine what is going on.  EC2 support might know more.  This is an issue about network and packets and the cloud config, so talking to those who run it may help. ... View more

@dewdrop,   If your browser cannot connect to ip-172-31-44-224.us-west-2.compute.internal:7180, then make sure that curl can connect  You tested with curl, but used "localhost" so the test was not as close a comparison as one would like.   Make sure that the ip-172-31-44-224.us-west-2.compute.internal hostname can be resolved to a proper IP address and that your host has a route to that IP address.   WireShark is a great tool decoding packets sent over the wire and can give you perspective about where the problem may be occurring   I'm not saying that it isn't firewalls or otherwise, but it would be good to clarify what is stopping the client from reaching the server host/port ... View more

@Chuang,   I see you state "my hdfs in datanode could not work".   Please show us or explain what you tried to do, what you expected, and what actually happened. What helped you conclude there was something wrong with HDFS?   Cheers,   Ben ... View more

@sandy05,   No, not usually and that is why we didn't code to add sleep there.  To be honest, I don't know the history of the need for the "sleep" in some OSes and not others.  Indeed, it has not been needed for el6 as far as I know. Based on your report of the issue, though, the situation usually ends up being resolved (in Cloudera internal cases) by inserting a sleep of 1 second.   If that doesn't help, let us know and share with us the edited file so we can verify the change.   Ben ... View more