Re: Disable Cloudera Management Debug WebUIs (Host...

michalis · ‎04-12-2016

"I am putting the following HTTP URL in the address bar, e.g. " the URL that you type in the address bar is translated into GET by the web browser, not a PUT method. You could use a REST client (web browser) plugin to assist your with submitting PUT/POST/DELETE methods from your browser.

michalis · ‎04-12-2016

" I put a HTTP PUT operation in the browser " Can you elaborate how you use HTTP PUT in your web browser? Please write down the steps you are using to do this, for example are you using a browser plugin, if so which?

michalis · ‎04-08-2016

> but everything I put a HTTP PUT operation in the browser Can you clarify how you are submitting PUT request from the browser, so we can reproduce it in-house. Try the following from a linux terminal #PUT # curl -i -X PUT "http://XX.XX.XX.XX:50070/webhdfs/v1/tmp/mapreduce?op=MKDIRS" #GET # curl -i -X GET "http://XX.XX.XX.XX:50070/webhdfs/v1/tmp?op=LISTSTATUS" Reference: https://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Make_a_Directory

michalis · ‎03-25-2016

> write(2, 0x7ffeb5d5c650, 54effective uid is not 0, is sudo installed setuid root?) = 54 The above could indicate an issue also, looking into the "stat" output have you change the permission on /usr or only in /usr/java/...; can you check if you sudo works try this command sudo su - ; can you list your mount flags (cat /proc/mounts) ?

michalis · ‎03-24-2016

Great to hear.

michalis · ‎03-24-2016

can provide the output for following command $ stat /usr/java/jdk1.8.0_72/bin/java

michalis · ‎03-24-2016

Hello, The process is actually documented here http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ag_restore_server.html What database is Cloudera Manager server is connected to? michalis

michalis · ‎03-20-2016

Hi Romulus, OS level dependencies causing the conflict, since you have Red Hat Enterprise Linux Server you should file a ticket with RHEL to troubleshoot the issue. At first glance it looks like your IUS repo is causing the conflict. Hope this helps.

michalis · ‎03-18-2016

In you cloudera-quickstart-vm-5.5.0-0-virtualbox over virtualbox, open the web browser navigate and login to Cloudera Manager. In Cloudera Manager Server Homepage using your keyboard hit the Search Hotkey " / " (a forward slash without the quotes). Then again using your keboard type acl - from the drop-down list select the 'hdfs: Enable Access Control Lists' < this should take you to the relevant configuration page where you have to click on the checkbox > [Save] and restart the hdfs service.

michalis · ‎03-18-2016

Is your cluster managed by Cloudera Manager server, or are you managing the Hadoop configurations using an editor?

michalis · ‎03-16-2016

Yes you can stop service and it's individual roles via CM API [1] > Is there a specific table it stores relevant data for us to query and check? Service status can be check directly from the CM API querying the relevant endpoint, specifically for the service you'd need to GET (open your browser) to the service endpoint [2] which will give you its state [3] As a start point you could see the example [4a][4b] [1] https://github.com/cloudera/cm_api/wiki/Start-Stop-Restart-JobTracker-Role [2] https://cloudera.github.io/cm_api/apidocs/v11/path__clusters_-clusterName-_services_-serviceName-.html#GET [3] https://cloudera.github.io/cm_api/apidocs/v11/ns0_apiServiceState.html [4a] https://github.com/cloudera/cm_api/tree/master/python/examples [4b] https://github.com/gdgt/cmapi

michalis · ‎03-15-2016

Can you summarise your end goal, so I can provide you with an appropriate information.

michalis · ‎03-15-2016

So far we confirmed that nothing is actually running on the nodes and attempting a simple workaround has not given you a positive outcome. Let's do it the hard way 😉 per your description 1 host(s) have stale Kerberos client configuration but will not be updated because there is a process running. Consider stopping roles on these hosts to ensure that they are updated by this command: can you check in your CM database under the _roles_ table and list all the _configured_status - we could set it to STOPPED and see if you can proceed withe deploying Kerberos client-config ie: login to your cloudera manager database and execute the below SQL query; select name, configured_status from roles; can you paste the output so I can check which role is RUNNING

michalis · ‎03-07-2016

Can you attempt the following workaround; Stop the cluster: Cloudera Manager > Cluster > Stop Stop Cloudera Management Service: Cloudera Manager > Management Service > Stop Deploy Kerberos Client Configuration: Cloudera Manager > Cluster > Deploy Kerberos Client Configuration Start Cloudera Management Service: Cloudera Manager > Management Service > Start Start the cluster: Cloudera Manager > Cluster > Start let me know if this helps

michalis · ‎03-07-2016

> 1 host(s) have stale Kerberos client configuration but will not be updated because there is a process running. In the CM UI> Hosts> click (affected host)> click [Processes] tab to see what processes are running on that host. Alternative, on the affected host terminal. Check [1] the child CDH role/processes that supervisord is controlling; Stop the relevant role/process from Cloudera Manager Let me know if this helps, [1] option 1. [bash]# for process in $(pgrep -P $(ps aux | grep [s]upervisord | awk '{print $2}')); do ps -p $process -o euser,pid,command | awk 'NR>1'; done option 2. ps auxwf

michalis · ‎03-03-2016

>... query by id filter, ... For clarification, when you say "=id==" are you referring to "hostId", or the unique "id" of the event. Reason is that in your supplied URL "... query=id==<hostid>", the "id" is a unique ID for an event [1]; however, <hostid> ( HOST_IDS) key name within the " attributes" listing [1]. I think your filter is expected to return 0, unless you have valid the "id" for the event. Does it pull info if you change your query to filter the attributes.HOST_IDS? ie: http://cm:7180/api/v10/events ?query=attributes.HOST_IDS==%22<hostid>%22 Let me know if this helps. [1] https://cloudera.github.io/cm_api/apidocs/v11/ns0_apiEvent.html

michalis · ‎03-02-2016

> If I have either the hostid or hostname, how can I pull back why the node is in a concerning status in CM using only the API? There should be a record in your events [1a,b], using the CM API try filtering the hostid/host attributes [2] Hope this helps [1a] http://cloudera-server:7180/cmf/events [1b] http://cloudera-server:7180/api/v10/events [2] https://cloudera.github.io/cm_api/apidocs/v11/path__events.html

michalis · ‎01-30-2016

That seems to be Amazon Linux as you've mentined; are you able to choose RHEL/CentOS AMI, or any of the following listed supported OS [1][2] [1] http://www.cloudera.com/documentation/enterprise/5-4-x/topics/cm_ig_cm_requirements.html#cmig_topic_4_1_unique_1 [2] http://www.cloudera.com/documentation/director/latest/topics/director_deployment_ami.html

michalis · ‎01-30-2016

The first issue has to do with NTP [1] and the second, is when the Agent attempted to read the json contents (possibly Service Monitor metrics), it encoutered an error hece the Error fetching metrics at 'http://host-hd-01.corp.nodalpoint.com:8086/jmx' Do you know which for which role the health check is reporting - for a reference, can you attach a screenshot? [1] http://www.cloudera.com/documentation/enterprise/latest/topics/install_cdh_enable_ntp.html

michalis · ‎01-30-2016

> Now upgrading to CDH5.5.1 using parcels would be okay? Or do you foresee any challenges Please follow the "Migrating from Packages to Parcels" [1] carefully. The documentation is straighforward and I don't forsee any challenges. I am here to help if you get stuck. [1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_migrating_packages_to_parcels.html

michalis · ‎01-30-2016

> With Cloudera Express edition, I believe that we cannot use parcels for installation. Please confirm this. You can use parcels or packages with Cloudera Express edition. Listed here [1] are the features compared to Enterprise. [1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_feature_differences.html

michalis · ‎01-29-2016

As the error indicates the listed hosts are still running CDH5.4.2, if you are using Parcels you will have to push the new parcels on the hosts. Use the following docs [1][2] as a guide. hope this helps [1] http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ag_upgrade_cm5.html [2] http://www.cloudera.com/documentation/enterprise/latest/topics/install_upgrade_to_cdh55_parcels.html

michalis · ‎01-27-2016

Yes

michalis · ‎01-27-2016

> This indicates that agent is unable to connect to HOSTMONITOR, is HMON running? [27/Jan/2016 03:16:44 +0000] 3069 MonitorDaemon-Reporter throttling_logger ERROR Error sending messages to firehose: MGMT-HOSTMONITOR-244944378552b77b5c898d702d752f7f > This indicates that the command timed out when attempting to run a DNS test [1] [27/Jan/2016 03:19:46 +0000] 3069 DnsResolutionMonitor throttling_logger ERROR Timeout with args ['/usr/java/jdk1.7.0_67-cloudera/bin/java', '-classpath', '/usr/share/cmf/lib/agent-5.5.1.jar', 'com.cloudera.cmon.agent.DnsTest'] Example: [bash]# /usr/java/jdk1.7.0_67-cloudera/bin/java -classpath /usr/share/cmf/lib/agent-5.5.1.jar com.cloudera.cmon.agent.DnsTest > This indicates that supervisord [2] was not running at that time [27/Jan/2016 03:08:52 +0000] 3012 MainThread agent ERROR Failed to connect to previous supervisor. [1] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_networknames_configure.html [2] "Agent Process Supervision in Detail" http://blog.cloudera.com/blog/2013/07/how-does-cloudera-manager-work/

michalis · ‎01-26-2016

At first I'd make sure that there's no errors in the the agent log ie: # less /var/log/cloudera-scm-agent/cloudera-scm-agent.log Also, when you issue the below command is it pointing to the right folder? # alternatives --display impala-shell > ... To specify the parcel directory, local parcel repository, add a parcel repository ... that value is only for the .parcel files that CM Server downloads Please modify your -agent/config.ini uncomment and set the value for the below property, do the same if you have issues on other nodes. # vi /etc/cloudera-scm-agent/config.ini ... # Parcel directory. Unpacked parcels will be stored in this directory. # Downloaded parcels will be stored in <parcel_dir>/../parcel-cache # parcel_dir=/opt/cloudera/parcels ... > Can I just symlink to CDH/bin? That is one of Cloudera Agents [1] function - hard restart the agent will normally fix the alternative issues [1] Agent parse the parcel metadata and creates the alternatives link, see /opt/cloudera/parcels/CDH/meta/alternatives.json

michalis · ‎01-26-2016

Is your CDH deployed using parcels or packages? As for the binaries: - packages: can you list your packages ie RHEL/Centos # rpm -qa | grep -i cdh (or grep -i impa) - parcels: you should be able to find them under /opt/cloudera/parcels/CDH (CM should have symlink the necessary shell commands) - Is it only impala-shell that is failing, have you tested hadoop, hive etc? Can you link the documentation you've followed to perform the install.

michalis · ‎01-25-2016

- What browser are you using, have you tried another browser? - May I ask you try performing a tcpdump on your CM host? 1. Open a terminal (ssh/putty) into your CM Server host 2. # ifconfig | grep -B1 "inet " ... eth0 Link encap:Ethernet HWaddr DE:AD:BE:EF:DE:AD inet addr:10.17.81.7 Bcast:10.17.81.255 Mask:255.255.254.0 ... my host IP is "inet addr: 10.17.81.7" on interface eth0 use these info for step 3. and 4. 3. # tcpdump -A -s 0 'tcp port 7180' -i eth0 4. Put side by side the terminal window and open your browser url: http://[ip-of-your-cm]:7180/cmf/login - does your terminal window show any output/traffic?

michalis · ‎01-25-2016

Troubleshooting tip: On the node where you have installed Cloudera Manager Server, check if it is actually running; [linux shell]# service cloudera-scm-server status [linux shell]# /usr/java/jdk1.7.0_67-cloudera/bin/jcmd | grep "cmf.Main" 11082 com.cloudera.server.cmf.Main ... Check if your CM is listening on port 7180 [linux shell]# if (exec 6<>/dev/tcp/$(hostname)/7180) 2> /dev/null; then echo "CM _IS_ listening on port 7180"; fi [linux shell]# netstat -lntup | grep LIST ... tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 11082/java ... If any of the above fails check your server log files [linux shell]# less /var/log/cloudera-scm-server/cloudera-scm-server.log > web UI shows blank page. Are you behind proxy?

michalis · ‎01-25-2016

Can you let me know the output of your .../parcels is, for example mine shows "items" : [ { "product" : "CDH", "version" : "5.5.1-1.cdh5.5.1.p0.11", "stage" : "ACTIVATED", "clusterRef" : { "clusterName" : "example" } }, This is equivalen of having the following services (HIVE, HDFS ...) versions [1] "parcels": [ { "parcelName": "CDH-5.5.1-1.cdh5.5.1.p0.11-xxxx.parcel", "components": [ { "pkg_version": "0.7.0+cdh5.5.1+0", "pkg_release": "1.cdh5.5.1.p0.14", "name": "bigtop-tomcat", "version": "6.0.44-cdh5.5.1" }, { "pkg_version": "0.11.0+cdh5.5.1+77", "pkg_release": "1.cdh5.5.1.p0.14", "name": "crunch", "version": "0.11.0-cdh5.5.1" }, { "pkg_version": "1.6.0+cdh5.5.1+29", "pkg_release": "1.cdh5.5.1.p0.16", "name": "flume-ng", "version": "1.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-0.20-mapreduce", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-hdfs", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-httpfs", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-kms", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-mapreduce", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "2.6.0+cdh5.5.1+924", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hadoop-yarn", "version": "2.6.0-cdh5.5.1" }, { "pkg_version": "1.0.0+cdh5.5.1+274", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hbase", "version": "1.0.0-cdh5.5.1" }, { "pkg_version": "1.5+cdh5.5.1+57", "pkg_release": "1.cdh5.5.1.p0.14", "name": "hbase-solr", "version": "1.5-cdh5.5.1" }, { "pkg_version": "1.1.0+cdh5.5.1+327", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hive", "version": "1.1.0-cdh5.5.1" }, { "pkg_version": "1.1.0+cdh5.5.1+327", "pkg_release": "1.cdh5.5.1.p0.15", "name": "hive-hcatalog", "version": "1.1.0-cdh5.5.1" }, { "pkg_version": "3.9.0+cdh5.5.1+333", "pkg_release": "1.cdh5.5.1.p0.14", "name": "hue", "version": "3.9.0-cdh5.5.1" }, { "pkg_version": "2.3.0+cdh5.5.1+0", "pkg_release": "1.cdh5.5.1.p0.17", "name": "impala", "version": "2.3.0-cdh5.5.1" }, { "pkg_version": "1.0.0+cdh5.5.1+116", "pkg_release": "1.cdh5.5.1.p0.14", "name": "kite", "version": "1.0.0-cdh5.5.1" }, { "pkg_version": "1.0.0+cdh5.5.1+0", "pkg_release": "1.cdh5.5.1.p0.14", "name": "llama", "version": "1.0.0-cdh5.5.1" }, { "pkg_version": "0.9+cdh5.5.1+26", "pkg_release": "1.cdh5.5.1.p0.15", "name": "mahout", "version": "0.9-cdh5.5.1" }, { "pkg_version": "4.1.0+cdh5.5.1+223", "pkg_release": "1.cdh5.5.1.p0.14", "name": "oozie", "version": "4.1.0-cdh5.5.1" }, { "pkg_version": "1.5.0+cdh5.5.1+176", "pkg_release": "1.cdh5.5.1.p0.15", "name": "parquet", "version": "1.5.0-cdh5.5.1" }, { "pkg_version": "0.12.0+cdh5.5.1+72", "pkg_release": "1.cdh5.5.1.p0.15", "name": "pig", "version": "0.12.0-cdh5.5.1" }, { "pkg_version": "1.5.1+cdh5.5.1+106", "pkg_release": "1.cdh5.5.1.p0.15", "name": "sentry", "version": "1.5.1-cdh5.5.1" }, { "pkg_version": "4.10.3+cdh5.5.1+325", "pkg_release": "1.cdh5.5.1.p0.15", "name": "solr", "version": "4.10.3-cdh5.5.1" }, { "pkg_version": "1.5.0+cdh5.5.1+94", "pkg_release": "1.cdh5.5.1.p0.15", "name": "spark", "version": "1.5.0-cdh5.5.1" }, { "pkg_version": "1.99.5+cdh5.5.1+33", "pkg_release": "1.cdh5.5.1.p0.14", "name": "sqoop2", "version": "1.99.5-cdh5.5.1" }, { "pkg_version": "1.4.6+cdh5.5.1+29", "pkg_release": "1.cdh5.5.1.p0.14", "name": "sqoop", "version": "1.4.6-cdh5.5.1" }, { "pkg_version": "0.9.0+cdh5.5.1+17", "pkg_release": "1.cdh5.5.1.p0.15", "name": "whirr", "version": "0.9.0-cdh5.5.1" }, { "pkg_version": "3.4.5+cdh5.5.1+91", "pkg_release": "1.cdh5.5.1.p0.15", "name": "zookeeper", "version": "3.4.5-cdh5.5.1" } ], http://archive.cloudera.com/cdh5/parcels/5.5.1.11/manifest.json

michalis · ‎01-25-2016

> How can i get version of various services (like Hive, Impala etc) using Cloudera Manager API. This is not exposed in the CM API -- however you can associate the version of various services via either going to their respective web console, or if you would like to use CM API you can check by viewing the ACTIVATED Parcel [1] and associate it with the CDH release version [2] or against the manifest.json [3]. Alternative, through CM web UI you can execute Host Inspector (CM> Hosts> [Host Inspector]) [1] http://[your-cm-server]:7180/api/v11/clusters/[cluster-name]/parcels [2] http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_vd_cdh_package_tarball.html [3] example for latest CDH5 http://archive.cloudera.com/cdh5/parcels/latest/manifest.json

Online	Offline
Last Visited	‎05-15-2022 05:39 AM

Date Registered	‎07-08-2013 12:09 PM
Last Visited	‎05-15-2022 05:39 AM
Total Messages Posted	543
Total Kudos Received	57

Re: Disable Cloudera Management Debug WebUIs (Host...

Re: ODBC Drivers for Hive - Cloudera vs. Hortonwor...

Re: Table 'metastore.CM_VERSION' doesn't exist whe...

Re: Cloudera Manager Install: scm_prepare_database...

Re: Enabling Keberos for cluster fails when import...

Re: WebHDFS error with HTTP PUT operations in Clou...

Re: WebHDFS error with HTTP PUT operations in Clou...

Re: WebHDFS error with HTTP PUT operations in Clou...

Re: cloudera manager will not start - fresh instal...

Re: Moving Cloudera Manager to a different server

Re: cloudera manager will not start - fresh instal...

Re: Moving Cloudera Manager to a different server

Re: Cloudera Manager Agent installation - package ...

Re: HDFS ACLs

Re: HDFS ACLs

Re: Need Guidance : Information of restarts for cl...

Re: Need Guidance : Information of restarts for cl...

Re: Kerberos client configuration will not be upda...

Re: Kerberos client configuration will not be upda...

Re: Kerberos client configuration will not be upda...

Re: Why is node in concerning status in CM using A...

Re: Why is node in concerning status in CM using A...

Re: Support for Amazon Linux 2015

Re: Web Server Status Bad Health

Re: Upgrade to CDH 5.5.1 Cloudera Express issues

Re: Upgrade to CDH 5.5.1 Cloudera Express issues

Re: Upgrade to CDH 5.5.1 Cloudera Express issues

Re: Can we configure Name Node High Availability u...

Re: Quickstart on AWS EC2 : DnsResolutionMonitor t...

Re: No binaries available in /usr/bin/

Re: No binaries available in /usr/bin/

Re: Clouderamanger web UI is blank

Re: Clouderamanger web UI is blank

Re: version of various services using Cloudera Man...

Re: version of various services using Cloudera Man...