@PDDF_VIGNESH  I hope you have connected to the port configured for your cluster.Does the URL return successful response on browser? The responses need to be checked only during the time of issue.   The JMX is generated by datanode. Not getting a response means there are either issues with datanode or any network issues with the coomunication.       ... View more

@PDDF_VIGNESH    See if you are able to get successful response from the agent to the host reported in the logs below: http://datanode02.hadoop:1006/jmx Few checks: 1. curl http://datanode02.hadoop:1006/jmx 2. telnet datanode02.hadoop:1006   If this is successful, restart the agent. If there is issue with response, you need to review DN logs for issues/workarounds suggested previously.   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.   ... View more

@Juris     The default filter value of 7 days is hardcoded in Hue.  You can find this set in your cluster hosts /usr/lib/hue/apps/jobbrowser/src/jobbrowser/templates/job_browser.mako [default file] as shown below. As a workaround this file can be edited across all hosts followed by Hue restart but changes would persist for all users. We generally don't recommend to edit the install files but just providing you with the information here.   self.timeValueFilter  = ko.observable  (7).extend({ throttle: 500 }); self.timeUnitFilter   = ko.observable  ('days').extend({ throttle: 500 });   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@Jua    Cloudera expects HA solutions for RDBMS to be transparent to Cloudera software, and therefore are not supported and debugged by Cloudera.   Having said that you can try to modify the /etc/cloudera-scm-server/db.properties with the URL in below format and test it out. com.cloudera.cmf.db.type=oracle com.cloudera.cmf.orm.hibernate.connection.driver_class=oracle.jdbc.driver.OracleDriver com.cloudera.cmf.orm.hibernate.connection.url=jdbc:oracle:thin:@(DESCRIPTION=(LOAD_BALANCE=off)(FAILOVER=on) / (CONNECT_TIMEOUT=5)(TRANSPORT_CONNECT_TIMEOUT=3)(RETRY_COUNT=3)(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1)(PORT=1521)) / (ADDRESS=(PROTOCOL=TCP)(HOST=hostname2)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=<Oracle-SID>))) com.cloudera.cmf.orm.hibernate.connection.username=<CM-Oracle-user> com.cloudera.cmf.orm.hibernate.connection.password=<password>   I suggest to check with your DB vendor to validate the jdbc URL if this still fails.   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@Zhaojie    The errors to focus here are the below ones which are causing unknown health alerts. This is being caused because CM agent is not able to connect to Host Monitor.   Please ensure the hosts generating the alerts have space available and are not overloaded in terms of COU and memory. You can also try to hard restart the CM agents if this is feasible option since hard restart will stop the services running on the host and clear any thread deadlocks in cluster services.   The health test result for HOST_SCM_HEALTH has become bad: This host is in contact with the Cloudera Manager Server. This host is not in contact with the Host Monitor. The health test result for HOST_AGENT_LOG_DIRECTORY_FREE_SPACE has become unknown: Not enough data to test: Test of whether the Cloudera Manager Agent's log directory has enough free space.   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.   ... View more

@akshay0103    Please check the Hue.ini content under field  [useradmin] if there are any non default permissions being used? Are you adding the user using create home directory permissions?   ... View more

@Mondi  Please refer below document which explains steps to configure jar for UDFs.  You can configure the cluster in one of several ways to find the JAR containing your UDF code, and then you register the UDF in Hive. https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_mc_hive_udf.html#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ea3   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@rmr1989    Ideally, for a missing mounts in the cluster you would automatically get alerts for services if any hadoop service directories cannot be accessed which are mapped to the mountpoint. They can be generic though like "no such file or directory" or "file not found" errors.   If you are looking for specific mountpoint availability, you should consider using script to scan for mountpoints in the cluster hosts and can send email alerts using SMTP from the host instead of Cloudera Manager.   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.           ... View more

@rmr1989    You can edit the expression manually to include the NOT operator. Check the below example which uses NOT operator for a trigger. A trigger expression takes the form: IF (CONDITIONS) DO HEALTH_ACTION   A condition is any valid tsquery statement with below syntax.  SELECT [metric expression]WHERE [predicate]   Mount point is an attribute which can be used in filter conditions but not metric expression. Hence, I believe it would not be possible to create a trigger on mount point scans.   Refer below document which lists the supported metrics: https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_metrics.html#xd_583c10bfdbd326ba--7f25092b-13fba2465e5--7e52   Example:   IF (SELECT total_xceivers_across_datanodes WHERE serviceName=$SERVICENAME AND last(total_xceivers_across_datanodes) != 0 AND entityName = "HDFS-1:ns1" AND category = "SERVICE") DO health:concerning   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@Prav    You can leverage CM API to track parcel distribution status: /api/v19/clusters/{clusterName}/parcels - This can be used to note the parcel name and version the cluster has access to   /api/v19/clusters/{clusterName}/parcels/products/{product}/versions/{version} - This can be used to track the parcel distribution status   Refer below link for more details http://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_parcels_products_-product-_versions_-version-.html   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@sarm    Minimum heap size should be set to : 4 GB Increase the memory for higher replica counts or a higher number of blocks per DataNode. When increasing the memory, Cloudera recommends an additional 1 GB of memory for every 1 million replicas above 4 million on the DataNodes. For example, 5 million replicas require 5 GB of memory. Set this value using the   Java Heap Size of DataNode in Bytes   HDFS configuration property. Reference: https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

@SeanU    This level of detailed log scanning and alert functionality is not available. The existing service role logs for which rules can be set will not contain each application exceptions logged since detailed information is present in the application logs.    You can check the available job history server logs and resource manager logs available to check if the logged in information during application run time helps serve your purpose. ... View more

@Mondi    What are these properties set to in your yarn configurations? 'yarn.app.mapreduce.am.staging-dir' 'yarn.app.mapreduce.am.staging-dir'    Also, can you try running below query before your hive query and share results >  set hive.exec.scratchdir=/tmp/mydir;   Regards, Paras ... View more

@AnjaliRocks    Can you please share your workflow.xml to understand exactly how you are trying to execute the operations? What is the behaviour you observe when you run insert overwrite query from beeline?   You can use the below query to set the hive scratch directories to set the temporary output path. set hive.exec.scratchdir=/tmp/mydir; Let me know if this helps.   ... View more

@Mondi    You can set the hive scratch directories to modify the intermediate data storage location from default /tmp hive.exec.local.scratchdir-  Local Directory where Hive stores jars and data when performing a MapJoin optimization. If not specified, Hive uses a default location. hive.exec.scratchdir Directory in HDFS where Hive writes intermediate data between MapReduce jobs. If not specified, Hive uses a default location.   Refer https://docs.cloudera.com/documentation/enterprise/6/properties/6.3/topics/cm_props_cdh600_hive.html#concept_6.3.x_hiveserver2_props__section_advanced_props   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.   ... View more

@SeanU    You can select the role group for the YARN role for which you want to configure log events, or search for "Rules to Extract Events from Log Files". These generate the events for the service and you can set alerts for these log events. Alert publisher can be configured to generate emails for alerts.   Refer https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_dg_monitoring_settings.html#cmug_topic_4_12_4__section_vlc_qr3_dr   Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

Replied on the new thread ... View more