Member since
01-15-2019
105
Posts
10
Kudos Received
22
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
161 | 09-21-2020 03:44 AM | |
532 | 08-12-2020 08:24 AM | |
555 | 08-11-2020 06:53 AM | |
309 | 08-04-2020 08:42 PM | |
394 | 07-23-2020 03:36 AM |
04-05-2021
05:16 AM
@akshay0103 Please check the Hue.ini content under field [useradmin] if there are any non default permissions being used? Are you adding the user using create home directory permissions?
... View more
09-21-2020
03:44 AM
1 Kudo
@Mondi Please refer below document, you can setup a local package repository for Cloudera Manager upgrade. https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_package_repo.html#internal_package_repo For CDH upgrade you can follow below document https://docs.cloudera.com/cdp-private-cloud-base/7.1.3/installation/topics/cdpdc-using-local-parcel-repository.html Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-12-2020
08:24 AM
@Mondi It is not compulsory to enable SSL but recommended to prevent the passage of plain text key material between the KMS and HDFS data nodes. You can continue to install Java Keystore KMS without adding SSL configurations. Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-11-2020
06:53 AM
@Mondi KMS service should be installed on your CDH cluster. Before installing KMS, you should have a dedicated cluster added using the Cloudera manager Add Cluster option which has the KTS service roles installed. If you are installing default Hadoop KMS Java Keystore KMS, the above can be ignored since the default Hadoop KMS included in CDH uses a file-based Java KeyStore (JKS) for its backing keystore. You can simply add the service from Cloudera Manager. Cloudera strongly recommends that you enable TLS for both the HDFS and the Key Trustee KMS services to prevent the passage of plain text key material between the KMS and HDFS data nodes. Refer below document https://docs.cloudera.com/documentation/enterprise/latest/topics/sg_hdfs_encryption_wizard.html#concept_fcq_phr_wt Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-10-2020
06:25 AM
@Mondi Cloudera provides two implementations of the Hadoop KMS. Refer below document for more details. https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_kms.html You need to install Key Trustee KMS only when using KTS as backing keystore instead of the file-based Java KeyStore (JKS) used by the default Hadoop KMS. There should be a separate cluster for keytrustee server. This would be mentioned as one of the steps when you enable HDFS encryption via the wizard. Refer below document https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/sg_hdfs_encryption_wizard.html#concept_n2p_5vq_vt Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-07-2020
01:09 AM
@Mondi You do not need to install Cloudera Navigator for KMS and KTS. Refer : https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/encryption_prereqs.html#concept_g23_454_y5__section_n4w_b5v_ls Please refer below documents for encrypting data at rest requirement and installing KMS and KTS. You must install Key Trustee Server before installing and using Key Trustee KMS. https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/encryption_planning.html#concept_c4m_knq_w5 https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/key_trustee_install.html#xd_583c10bfdbd326ba-590cb1d1-149e9ca9886--7b84 https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_ig_install_keytrustee.html#xd_583c10bfdbd326ba-590cb1d1-149e9ca9886--7860 Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
08-04-2020
08:42 PM
1 Kudo
@ebeb It seems there is some issue with either your credentials or connectivity to the URL. Please confirm if you are using credentials generated from the license file. Refer below document for obtaining username and password for adding to the baseurl https://docs.cloudera.com/cdp/latest/release-guide/topics/cdpdc-cm-download-information.html Also, confirm the URL connectivity from your CM server host using curl command : #ping archive.cloudera.com # curl https:// username : password @archive.cloudera.com/p/cm7/ 7.1.2 /redhat6/yum Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-29-2020
10:50 PM
@Mondi Please refer below document which explains steps to configure jar for UDFs. You can configure the cluster in one of several ways to find the JAR containing your UDF code, and then you register the UDF in Hive. https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_mc_hive_udf.html#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ea3 Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-24-2020
06:20 AM
@rmr1989 Ideally, for a missing mounts in the cluster you would automatically get alerts for services if any hadoop service directories cannot be accessed which are mapped to the mountpoint. They can be generic though like "no such file or directory" or "file not found" errors. If you are looking for specific mountpoint availability, you should consider using script to scan for mountpoints in the cluster hosts and can send email alerts using SMTP from the host instead of Cloudera Manager. Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-23-2020
04:09 AM
@rmr1989 You can edit the expression manually to include the NOT operator. Check the below example which uses NOT operator for a trigger. A trigger expression takes the form: IF (CONDITIONS) DO HEALTH_ACTION A condition is any valid tsquery statement with below syntax. SELECT [metric expression]WHERE [predicate] Mount point is an attribute which can be used in filter conditions but not metric expression. Hence, I believe it would not be possible to create a trigger on mount point scans. Refer below document which lists the supported metrics: https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_metrics.html#xd_583c10bfdbd326ba--7f25092b-13fba2465e5--7e52 Example: IF (SELECT total_xceivers_across_datanodes WHERE serviceName=$SERVICENAME AND last(total_xceivers_across_datanodes) != 0 AND entityName = "HDFS-1:ns1" AND category = "SERVICE") DO health:concerning Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-23-2020
03:36 AM
@fransyk It is not compulsory to have both the roles on different hosts. Having said that, in production environments it is recommended to differentiate worker hosts and master hosts. Please refer below document for recommended role allocations for different cluster sizes: https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_ig_host_allocations.html#concept_f43_j4y_dw Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-23-2020
02:55 AM
@fransyk Yes, you can have datanode and namenode role together on a host as long as the hardware requirements are met for the service to avoid any out of memory issues. Please refer https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-22-2020
06:55 AM
@pdev Yes, BDR is supported between different CDH versions. Refer below links for more details https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_bdr_replication_intro.html#concept_rt2_1wt_bx https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_pcm_bdr.html#bdr Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-22-2020
04:47 AM
@pdev Please refer document [1] for details on how to enable replication between cluster with kerberos configuration. To configure encryption of data transmission between source and destination clusters: Enable TLS/SSL for HDFS clients on both the source and the destination clusters. You may also need to configure trust between the SSL certificates on the source and destination. The certificates of the source cluster should be trusted by your destination cluster. Enable TLS/SSL for the two peer Cloudera Manager Servers. Refer link [2] for more details. [1] https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_bdr_replication_and_kerberos.html#xd_583c10bfdbd326ba-5676e95c-13ed333c3d9--7ff3 [2] https://docs.cloudera.com/documentation/enterprise/5-14-x/topics/cm_bdr_replication_and_encryption.html#concept_lrr_rcf_4r Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-22-2020
04:31 AM
@Prav You can leverage CM API to track parcel distribution status: /api/v19/clusters/{clusterName}/parcels - This can be used to note the parcel name and version the cluster has access to /api/v19/clusters/{clusterName}/parcels/products/{product}/versions/{version} - This can be used to track the parcel distribution status Refer below link for more details http://cloudera.github.io/cm_api/apidocs/v19/path__clusters_-clusterName-_parcels_products_-product-_versions_-version-.html Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-22-2020
04:08 AM
1 Kudo
@rok You can use the Cloudera Manager REST API to export and import all of its configuration data. The API exports a JSON document that contains configuration data for the Cloudera Manager instance. You can use this JSON document to back up and restore a Cloudera Manager deployment. Refer below document for the steps: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_intro_api.html#concept_dnn_cr5_mr Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-09-2020
02:05 AM
@sarm Minimum heap size should be set to : 4 GB Increase the memory for higher replica counts or a higher number of blocks per DataNode. When increasing the memory, Cloudera recommends an additional 1 GB of memory for every 1 million replicas above 4 million on the DataNodes. For example, 5 million replicas require 5 GB of memory. Set this value using the Java Heap Size of DataNode in Bytes HDFS configuration property. Reference: https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
07-09-2020
01:53 AM
1 Kudo
@tanishq1197 When you see this log entry, the Cloudera Manager Admin Console is ready: INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server Try to access the CM API instead, you should use the login credentials curl -u <username>:password http://<cm_host>:<port>/api/v19/clusters Are you able to reach out to the CM server from your desktop? #ping cmserverhost #telnet cmserverhost port
... View more
07-08-2020
11:59 PM
@SeanU This level of detailed log scanning and alert functionality is not available. The existing service role logs for which rules can be set will not contain each application exceptions logged since detailed information is present in the application logs. You can check the available job history server logs and resource manager logs available to check if the logged in information during application run time helps serve your purpose.
... View more
06-30-2020
05:50 AM
@Mondi What are these properties set to in your yarn configurations? 'yarn.app.mapreduce.am.staging-dir' 'yarn.app.mapreduce.am.staging-dir' Also, can you try running below query before your hive query and share results > set hive.exec.scratchdir=/tmp/mydir; Regards, Paras
... View more
06-30-2020
05:44 AM
@AnjaliRocks Can you please share your workflow.xml to understand exactly how you are trying to execute the operations? What is the behaviour you observe when you run insert overwrite query from beeline? You can use the below query to set the hive scratch directories to set the temporary output path. set hive.exec.scratchdir=/tmp/mydir; Let me know if this helps.
... View more
06-29-2020
03:15 AM
@Mondi You can set the hive scratch directories to modify the intermediate data storage location from default /tmp hive.exec.local.scratchdir- Local Directory where Hive stores jars and data when performing a MapJoin optimization. If not specified, Hive uses a default location. hive.exec.scratchdir Directory in HDFS where Hive writes intermediate data between MapReduce jobs. If not specified, Hive uses a default location. Refer https://docs.cloudera.com/documentation/enterprise/6/properties/6.3/topics/cm_props_cdh600_hive.html#concept_6.3.x_hiveserver2_props__section_advanced_props Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-29-2020
02:39 AM
@SeanU You can select the role group for the YARN role for which you want to configure log events, or search for "Rules to Extract Events from Log Files". These generate the events for the service and you can set alerts for these log events. Alert publisher can be configured to generate emails for alerts. Refer https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cm_dg_monitoring_settings.html#cmug_topic_4_12_4__section_vlc_qr3_dr Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-29-2020
02:03 AM
1 Kudo
Replied on the new thread
... View more
06-29-2020
01:56 AM
@AnjaliRocks You should consider clearing the user .Trash directory or consider increasing the disk space quota. You can try changing the 'yarn.app.mapreduce.am.staging-dir' in YARN configuration. 'yarn.app.mapreduce.am.staging-dir' is the property used to set the root HDFS directory of the staging area for users' MR2 jobs. By default this is set /user. You can change this to a different location that doesn't have any quota. YARN staging directories will be created under the new path always named after the user.
... View more
06-26-2020
07:00 AM
@AnjaliRocks It seems there are issues is removing trash from the user folder for which disk space quota is enabled leading to errors exceeding quota. What is the spark version? The support for purge was added in Spark 2.0: As a workaround you can alter table properties to enable auto purge. ALTER TABLE table_name SET TBLPROPERTIES('auto.purge'='TRUE'); Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-26-2020
06:40 AM
@Morbious Please check the Service Monitor Logs for getting details on impala query status. Yes, different timezone will impact the query visibility. If you see messages related to queries outside of acceptance window, please ensure that the timezone is same across all Impala daemons and CM. If the same CM is managing 3 clusters, you should also consider increasing the SMON heap size and check if queries are visible. You should also see slow queries messages in the SMON logs. Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-25-2020
08:52 PM
@syedshakir For existing DB, stop the hue service and alter the DB using below query: ALTER DATABASE hue CHARACTER SET utf8 ; ALTER DATABASE hue COLLATE = 'utf8_general_ci'; Start the hue service. Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-16-2020
10:40 PM
@Tar Are you looking for CDP installation guide? Refer https://docs.cloudera.com/cloudera-manager/7.0.3/installation/topics/cdpdc-installation.html Hope this helps, Paras Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
06-16-2020
10:26 PM
@Mondi The group where the admin user is present should be added as the sentry admin group. This would also enable all other users in the group to act as sentry admins. If you want to se the user alone add it to the sentry admin groups as each user has a corresponding group entry created by the same name locally. Please ensure that the user and group exists on all nodes on the cluster.
... View more