Member since
01-19-2017
3681
Posts
633
Kudos Received
373
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1826 | 06-04-2025 11:36 PM | |
| 2253 | 03-23-2025 05:23 AM | |
| 1073 | 03-17-2025 10:18 AM | |
| 4195 | 03-05-2025 01:34 PM | |
| 2816 | 03-03-2025 01:09 PM |
04-26-2019
01:26 AM
@Geoffrey Shelton Okot Thank you. I think Decommissioning is a best way to do this. Do I need to turn on maintainance mode while I do decommissioning datanode? Also Do I need to stop the services like - NodeManager, Ambari metrics for that datanode after decommissioning?
... View more
04-23-2019
08:31 PM
Very intresting question!
... View more
04-30-2019
11:04 AM
@Naveenraj Devadoss What was the solution? Did you update the network config in /etc/host?
... View more
04-15-2019
04:47 AM
@Sandeep R It seems to be an SSL issue can you validate your LDAP, the port 636 is LDAPS and 389 is for LDAP. To enable LDAPS, you must install a certificate that meets the following requirements: The LDAPS certificate is located in the Local Computer's Personal certificate store (programmatically known as the computer's MY certificate store). A private key that matches the certificate is present in the Local Computer's store and is correctly associated with the certificate. The private key must not have strong private key protection enabled. The Enhanced Key Usage extension includes the Server Authentication (1.3.6.1.5.5.7.3.1) object identifier (also known as OID). The Active Directory fully qualified domain name of the domain controller (for example, DC01.DOMAIN.COM) must appear in one of the following places: The Common Name (CN) in the Subject field. DNS entry in the Subject Alternative Name extension. The certificate was issued by a CA that the domain controller and the LDAPS clients trust. Trust is established by configuring the clients and the server to trust the root CA to which the issuing CA chains. You must use the Schannel cryptographic service provider (CSP) to generate the key. Hope that helps
... View more
03-04-2019
02:20 PM
1 Kudo
@harish Prerequisite: MUST DO If you want to copy a hive table across to another REALM you need to setup cross-realm trust between two MIT KDC's.This will enable the destination REALM user to have a valid Kerberos ticket to run operations on the source Cluster. Having said that, you should forget to revise your ranger policies to reflect the new REALM access privilege if the Ranger plugin has been enabled in the source cluster which I assume is the case to leverage the Ranger authorization. Here is a link to an HCC document that could help you set up the REALM trust PROCEDURE Follow the steps below to migrate a Hive database from one cluster to another: 1. Install Hive on the new cluster and make sure both the source and destination clusters are identical. 2. Transfer the data present in the Hive warehouse directory (/user/hive/warehouse) to the new Hadoop cluster. hadoop distcp <src> <dst> 3. Take a backup of the Hive Metastore. mysqldump hive > /tmp/mydir/backup_hive.sql 4. Install MySQL on the new Hadoop cluster. 5. Open the Hive MySQL-Metastore dump file and replace the source NameNode hostname with the destination hostname. hdfs://ip-address-old-namenode:port ---> hdfs://ip-address-new-namenode:port 6. Restore the edited MySQL dump into the MySQL of new the Hadoop cluster. mysql hive < /tmp/mydir/backup_hive.sql 7. Configure Hive as normal and perform the Hive schema upgrade if needed Impact Hive metadata contains the information about the database objects, and the contents are stored in the Hadoop Distributed File System (HDFS). Metadata contains HDFS URI and other details. Therefore, if you migrate Hive from one cluster to another cluster, you have to point the metadata to the HDFS of the new cluster. If you don't do this, it will point to the HDFS of the older cluster and the migration will fail. In case of any failure, initialize the Hive Metastore of the destination cluster and resume the migration following the correct steps. /bin/schematool -initSchema -dbType mysql On CDH If you are on Cloudera then you can proceed using Backup and Disaster recovery procedure HTH
... View more
02-12-2019
03:14 PM
@Michael Bronson Just create the home directory as follows # su - hdfs
$ hdfs dfs -mkdir /user/slider
$ hdfs dfs -chown slider:hdfs /user/slider That should be enough .. good luck
... View more
02-08-2019
02:58 PM
The below steps describe how to change the Namenode log level while logged on as hdfs with the below steps, without the need to restart the namenode Get the current log level $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange
SubmittedLogName:BlockStateChange
LogClass: org.apache.commons.logging.impl.Log4J
LoggerEffectiveLevel: INFO Change to DEBUG $ hadoop daemonlog -setlevel {namenode_host}:50070BlockStateChange DEBUG Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=DEBUG
SubmittedLogName:BlockStateChange
LogClass: org.apache.commons.logging.impl.Log4J
LoggerSubmittedLevel: DEBUG
SettingLevel to DEBUG ...
EffectiveLevel: DEBUG Validate DEBUG mode $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange
SubmittedLogName:BlockStateChange
LogClass: org.apache.commons.logging.impl.Log4J
LoggerEffectiveLevel: DEBUG You should be able to notice the logging level in namenode.log has been updated, without restarting the service. After finishing your diagnostics you can reset the logging level back to INFO Reset to INFO $ hadoop daemonlog -setlevel {namenode_host}:50070BlockStateChange INFO Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=INFO
SubmittedLogName:BlockStateChange
LogClass: org.apache.commons.logging.impl.Log4J
LoggerSubmittedLevel: INFO
SettingLevel to INFO ...
EffectiveLevel: INFO Validate INFO $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange
SubmittedLogName:BlockStateChange
LogClass: org.apache.commons.logging.impl.Log4J
LoggerEffectiveLevel: INFO Happy hadooping !!!!
... View more
Labels:
02-10-2019
10:14 PM
1 Kudo
@Michael Bronson HWX doesn't recommend upgrading an individual HDP component because one never knows the incompatibilities that could impact the other components and component selective upgrades tend to be a nightmare during a version upgrade The lastest HDP Kafka version is 11-2.1.x delivered by HDP 3.1 but ASF has its own rollout version and naming convention HTH
... View more
02-08-2019
08:45 AM
@ram sriram If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer. Can you tag me for "Can you please send me a document for Ambari installation on ubuntu thread" so I see the information you already received.
... View more