Community Articles

Hey everyone,  We showed you what Data Hub is all about in an earlier video. Now we're going to roll up our sleeves and start getting into how to use the product. In this video we cover the basic knowledge you need to provision a basic Data Hub cluster.   Let us know by adding a comment below what topics you care about and keep checking back as we will be posting new videos about CDP and more at least once a week.   More information: Cloudera Data Hub - What You Should Know https://community.cloudera.com/t5/Community-Articles/Cloudera-Data-Hub-What-You-Should-Know/ta-p/284608 Documentation on CDP https://community.cloudera.com/t5/Community-Articles/Cloudera-Data-Platform-documentation/ta-p/270222  

This video describes how to use CA Signed Certificates for Ambari Metric System deployed in distributed mode with multiples Metrics Collectors.     Open YouTube video here   Ambari Metric System (AMS) HA   Ambari Metrics System is an Ambari-native pluggable and scalable system for collecting and querying Hadoop Metrics, that includes Grafana, a powerful dashboard builder that is fully open source with a wide community adoption. By default, Metrics Collector is the REST API component that receives metrics payload as JSON over HTTP from the Sinks and Monitors. The metrics are written into the HBase storage layer which is dedicated storage for metric data and managed as a part of AMS, separate from the cluster HBase. The HBase schema is defined using Phoenix and all the read write operations from AMS are Phoenix jdbc API calls. The Sink implementations are native to AMS and are placed in the classpath of the supported Hadoop ecosystem services by Ambari. The Monitor

This video explains how to configure Spark2 to use HiveWarehouseConnector.   Open the video on YouTube here   To access Hive from Spark2 on HDP3, there are some requirements to meet and use the HiveWarehouseConnector. The configuration steps to use the HiveWarehouseConnector can be set at the Cluster level and/or Job level. This requires to collect base information from our Hive service to later on be configured on Spark2 via Ambari, or per application submission from a terminal providing the same configurations as arguments to the Spark2 client.

On HDP3, SparkSQL API will directly query Spark2 own catalog namespace. The Spark catalog is independent of the Hive catalog. Hence, a HiveWarehouseConnector was developed to allow Spark users to query Hive data through the HiveWarehouseSessionAPI. Hive tables on HDP3 are ACID by default, given that Spark2 does not operate on ACID tables yet. To guarantee data integrity, the HiveWarehouseConnector will process queries through the HiveServer2Interactive (LLAP) service. This is not the case for External tables.   This video will explain how to access Hive from Spark2 on HDP3 along with some architectural changes and the support provided for particular use cases.   Open the video on YouTube here

This video describes an easy to use Python script to generate data for Hive, based on an input table schema. This data generator for Hive solves the issue of loading data into tables with a lot of columns (such as more than 1500 columns). This automation script supports faster testing of queries and analyzing performance.To get the code, see the KB link (for customers only).   Open the video on YouTube here

This Video Describes how Kafka ACLs work in HDP. This method is not supported in CDP7, please investigate Ranger Authorization for ACLs in CDP.   Open the video on YouTube here   Apache Kafka comes with an authorizer implementation that uses ZooKeeper to store all the ACLs. The ACLs have to be set because the access to resources is limited to super users when an authorizer is configured. By default, if a resource has no associated ACLs, then no one is allowed to access the resource, except super users.The following are the main ACL commands:Add ACLs: bin/kafka-acls.sh --authorizer-properties zookeeper.connect=<zkHost>:<zkPort> --add --allow-principal User:<username> --operation All --topic <topicName> --group=* In the above command, ACLs are added to allow a principal to have All operations available over the topic specified. The following are the available operations: Read Write Create Delete Alter Describe Cluste

Many a times, it is necessary for a engineer/administrator to manipulate the content of Ambari-Infra-Solr using the command line utilities. They might or might not have access to the GUI interface.   This video helps to understand the basic manipulation of: Listing collections and checking cluster status of Solr cloud. Creating new collections. Deleting the existing collections.   To check if ambari-infra-solr server instance is running on the node, run the following: # ps -elf | grep -i infra-solr# netstat -plant | grep -i 8886 If the cluster is Kerberized. Check for valid kerberos tickets: # klist -A Obtain a kerberos ticket, if not present: # kinit -kt /etc/security/keytabs/ambari-infra-solr.service.keytab $(klist -kt /etc/security/keytabs/ambari-infra-solr.service.keytab |sed -n "4p"|cut -d ' ' -f7) List SOLR collections: curl --negotiate -u : "http://$(hostname -f):8886/solr/admin/collections?action=list" Create a collection: # curl --negotiate -u : "http://$(h

This video describes how to upgrade Ambari 2.6.2.2 to Ambari 2.7.3.   Open the video on YouTube here   Apache Ambari 2.7.3 is the latest among Ambari 2.7.x releases. Ambari 2.7.0, which was the first release in the 2.7.x series introduced significant improvements from its predecessor - Ambari 2.6.2. This video will help users upgrade from Ambari 2.6.2.2 to Ambari 2.7.3. Procedure I. Prerequisites Take a backup of the Ambari configuration file: # mkdir /root/backups# cp /etc/ambari-server/conf/ambari.properties /root/backups Turn off Service Auto Restart:  From Ambari UI: Admin > Service Auto Start. Set Auto Start Services to Disabled. Click Save. Run Service Checks on all Ambari services. On each of the Ambari services installed on the cluster, run Service Checks as the following:  From Ambari UI: <Service_Name> > Service Actions > Run Service Check For example: HDFS > Service Actions > Run Service Check. Start and Stop all of the

This video describes a step by step process for getting an HDP 3 cluster up and running on Centos 7. The video follows the Hortonworks Documentation and Support Matrix recommendations. Public repositories were used for a minimal two node install on CentOS 7.5. Services installed on Ambari node: ambari-server Services installed on node1: SmartSense, Ambari Metrics. Open the video on YouTube here Get ready Clean yum cache yum clean all Rebuild cache yum makecache  Install utilities yum install openssl openssh-clients curl unzip gzip tar wget Double-check free RAM memory in the system free -m Check limits configuration  ulimit -n -u Set limits, temporarily  ulimit -n 32768 ulimit -u 65536 Set limits, permanently vim /etc/security/limits.conf root - nofile 32768 root - nproc 65536 Generate RSA SSH key ssh-keygen Send public RSA key to node1 and configure it in the authorized keys file ssh-copy-id 10.200.82.41 Test passwordless connection ssh 10.

From Ambari 2.6, for all MYSQL_SERVER components in a blueprint, the mysql-connector-java.jar needs to be manually installed and registered. This video describes how to install and register MySQL connector to replace the embedded database instance that is by default used by Ambari Server.     Open YouTube video here   For certain services, Cloudbreak allows registering an existing RDBMS instance as an external source for a database. After registering the RDBMS with Cloudbreak, it can be used for multiple clusters. However, as this configuration needs to be used by Ambari before its installation, MySQL Connector needs to be connected the remote MySQL database. To manually install and register MySQL connector, do the following:    Preparing MySQL Database Server  Install MySQL Server on CentOS Linux 7: # yum -y localinstall https://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm # yu

Sometimes, a node needs to be decommissioned or have undetermined downtime for repairs. If the node has a Resource Manager, move it to a new host using the Resource Manager Move Wizard from Ambari Web User Interface.The Resource Manager Move wizard describes the set of automated steps to be taken to move one Resource Manager to new host. Since YARN and Mapreduce2 will be restarted, a cluster maintenance windows must be planned and be prepared for cluster downtime.   This video describes how to move a Resource Manager to a new host using the Resource Manager Move Wizard from Ambari Web User Interface.   Open the video on YouTube here   To move YARN Resource Manager to a new host with Ambari, do the following:  In Ambari Web, browse to Services > YARN > Summary. Select Service Actions and choose Move ResourceManager. The Move ResourceManager wizard launches, describing a set of automated steps t

If an active Resource Manager in a cluster fails, to ensure that another Resource Manager is available, the Resource Manager high availability (HA) should be enabled and configured.   In HDP 2.2 or later environment, high availability (HA) can be configured for ResourceManager by using the Enable ResourceManager HA wizard. To ensure this, there must be at least three hosts in the cluster and Apache ZooKeeper servers should be running.   The Enable ResourceManager high availability section from the documentation contains the steps mentioned in this video.   Open the video on YouTube hereRecommended links: Product documentation page Community Forum

To ensure that another NameNode in a cluster is always available when an active NameNode host fails, the NameNode high availability should be enabled and configured on the cluster from the Ambari Web User Interface. This video explains how to  enable HA wizard and the steps that must be followed to set up NameNode high availability.   Open the YouTube video here As a prerequisite, ensure the following: If the HDFS or ZooKeeper services are in Maintenance Mode the NameNode HA wizard will not complete successfully. HDFS and ZooKeeper must be stopped and started when enabling NameNode HA as the Maintenance Mode will prevent those start and stop operations from occurring. The Enable NameNode high availability section from the documentation contains the steps mentioned in this video.Recommended links: Product documentation page Community Forum

Apache Ranger is one of the easiest, robust and flexible framework to manage the authorization for different components of a cluster. However, if your policies are not syncing, that may become an issue.   In this video, we will be troubleshooting the Ranger policy synchronization. We will look into the components that are involved, the components that interact between them and a few other most common issues.   The video is to understand what are the main factors to check on a policy synchronization issue and how to solve them.   Open the video on YouTube here

The Zookeeper transaction logs and Snapshots files are not human readable by default. Running a cat command on these files do not give clear information on the content of the files.   The following video explains how to read Zookeeper transaction logs and Snapshots?   Open the video on YouTube here   To view the content of these files, use the following:To read the snapshots: java -cp /usr/hdp/current/zookeeper-server/zookeeper.jar:/usr/hdp/current/zookeeper-server/lib/* org.apache.zookeeper.server.SnapshotFormatter <Snapshot file name> To read the transaction logs: java -cp /usr/hdp/current/zookeeper-server/zookeeper.jar:/usr/hdp/current/zookeeper-server/lib/* org.apache.zookeeper.server.LogFormatter <Log file name> The classes that need to be used are located under /usr/hdp/current/zookeeper-server and /usr/hdp/current/zookeeper-server/lib.

This video article provides the steps on how to use Reassign Partitions Tool:   Open YouTube video here   Create a file named topics-to-move.json with the following content: {"topics": [{"topic":"<partitionName>"}], "version":1} Run the following command: ./kafka-reassign-partitions.sh --zookeeper master:2181 --topics-to-move-json-file topics-to-move.json --broker-list "<brokerID>" --generate Take the output from previous Proposed partition reassignment configuration and create another json file reassign-partition.json. Run the following command:  ./kafka-reassign-partitions.sh --zookeeper master:2181 --reassignment-json-file reassign-partition.json --execute

At times, Kafka Brokers can find one of its log directory utilization at 100% and the broker process would fail to start.   This article provides the instructions to manually move partition data between different log directories within a Kafka Broker.   Open the video on YouTube here  The Kafka brokers maintain two offset checkpoint files inside each log directory: replication-offset-checkpoint recovery-point-offset-checkpoint And both these files have the following format: (a) 1st line: Version Number(b) 2nd line: Number of topic-partition entries in the file(c) All the remaining lines: Replication Offset/Recovery Point Offset, for every partition data maintained within the current log directory.

Ambari Infra Solr can be present on HDP and HDF clusters. HDPSearch Solr is a separate product that can be installed on top of an HDP cluster.   This video talks about the goals and differences between them and how to correctly select the product category when opening a support case.    Open YouTube video here

This articles describe the steps required to complete the setup for accessing Grafana using HTTPS with CA Signed Certificates.   Open YouTube video here     Ambari Metrics System includes Grafana, which is a daemon that runs on a specific host in the cluster and serves pre-built dashboards for visualising metrics collected in the Metrics Collector. For this article the following servers is used: 172.25.33.152 c3132-node1.user.local (Ambari Server) 172.25.36.9   c3132-node2.user.local (Ambari Metrics Collector + Grafana) 172.25.40.27  c3132-node3.user.local (Ambari Metrics Collector) 172.25.33.163 c3132-node4.user.local By default, Grafana listen on  port TCP/3000: # for i in $(netstat -utnlp | awk '/grafana/ {print substr($7, 1, length($7)-13)}' | sort -u) ; do echo ; ps -eo pid,user,command --cols 128 | grep $i | grep -v grep ; netstat -utnlp | grep $i ; echo ; done270925 ams /usr/lib/ambari-metrics-grafana/bin/grafana-

This video explains how to configure Ambari Metrics System AMS High Availability.   Open YouTube video here   To enable AMS high availability, the collector has to be configured to run in a distributed mode. When the Collector is configured for distributed mode, it writes metrics to HDFS, and the components run in distributed processes, which helps to manage CPU and memory.   The following steps assume a cluster configured for a highly available NameNode. Set the HBase Root directory value to use the HDFS name service instead of the NameNode hostname. Migrate existing data from the local store to HDFS prior to switching to a distributed mode. To switch the Metrics Collector from embedded mode to distributed mode, update the Metrics Service operation mode and the location where the metrics are being stored. In summary, the following steps are required: Stop Ambari Metrics System Prepare the Environment to migrate from Local File System to HDFS Migrate Collector Dat

In Kafka, sometimes the topics are marked for deletion. However, even after restarting brokers, the topics are not deleted. This video explains the steps to delete a topic manually. Open the video on YouTube here

This video provides the steps to configure Kafka Mirror Maker in Kerberized clusters. Open YouTube video here   To run Kafka Mirror Maker using Kerberos clusters, do the following: Environment: Cluster A (Source-Kerberized): c189-node2.squadron-labs.com Broker1c189-node3.squadron-labs.com Broker2c189-node4.squadron-labs.com Broker3 Cluster B (Destination-Kerberized): c289-node2.squadron-labs.com Broker1c289-node3.squadron-labs.com Broker2c289-node4.squadron-labs.com Broker3 * In this example, both environments are sharing same KDC and it was created on specific principal for the same. Destination Files  consumer.propertiesbootstrap.servers=<brokerSourceHost>:<brokerPort>,<brokerSourceHost>:<brokerPort>group.id=<consumerGroupName>security.protocol=PLAINTEXTSASL producer.propertiesbootstrap.servers=<brokerDestinationHost>:<brokerPort>,<brokerDestinationHost>:<brokerPort>key.serializer=org.apache.kafka.common.serialization.

This video explains steps to install the required support for Intel® Intelligent Storage Acceleration Library on a Hortonworks Data Platform version 3 cluster, over Linux CENTOS 7.   Open the video on YouTube here   ISA-L is a collection of optimized low-level functions targeting storage applications.It is developed by Intel, and its name means Intel(R) Intelligent Storage Acceleration Library. ISA-L includes the following: Erasure codes - Fast block Reed-Solomon type erasure codes for any encode/decode matrix in GF(2^8). CRC - Fast implementations of cyclic redundancy check. Six different polynomials supported. iscsi32, ieee32, t10dif, ecma64, iso64, jones64. Raid - Calculate and operate on XOR and P+Q parity found in common RAID implementations. Compression - Fast deflate-compatible data compression. De-compression - Fast inflate-compatible data compression. To check for ISA-L library, do the following: $ hadoop checknative ISA-L: false Loading ISA-L failed: Fai

This video explains step by step instructions for protecting HDFS directories from unintended deletions.   There are times when Hadoop admins need to protect directories from accidental deletions, even from admins themselves. This is a guide for accomplishing that, so sensitive directories cannot be deleted by mistake and extra procedures have to be performed and double-checked before actually deleting those specific folders.   Open the video on YouTube here

The video provides the steps to connect to the Kafka server using SASL_SSL protocol.   Open the video on YouTube here   To connect to Kafka server using SASL_SSL protocol using one way SSL, do the following: Server side Configure the following properties in Ambari server > Kafka > config > Custom kafka-broker. ssl.keystore.location=path-to-your-keystoressl.keystore.password=keystore-passwordssl.truststore.location=path-to-your-truststoressl.truststore.password=keystore-password Under Ambari server > Kafka > config > Kafka Broker > Listeners can add the security protocol as: SASL_SSL://localhost:<port> Since this is a one way SSL communication between client-server, ensure to have enabled the property ssl.client.auth=none. This means that a client authentication is not required. By default this property is set to None. This can be double checked from Am