Community Articles

1. Install MySQL data Directories on a Non-root partition (Not /var/lib) 2. Create a dedicated least privileged account for Mysql deamon3. Disable MySQL Command History, Command History may contain passwords which is viewable by other users4. Disable interactive Login5. Disable login from nodes other than the those used by hive services6. Provide only hive user permission to the Hive metadata database within MySQL7. During installation do not specifify passwords in command line8. Ensure Data Directories for Mysql has appropriate permissions and ownerships9. Ensure only DBA administrators have full database access10. Ensure that database logging is enabled for error logs and log files are maintained on a non system partition11. Ensure that old passwords is not set to 1 12. Ensure secure_auth is set to ON13. Consider if your Component can work with the MYSQL "Connect using SSL" option

Objective The objective of this tutorial is to walk you through process of updating your Cloudbreak Deployer on Amazon AWS from 1.3 to 1.4. Once this is complete, you can now deploy an HDP 2.5 TP cluster.   Prerequisites You should have deployed Cloudbreak 1.3 on Amazon AWS using the instructions found here: Cloudbreak Documentation 1.3 - AWS. You should add TCP port 443 to the security group on Amazon AWS as Cloudbreak 1.4+ appears to proxy requests from port 443 now. Do not run cbd start.   Note: This process will update the Cloudbreak Deployer to the latest available version. At the time of initial writing, this was 1.4. As of September 1, 2016, the latest version is 1.6. Steps 1. Connect to your Cloudbreak Amazon AWS instance You should have access to your key file from amazon. Log into your Cloudbreak deployer instance using: ssh -i <amazon key file> cloudbreak@<amazon instance public ip> Note: If you have permission issues connecting via ssh, make sure you

- One approach can be to check the Ambari Blueprint to see what changes are made by comparing last two results. Following approach can be integrated inside a shell script (cronjob) to poll the config changes at some regular interval and to find the "diff" between current and last output files "/tmp/cluster-`date +'%Y%m%d%H%M'`.txt"Example: curl -o "/tmp/cluster-`date +'%Y%m%d%H%M'`.txt" -u admin:admin -H "X-Requested-By: ambari" -X GET http://c6401.ambari.apache.org:8080/api/v1/clusters/ClusterDemo?format=blueprint .- Users can also check the output of below REST call to browse "service_config_versions"curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET http://c6401.ambari.apache.org:8080/api/v1/clusters/ClusterDemo/configurations/service_config_versions.Above would list the service config versions of all services. At any time users make the above call, it would provide the complete history of service config changes. For more information on this please refer to: https://cwiki.

Maintenance Mode supports suppressing alerts and skipping bulk operations for specific services, components and hosts in an Ambari-managed cluster.There are many scenarios when we may want to turn on Maintenance Mode specially when we are performing hardware or software maintenance, changing configuration settings, troubleshooting, decommissioning, or removing cluster nodes. In such scenario we can place a service, component, or host object in Maintenance Mode before we start to perform necessary maintenance or troubleshooting tasks. In maintenance mode following things happen: 1. It causes suppression of alerts, warnings and status change indicators generated for the object. 2. It exempts an object from host-level or service-level bulk operations. Although it can be done from the ambari UI, However here in this article we will see how to manage the maintenance mode for some Services like HBase using the APIs.Turning ON Maintenance Mode for any specific component (like: region server o

Many times it is needed to monitor the ambari memory usage. JConsole and JVisualVM are the JDK provided JMX client utilities that can be used for the same. Here in this demo we will see how to monitor ambari server's JVM memory using $JAVA_HOME/bin/jvisualvm (same steps will be applicable for jconsole as well)Step-1). We will need to add the "com.sun.management.jmxremote" related JVM remoted jmx options to the "/var/lib/ambari-server/ambari-env.sh" as following.export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=4444 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false'For more information on JVMs standard jmxremote options please refer to : http://docs.oracle.com/javase/7/docs/technotes/guides/management

Steps to configure ambari-server to archive log files.1. Open /etc/ambari-server/conf/log4j.properties file,log4j.appender.file=org.apache.log4j.RollingFileAppenderlog4j.appender.file.File=${ambari.log.dir}/${ambari.log.file}log4j.appender.file.MaxFileSize=80MBlog4j.appender.file.MaxBackupIndex=60log4j.appender.file.layout=org.apache.log4j.PatternLayoutlog4j.appender.file.layout.ConversionPattern=%d{DATE} %5p [%t] %c{1}:%L - %m%ntolog4j.appender.file=org.apache.log4j.rolling.RollingFileAppenderlog4j.appender.file.rollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy log4j.appender.file.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy log4j.appender.file.File=${ambari.log.dir}/${ambari.log.file}log4j.appender.file.triggeringPolicy.MaxFileSize=10485760log4j.appender.file.layout=org.apache.log4j.PatternLayoutlog4j.appender.file.layout.ConversionPattern=%d{DATE} %5p [%t] %c{1}:%L - %m%nlog4j.appender.file.rollingPolicy.FileNamePattern=${ambari.log.dir}/${amba

This has been observed for Ambari 1.7. I know Ambari 1.7 is very old now however if some people are still using it and facing same issue then this post can save your lot of time! 🙂.We know that Ambari 1.7 has Ganglia for reporting metrics..Our issue was, we were unable to get service metrics for YARN service, below was the exact scenario:1. In Ganglia UI, I was able to get graphs for the YARN Metrics.2. In Ambari UI, I was able to see metrics for other services like HDFS etc.3. This issue was observed on one of our customer's cluster.4. I did setup same cluster on my local system however I was not able to reproduce this issue.5. Only difference in customer's and my cluster was, customer was having RM HA and I had installed single node instance.6. There was no error while getting metrics, please see below screenshot when metrics were not visible..How to troubleshoot:1. Click on any of the metrics, say 'Nodemanagers' --> This will open graph in big window.2. Open the developer tools

Apache Knox provides a gateway to HiveServer2 which can be used to proxy connections to Hive from BI tools like Tableau Desktop. With a secure cluster HiveServer2 is kerberized, which usually means the client must have a valid TGT in their ticket cache in order to authenticate to HiveServer2.The Knox gateway can simplify this in many environments, by supporting client authentication to kerberized Hive without a TGT. The client authenticates to Knox via, say, LDAP authentication, and HiveServer2 trusts Knox to proxy connections on behalf of this user. Another benefit of using Knox is that client doesn't need to know where HiveServer2 is hosted, so if an administrator needs to move it to another master node, this is transparent to the applications connecting via Knox.In the environment for this walkthrough, I have Tableau Desktop 10.0.1 installed, and I'll be using HDP 2.5. The first step is to ensure you have the latest Hortonworks ODBC Driver for Apache Hive, 2.1.5 at the time of this

Sqoop OverviewSqoop is a tool designed to transfer data between Hadoop and relational databases or mainframes. You can use Sqoop to import data from a relational database management system (RDBMS) such as MySQL or Oracle or a mainframe into the Hadoop Distributed File System (HDFS), transform the data in Hadoop MapReduce, and then export the data back into an RDBMS.Sqoop automates most of this process, relying on the database to describe the schema for the data to be imported. Sqoop uses MapReduce to import and export the data, which provides parallel operation as well as fault tolerance.Sqoop Performance Tuning Best PracticesTune the following Sqoop arguments in JDBC connection or Sqoop mapping to optimize performance batch•split-by and boundary-query•direct•fetch-size•num-mapper•2.Inserting Data in BatchesSpecifies that you can group the related SQL statements into a batch when you export data. The JDBC interface exposes an API for doing batches in a prepared statement with multiple

Using SQOOP with MySQL as metastoreTo set up MySQL for use with SQOOP: On the SQOOP Server host, install the connector. Install the connector RHEL/CentOS/Oracle Linux yum install mysql-connector-java SLES zypper install mysql-connector-java Confirm that .jar is in the Java share directory. ls /usr/share/java/mysql-connector-java.jar Make sure the .jar file has the appropriate permissions - 644. Create a user for SQOOP and grant it permissions. For example, using the MySQL database admin utility: # mysql -u root -p CREATE USER '<SQOOPUSER>'@'%' IDENTIFIED BY '<SQOOPPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SQOOPUSER>'@'%'; CREATE USER '<SQOOPUSER>'@'localhost' IDENTIFIED BY '<SQOOPPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SQOOPUSER>'@'localhost'; CREATE USER '<SQOOPUSER>'@'<SQOOPSERVERFQDN>' IDENTIFIED BY '<SQOOPPASSWORD>'; GRANT ALL PRIVILEGES ON *.* TO '<SQOOPUSER>'@'<SQOOPSERVERFQDN>'; FLUSH PR

Steps to connect from a client machine (MAC in this case) to Hadoop cluster using hive JDBC. Here are couple of links that i have used to build this out Link that talks about hive drivers and jar files https://streever.atlassian.net/wiki/pages/viewpage.action?pageId=4390924 Link that talks about how to setup java jdbc setup for hive jar files https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients1.Here are the jar files you need to connect to HS2 For HDP 2.3 You'll only need 2 jar files for the JDBC Client with HDP 2.3 # From /usr/hdp/current/hive-client hive-jdbc.jar (should be a symlink to the hive-jdbc-xxx-standalone.jar) # From /usr/hdp/current/hadoop-client hadoop-common.jar (hadoop-common-....jar) 2. Make sure that java home is set on your machine This is value that i have on my machine for JAVA_HOME echo $JAVA_HOME /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home 3. Move these jar files to java library directory on my machine /Library/Java/Ex

IntroductionBy default, Zookeeper runs without the option of becoming a superuser to administrate znodes in the ZK ensemble, for example, to fix ACLs, remove znodes that are not required anymore, or create new ones in specific locations. Zookeeper grants permissions through ACLs through different schemas or authentication methods, such as 'world', 'digest', or 'sasl' if we use Kerberos. We can potentially we locked out if we were to grant everyone just read permissions to a znode, as we would not be able to delete it or modify it anymore.Example of the problemFor example, we connect to Zookeeper through zkCli:/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server `hostname -f`:2181[zk: sandbox.hortonworks.com:2181(CONNECTED) 1] getAcl /config/topics'world,'anyone: r If we need to modify that znode so that, for example, user 'kafka' can have access to it to create new topics:[zk: sandbox.hortonworks.com:2181(CONNECTED) 2] setAcl /config/topics world:anyone:r sasl:kafka:cdrwaAuthenticati

ObjectiveIf you are managing multiple copies of the HDP sandbox for Docker (see my article here:How to manage multiple copies of the HDP Docker Sandbox.), you may find yourself running out of storage within your Docker VM image on your laptop. There is a way to increase the available storage space of the Docker VM image. Increasing the storage space will allow you to have more copies of sandbox containers in addition to other images and containers.This tutorial will guide you through the process of increasing the size of the base Docker for Mac VM image. We will increase the size from 64GB to 120GB. This tutorial is the second in a two part series. The first tutorial in the series is: How to move Docker for Mac vm image from internal to external hard drive.For a tutorial on increasing the base Docker VM image on CentOS 7, read my tutorial here: How to modify the default Docker configuration on CentOS 7 to import HDP sandboxPrerequisitesYou should have already completed the following tu

In previous post we have seen how to Automate HDP installation with Namenode HA using Ambari Blueprints. In this post, we will see how to Deploy single node HDP Cluster with Kerberos authentication via Ambari blueprint.Note - For Ambari 2.6.X onwards, we will have to register VDF to register internal repository, or else Ambari will pick up latest version of HDP and use the public repos. please see below document for more information. For Ambari version less than 2.6.X, this guide will work without any modifications.Document - https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-release-notes/content/ambari_relnotes-2.6.0.0-behavioral-changes.html.Below are simple steps to install HDP single node cluster with Kerberos Authentication(MIT KDC) using internal repository via Ambari Blueprints..Step 1: Install Ambari server using steps mentioned under below linkhttp://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Installing_HDP_AMB/content/_download_the_ambari_repo_lnx6.h

SYMPTOM: While performing step for Namenode HA, on the step of 'hdfs namenode -initializeSharedEdits' it failed with below error - ERROR: [root@localhost conf]# sudo su hdfs -l -c 'hdfs namenode -initializeSharedEdits'16/11/22 09:43:56 INFO namenode.NameNode: STARTUP_MSG:/************************************************************STARTUP_MSG: Starting NameNodeSTARTUP_MSG: host = locahost/10.200.206.12STARTUP_MSG: args = [-initializeSharedEdits]STARTUP_MSG: version = 2.7.1.2.3.0.0-2557STARTUP_MSG: classpath = /usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/commons-cli-1.2.jar:/us r/hdp/2.3.0.0-2557/hadoop/lib/xmlenc-0.52.jar:/usr/hdp/2.3.0.0-2557/hadoop/lib/jsch-0.1.42.jar:/usr/hdp/2.3.0.0-2557/ ...STARTUP_MSG: build = git@github.com:hortonworks/hadoop.git -r 9f17d40a0f2046d217b2bff90ad6e2fc7e41f5e1; compiled by 'jenkins' on 2015-07-14T13:08ZSTARTUP_MSG: java = 1.8.0_51************************************************************/16/

To post random data to Azure Event hubs using HTTP, use a generate flowfile processor to first generate the random data. Connect this to an UpdateAttribute processor to add your SAS (Share Access Signature) as a new attribute called "Authorization" to the flow file's attributes. Finally use PostHTTP to submit the data to Event HubsPlace the following processors on the pallet and set the run schedule of the generate flowfile processor to something reasonable like 10 seconds.Configure Update Attribute, adding the "Authorized" property with a SAS token. For more information on creating a SAS token see https://community.hortonworks.com/articles/69823/creating-shared-access-signature-sas-for-posting-d.htmlConfigure the PostHTTP to post the URL https://eventhub-nifi.servicebus.windows.net/hub-nifi/messagesNote: I set the content type to text/plain you should set this to suit your use case.Only the truststore needs to be configured for this example as we only need to trust the server certific

ISSUE : Ranger HA fails if admin password is changed to non default passwordError stack :Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 208, in <module> RangerAdmin().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 59, in install setup_ranger_admin_passwd_change() File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 598, in setup_ranger_admin_passwd_change Execute(cmd, environment={'JAVA_HOME': params.java_home, 'RANGER_ADMIN_HOME': params.ranger_home}, user=params.unix_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_manag

ISSUE : Ranger HA fails if admin password is changed to non default passwordError stack :Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 208, in <module> RangerAdmin().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py", line 59, in install setup_ranger_admin_passwd_change() File "/var/lib/ambari-agent/cache/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 598, in setup_ranger_admin_passwd_change Execute(cmd, environment={'JAVA_HOME': params.java_home, 'RANGER_ADMIN_HOME': params.ranger_home}, user=params.unix_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_manag

HDP 2.5.3.0 and Ambari 2.4.2.0 with SmartSense 1.3.1 is now available for immediate use.Being maintenance releases, there are no features introduced. These releases contain bug fixes. HDP 2.5.3.0 has about 300 bugs since HDP 2.5.0.0. Ambari 2.4.2.0 contains about 140 fixes since Ambari 2.4.0.1 was released.Please consult the HDP 2.5.3.0 documentation and release notes. Also the Ambari 2.4.2.0 documentation and release notes.Thank you

Quick tips on how to find low level hardware performance stats. I use it often for NiFi/Spark/Hadoop. This is not limited to those use services. Additionally, this is not a exhaustive list nor am I advocating one tool over the another. Just a few I have ran over the years during my implementations/POCs experience. These give me insights whether I have allocated enough physical resources to run the services. I highly recommend not assuming what your hardware can or can't do. Benchmark it! How? Read my article here.Lets get to it.CPU stats iostat -c 1 3 will provide you cpu stats every 1 second 3 times. Output of the report (sourced right from here😞 CPU Utilization ReportThe first report generated by the iostat command is the CPU Utilization Report. For multiprocessor systems, the CPU values are global averages among all processors. The report has the following format:%userShow the percentage of CPU utilization that occurred while executing at the user level (application).

After completing Ambari upgrade from Ambari 1.7 to Ambari 2.0.2, Ambari failed to start because of below error(s):[root@xyz ~]# service ambari-server start Using python /usr/bin/python2.6 Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log Waiting for server start.................... ERROR: Exiting with exit code -1. REASON: Ambari Server java process died with exitcode 255. Check /var/log/ambari-server/ambari-server.out for more information. .If you check /var/log/ambari-server/ambari-server.out logs, you get below error(s):[root@xyz ~]# cat /var/log/ambari-server/ambari-server.out [EL Warning]: metadata: 2016-12-01 10:01:23.436--ServerSession(306824177)--The reference column name [resource_type_id] mapped on the element [field perm

This article describes how to launch Apache NiFi 1.0.0 on docker. To launch Apache NiFi 1.1.0 on docker go here.During my development of the Json2CSV processor here, I quickly found a need for an environment to test my processor. I don't want to build and install NiFi from my laptop since I need all my applications isolated from each other for ease of maintenance. Docker to the rescue! Similar to how I launch a PyCharm IDE from a docker image here which will rerender back to my laptop. Isolation! I like to keep it simply. Put everything in a Dockerfile and allow myself to quickly launch a NiFi Docker image. Here are the steps to get you up and runningPrerequisitesDownload latest virtualbox from here.To run docker containers or build images a docker machine is required. Download docker machine from here.First pull the prebuilt and complied docker image (https://hub.docker.com/r/sunileman/dockernifi/) by running this command:docker pull sunileman/dockernifiNow you have the docker

There are many ways to run Hadoop on virtual machines. Earlier this year I tried several approaches, and ended up using a helpful Quick Start Guide written by Yusaku Sako. The Quick Start uses VirtualBox, Vagrant, and predefined scripts to set up a multi-node HDP cluster. You can choose which version of Ambari to install, and then choose and install an associated version of the HDP stack. For anyone new to virtual machines, there is now a Quick Start for New VM Users. The extended version adds background information and additional details for installing Ambari and the HDP stack. Topics include: Terminology Prerequisites Installing VirtualBox and Vagrant Starting Linux Virtual Machines Accessing Virtual Machines Installing Ambari Installing the HDP Stack Troubleshooting Reference information for basic Vagrant commands

INTRODUCING THE HORTONWORKS CONNECTED DATA CLOUD TECHNICAL PREVIEW To this end, we are introducing the “ Hortonworks Connected Data Cloud” Technical Preview. This Technical Preview gives you a way to quickly spin up Apache Hive and Apache Spark clusters that are ready to run ephemeral workloads in your Amazon Web Services (AWS) environment.Source Step 1 Follow the video in this http://hortonworks.github.io/hdp-aws/launch/ Step 2 Create and Terminate a cluster. Ambari Services starting Install Completed GUI access different views

Over the last year, Oracle has continued to update and add support for Hortonworks HDP. Below is a list of products which support using Hortonworks HDP 2.5.0.0Big Data SQL https://www.oracle.com/database/big-data-sql/index.htmlBig Data Connector https://www.oracle.com/database/big-data-connectors/certifications.html Includes -Oracle SQL Connector for HDFS-Oracle Loader for Hadoop-Oracle Data Integrator-Oracle XQuery for Hadoop-Oracle R Advanced Analytics for Hadoop-Oracle Datasource for HadoopSpatial and Graph https://www.oracle.com/database/spatial/index.htmlGoldenGate for Big Data https://www.oracle.com/middleware/data-integration/goldengate/big-data/index.htmlOracle Data Integrator Enterprise Edition https://www.oracle.com/middleware/data-integration/enterprise-edition-big-data/index.htmlBig Data Discovery https://www.oracle.com/big-data/big-data-discovery/index.html