Community Articles

Changing Ambari Service name (via Rest API) using a customized single script: Create a property file (ambServActNmChg.properties) with following parameters :export AMBARI_ADMIN_USERID=adminexport AMBARI_ADMIN_PASSWORD=adminexport AMBARI_SERVER=<Ambari Host>export AMBARI_SERVER_PORT=<Ambari port>export CLUSTER_NAME=<Ambari Cluster>export NEW_SRV_USER_NAME=<New User Suffix>Create shell script : #!/bin/bashecho " ****** Starting Script to Change Service User names in Ambari ***** "######################################################################################################### This script changes the service account user names in ambari using inputs from properties files. ##########################################################################################################. `dirname ${0}`/ambServActNmChg.propertiescurr_date=`date +"%Y%m%d_%H%M%S"`function check_ambari_server() { echo " Check and Start Ambari Server : " server_start=`ambari-server

GoalCreate a new Ambari view for Hive Interactive. Use this link to get detailed information on configuring views.https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-views/content/settings_and_cluster_configuration.htmlSteps Navigate to Ambari page with admin privileges and click on the username dropdown icon Select the views link to explore all the views available in Ambari Collapse the "Hive" dropdown and click on "Create Instance" to create a new view for LLAP/Interactive Give the name of this instance per your requirement Ensure that under "Settings" tab, "User Interactive Mode" is set to true If the cluster is kerberized, use proper auth method and principal name Also update the proper JDBC URL with principal nameNOTE: If ranger is enabled, ensure that the user trying to access the database objects does have the permissions to browse the contents of the database(s).

This is the process to load balance zookeeper's behind a haproxy LB and the can be used for any other components as well .1) Install HAproxyyum install haproxy2) Edit /etc/haproxy/haproxy.cfg and append the below config #This specifies port which the end users will use to access zookeeper. listen appname 0.0.0.0:80 #Specifies the access method tcp/http/udp , should be tcp for zookeeper mode tcp #Algorithm used for balancing requestsbalance roundrobin #It forwards the client ip requesting the service to zookeeper option forwardfor #The zookeeper server list and port server mshaik-2 172.26.81.203:2181 check server mshaik-3 172.26.81.204:2181 check server mshaik-4 172.26.81.206:2181 check3) Reload haproxy config filehaproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)4) Test the connection through beelinebeeline> !connect jdbc:hive2://mshaik-1.htworks.com:80/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2Connectin

A Machine Learning Model learns from data. As you get new incremental data, the Machine Learning model needs to be upgraded. A Machine Learning Model factory ensures that as you have deployed model in production, continuous learning is also happening on incremental new data ingested in the Production environment. As deployed ML Model's performance decays, a new trained and serialized model needs to be deployed. An A/B test between the deployed model and the newly trained model can score them to evaluate the performance of the deployed model versus the incrementally trained model. In order to build a Machine Learning Model factory, we have to establish a robust road to production, first. The foundational framework is first to establish three environments: DEV, TEST and PROD. 1- DEV - A development environment where the Data Scientists have their own data puddle in order to perform data exploration, profile the data, develop the machine learning features from the data, build the model,

The preferred configuration for integrating with Active Directory is to use a standalone KDC and create a cross realm trust. I have done several of these deployments on physical hardware. Recently I built a test system on our Open Stack lab cluster using a small instance for the KDC. I followed the instruction in the HDP Security guide for configuring a KDC. When I created the database I noticed that the krb5util create –s command was stalling out. I tried several fixes and it took way too long.I did some searching on Kerberos and learned how the Kerberos utilities create the random data needed for encryption. The designers of Kerberos wanted a truly random data generator. They decided to base their random data generator on OS activities. There is a kernel parameter /proc/sys/kernel/random/entropy_avail. You can cat this value to see how much entropy your system has available. Since a VM is mostly idle you will get a small value.RedHat provides a package called rng-tools that you can i

Issue: Upgrade Ambari server from X.X.X version to Y.Y.Y version which was successful with errors as showing below -sudo ambari-server upgrade Using python /usr/bin/python Upgrading ambari-server Updating properties in ambari.properties ... WARNING: Can not find ambari-env.sh.rpmsave file from previous version, skipping restore of environment settings Fixing database objects owner Ambari Server configured for Embedded Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (y)? y Upgrading database schema Error output from schema upgrade command: com.google.inject.ProvisionException: Guice provision errors: 1) null returned by binding at org.apache.ambari.server.state.ServiceComponentHostFactory.createExisting() but parameter 2 of org.apache.ambari.server.state.svccomphost.ServiceComponentHostImpl.<init>() is not @Nullable while locating org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntity annotated with @com.google.inject.assistedinject.Assi

In HDP 2.5, the addition of RANGER-606 has introduced the ability to explicity deny access to a Hadoop resource via a Ranger Policy. RANGER-876 makes these types of policies optional by default for all except tag-based policies. To enable them, you must set enableDenyAndExceptionsInPolicies to true in the Service Definition for each of the Ranger Repository types as below, via the REST API:{ "name": "hdfs", "description": "HDFS Repository", "options": { "enableDenyAndExceptionsInPolicies": "true" }}How ToIf deny policies are not enabled, the Ranger “Create Policy” UI will look like this:Get the current service definition of the desired repository via a curl command and output to a file:curl -u admin:admin ranger-admin-host.hortonworks.com:6080/service/public/v2/api/servicedef/1 > hdfs.jsonIt should look something like this:{"id":1,"guid":"0d047247-bafe-4cf8-8e9b-d5d377284b2d","isEnabled":true,"createTime":1476173228000,"updateTime":1476173228000,"version":1,"name":"hdfs","imp

Please follow the below steps:1. SSH into Sandbox by typing:ssh root@127.0.0.1 -p22222. Download the RStudio rpm package and install itwget https://download2.rstudio.org/rstudio-server-rhel-0.99.893-x86_64.rpmsudo yum install --nogpgcheck rstudio-server-rhel-0.99.893-x86_64.rpm3. Install dpkg to divert the location of /sbin/initctl.yum install dpkgdpkg-divert --local --rename --add /sbin/initctlln -s /bin/true /sbin/initctl4. Install R and then verify the installation of RStudioyum install -y Ryum -y install libcurl-develrstudio-server verify-installation5. The default port of RStudio server is 8787 which is not opened in the Docker Sandbox. You can either add it by following this tutorial or just edit rserver.conf file to change the port to another which is opened. Let us edit the rserver.conf file, run the following sed command:sed -i "1 a www-port=8090" /etc/rstudio/rserver.conf 6. Restart the server by typingexec /usr/lib/rstudio-server/bin/rserver It will take you out from the do

You can setup encryption for the passwords stored in the Ambari properties file by running: On the Ambari server host, ambari-server setup-security This creates two files under /var/lib/ambari-server/keys, 1. credentials.jceks 2. master This article explains how to change the alias passwords stored in credentials.jceks without resetting encryption. Step 1: Take a backup of existing files cd /var/lib/ambari-server/keys cp master master.old cp credentials.jceks credentials.old.jceks If your cluster has JDK 1.8.xx installed, follow the below procedure: <JDK_HOME>/bin/keytool -list -v -keystore credentials.jceks -storetype JCEKS Enter keystore password: <JDK_HOME>/bin/keytool -delete -alias ambari.ldap.manager.password -keystore credentials.jceks -storetype JCEKS Enter keystore password: <JDK_HOME>/bin/keytool -importpass -alias ambari.ldap.manager.password -keystore credentials.jceks -storetype JCEKS Enter keystore password: Enter the password to be stored: Re-enter pa

While running Hcat query, it doesn't source hive-env.sh file but exports hcat-env.shAdd below parameter in hcat-env.sh from Ambari and restart hive services.HIVE_AUX_JARS_PATH=/tmp/testjar where /tmp/testjar is the directory for all custom jars.

Objective This tutorial demonstrates how to upgrade HDF from version 3.0.0 to 3.0.1.1 via the Ambari-managed Express Upgrade. Note: Additional documentation for HDF 3.0.1.1 upgrade can be found here.Environment This tutorial was tested using the following environment and components: Ambari 2.5.1 HDF 3.0.0Initial ConfigurationPrerequisites This tutorial assumes your HDF cluster is in a healthy state and that all preqrequisites for Ambari upgrade have been met.Upgrade StepsRegister Target Version 1. Select "Manage Ambari" from the Admin tab:2. Select "Versions" from the Clusters section: 3. Select the "Register Version" button: 4. Select "HDF-3.0" from the listed versions: 5. Enter the proper extension in the Name field, "HDF-3.0.1.1". Remove any OS's that are not relevant for your cluster. For your OS, add the proper HDF-3.0.1.1 base URL. For my CentOS7 enviro, it was: http://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.0.1.1/tars/hdf_ambari_mp/hdf-ambari-m

Spark Load testing framework built on a number of distributed technologies, including Gatling, Livy, Akka, and HDP. Using Akka Server powered by LIVY {Spark as a Service} provides the following benefits. REST friendly and Docker Friendly Low latency executionSharing cache across jobs Separation of concernMulti tenancyDirect Spark SQL executionConfiguration at one placeAuditing and LoggingComplete statement history and metricsLivy ServerLivy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN. Livy offers three modes to run Spark jobs: Using programmatic APIRunning interactive statements through REST APISubmitting batch applications with REST APILivy provides the following features: Interactive Scala, Python, and R shellsBatch submissions in Scala, Java, PythonMultiple users can share the same server (impersonation support)Can be used for submitt

Hadoop release itself wraps typical testing tools, and there are many other popular benchmarks for testing Apache Hadoop. When it comes to HDP, some of the testing tool needs to be configured before can be used, and more need to revise the coding for it can work on HDP, for example in order have more metrics supported or in case of kerberos environment. This article introduce configuration or modification for testing tools working in HDP.Let's start by going through typical service components form HDP, now I'm using HDP 2.6.1.0 with FreeIPA kerberized. HBase - YCSBwe begin with HBase because of more coding revise effort in bechmarking tools. Besides using HBase in-build evaluation tools org.apache.hadoop.hbase.PerformanceEvaluation to evaluation throughput and read/write performance, more popular tool is YCSB, which you download from git://github.com/brianfrankcooper/YCSB.gitgit clone git://github.com/brianfrankcooper/YCSB.gitbefore compiling we need to do following code modification b

Issue:Below is a sample test CREATE DATABASE IF NOT EXISTS tmp; DROP TABLE IF EXISTS tmp.orders; CREATE TABLE tmp.orders (username STRING, order_creation_date TIMESTAMP, amount DOUBLE); INSERT INTO TABLE tmp.orders VALUES ("jack", '2017-02-26 13:45:12', 88.2), ("jones", '2017-02-28 15:28:14', 92.4); HDP 2.4.2SELECT username FROM tmp.orders WHERE order_creation_date > '2017-02-27'; OK jonesselect * from tmp.orders; OK jack 2017-02-26 13:45:12 88.2 jones 2017-02-28 15:28:14 92.4 HDP 2.5.0 & 2.5.3SELECT username FROM tmp.orders WHERE order_creation_date > '2017-02-27'; OKSELECT username FROM tmp.orders WHERE order_creation_date > TO_DATE('2017-02-27'); OK You can notice the above 2 queries did not display any result Resolution:Workaround: Use cast function in hive or include hh:mm:ss as belowSELECT username FROM tmp.orders WHERE order_creation_date > '2017-02-27 00:00:00'; OK jonesThis is a known bug in hive Reference: Apache, Hortonworks

The HiveServer2 and HiveMetaStore can be configured for captured the GC logs based on Timestamp. This is useful in a production cluster, where having a timestamp on the log file add clarity and also avoids overwritting. Navigate as below in Ambari:Ambari UI > Hive > Configs > Advanced hive-env > hive-env templateAdd following :if [ "$SERVICE" = "metastore" ]; then export HADOOP_HEAPSIZE={{hive_metastore_heapsize}} # Setting for HiveMetastoreelse export HADOOP_HEAPSIZE={{hive_heapsize}} # Setting for HiveServer2 and Clientfiexport HADOOP_CLIENT_OPTS="-Xmx${HADOOP_HEAPSIZE}m -Xloggc:/var/log/hive/gc.log-$SERVICE-`date +'%Y%m%d%H%M'`-XX:ErrorFile=/var/log/hive/hive-metastore-error.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps $HADOOP_CLIENT_OPTS"if [ "$SERVICE" = "hiveserver2" ]; thenexport HADOOP_CLIENT_OPTS="-Xmx${HADOOP_HEAPSIZE}m -Xloggc:/var/log/hive/gc.log-$SERVICE-`date +'%Y%m%d%H%M'`-XX:ErrorFile=/var/log/hive/

This article focuses on setting up a custom ambari alert which monitors Yarn memory utilization by fetching from yarn JMX.Prerequisites - JSON file,access to node CLI, cluster credentials.Steps1: Create a alert.json file and save it on any location on node. There are two files attached (with comments and a template). Note: In the line ""Hadoop:service=ResourceManager,name=QueueMetrics,q0=root/AllocatedMB"" I am fetching the AllocatedMB from root queue. Thats visible in RM JMX.Step2: From the same node run this below command.It will create a new alert in ambari.curl -u user:password -i -H 'X-Requested-By:ambari' -X POST -d @alert.json

Good design goes mostly unnoticed. If a user interface works well, you simply complete a task and move on to the next. That time when something doesn't work as expected, or you think why do I have to do it this way, is when the design becomes much more noticeable. The following is a look back over recent months at some notable UI/UX improvements making their way to Apache NiFi. They vary a lot in size and complexity, but overall they contribute equally to make NiFi better by design.Provenance Storage Information NiFi conveniently provides storage information about flowfile and content repositories that it uses when processing. Now you can get those same storage metrics for provenance data within a cluster as well an instance's system diagnostics view. https://issues.apache.org/jira/browse/NIFI-4341Managing Variables The first step in configuring and managing reusable name/value pairs in NiFi was committed recently. As with access policies in NiFi, variables are created at a process

Enable SSL for Ambari Infra Solr: 1) Create directory on Infra Solr node: mkdir -p /etc/security/serverKeyscd /etc/security/serverKeys 2) Create Infra Solr keystore: keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass bigdata -storepass bigdata -validity 9999 -keystore infra.solr.keyStore.jks -ext SAN=DNS:{solr-hostname},IP:{solr-host-ip-address} -dname "CN={solr-hostname}, OU=Ambari, O=InfraSolr, L=Location, ST=State, C=Country" 3) Update the keystore file infra.solr.keyStore.jks ownership: chown infra-solr:hadoop infra.solr.keyStore.jks 4) Update below properties from Ambari under Ambari Infra Config Tab in Advanced infra-solr-env section: infra_solr_ssl_enabled (Enable SSL to Infra Solr) - trueinfra_solr_keystore_location (Infra Solr key store location) - /etc/security/serverKeys/infra.solr.keyStore.jksinfra_solr_keystore_type (Infra Solr key store type) - jksinfra_solr_keystore_password (Infra Solr key store password) - bigdata (Enter password given in -stor

Traditional data warehousing models espoused by Apache Hadoop and NoSQL stores entrench themselves on the belief that it is better to bring the processing to the data. Data flow, a hallmark of the HDF platform, is the mechanism by which we originally bring that data to processing platforms and/or data warehouses. Data can be processed, minimally, or fully, with products such as Apache NiFi for seamless ingestion into any number of data warehousing platforms. Apache MiNiFi is a natural extension of these paradigms where processing can be performed with the benefits that NiFi provides, within a much smaller memory footprint; however, this article will discuss using Apache MiNiFi as a data grooming and ingestion mechanism coupled with Apache NiFi to further leverage edge and IoT devices within an organization's infrastructure. How we view this: This article is intended to look at two problem statements. The first is using Apache MiNiFi in a slightly different way: as a mechanism to fron

SYNOPSIS Democratization of Big Data processing is here with Spark and Data Frames. Those days of writing complex parsers to parse and store weblogs data are over. You can easily use Spark's in-memory processing capabilities to quickly ingest and parse weblogs data. For this example, I downloaded some Omniture weblogs data from Adobe for a fictional company: The data consists of 421,266 records across 5 files which I put into HDFS Each record could contain up to 178 columns These are tab delimited text files Use Spark with Data Frames via PySpark to parse out the fields we need and output into new Parquet file Build an External Hive table over this Parquet file so analysts can easily query the data The code is at the end of this article. Before showing the code, I have some screenshots from Zeppelin that I took while doing this exercise to showcase Zeppelin's UI to quickly develop, analyze, and visualize your work. The code is a basic PySpark script to get you started with parsing te

The reason why Ambari is unable to start Namenode smoothly is bug and below is the workaround. Issue got fixed permanently in Ambari 2.5.x.Few lines of Error message from Ambari Ops logs:File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/decorator.py",line 55, in wrapper return function(*args,**kwargs) File"/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py",line 562, in is_this_namenode_active raise Fail(format("TheNameNode {namenode_id} is not listed as Active or Standby, waiting..."))resource_management.core.exceptions.Fail: TheNameNode nn2 is not listed as Active or Standby, waiting...ROOT CAUSE:https://issues.apache.org/jira/browse/AMBARI-18786 RESOLUTION:Increase the timeout in /var/lib/ambari-server/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.pyfrom this;@retry(times=5, sleep_time=5, backoff_factor=2,err_class=Fail)to this;@retry(times=25, sleep_time=25, backoff_factor=2,err_class=

Scenario: The spark log4j properties(Ambari > Spark > Configs) are not configured to log to a file. When running a job in yarn-client mode, the driver logs are spilled on the console. For long running jobs, it can be difficult to capture the driver logs due to various reasons like the user may lose connection with the terminal, or may have closed the terminal etc.The driver log is a useful artifact if we have to investigate a job failure.In such scenarios, it is better to have the spark driver log to a file instead of console.Here are the steps: Place a driver_log4j.properties file in a certain location (say /tmp) on the machine where you will be submitting the job in yarn-client modeContents of driver_log4j.properties#Set everything to be logged to the filelog4j.rootCategory=INFO,FILElog4j.appender.console=org.apache.log4j.ConsoleAppenderlog4j.appender.console.target=System.errlog4j.appender.console.layout=org.apache.log4j.PatternLayoutlog4j.appender.console.layout.ConversionPat

1.This option is tested in Ambari 2.2.2.0 and HDP 2.4.3.22.2.In the Ambari Background Operations dialog, stop all pending commands and jobs.3.Stop all services.4.Backup the Ambari database. (Default passwd for ambari : bigdata and mapred:mapred )[root@palm02 db_dumps]# pg_dump -U ambari ambari > ambari.sqlPassword: [root@palm02 db_dumps]# ls -lrttotal 10668-rw-r--r--. 1 root root 10920611 Oct 25 21:23 ambari.sql[root@palm02 db_dumps]# vim ambari.sql [root@palm02 db_dumps]# pg_dump -U mapred ambarirca > ambarirca.sqlPassword: [root@palm02 db_dumps]# ls -lrttotal 10680-rw-r--r--. 1 root root 10920611 Oct 25 21:23 ambari.sql-rw-r--r--. 1 root root 9189 Oct 25 21:24 ambarirca.sql5.Stop ambari-server and ambari-agents on all hosts. ambari-server stop ambari-agent stop6.Create *.json file with host names changes.[root@palm02 ~]# cat cluster_host.json { "palm" : { "palm02" : "palm02.hwx.com", "palm03" : "palm03.hwx.com", "palm20" : "palm20.hwx.com" }}[root@palm02 ~

SummaryBusiness users are continuously envisioning new and innovative ways to use data for operational reporting and advanced analytics. The Data Lake, a next-generation data storage and management solution, was developed to meet the ever-evolving needs of increasingly savvy users.This article explores existing challenges with the enterprise data warehouse and other existing data management and analytic solutions. It describes the necessary features of the Data Lake architecture and the capabilities required to leverage a Data and Analytics as a Service (DAaaS) model. It also covers the characteristics of a successful Data Lake implementation and critical considerations for designing a Data Lake.Current EDW ChallengesBusiness users are continuously envisioning new and innovative ways to use data for operational reporting and advanced analytics. With the evolution of users’ needs coupled with advances in data storage technologies, the inadequacies of current enterprise data warehousing

HIVE Beeline: ============Binary mode > !connect 'jdbc:hive2://prod07.app.hwx.com:10000/;transportMode=binary' http mode beeline -u 'jdbc:hive2://prod07.app.hwx.com:10001/;transportMode=http;httpPath=cliservice' In HS2 HA Environment with zookeeper out auto-discovery mode !connect jdbc:hive2://prod09.app.hwx.com:2181,prod10.app.hwx.com:2181,prod11.app.hwx.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 In Kerberos Environment - Hive Beeline command !connect 'jdbc:hive2://prod07.app.hwx.com:10001/default;principal=hive/prod07.app.hwx.com@EXAMPLE.COM;transportMode=http;httpPath=cliservice' KNOX with Beeline !connect jdbc:hive2://knox101.app.hwx.com:8443/default;transportMode=http;httpPath=gateway/default/hive;ssl=true knox with webhdfs curl -iku raj_ops -X GET https://knox101.app.hwx.com:8443/gateway/default/webhdfs/v1/tmp?op=LISTSTATUSif this article helps you.please up vote it.