Community Articles

IntroductionAmbari 2.2.2 provides a better understanding of cluster health and performance metrics through advanced visualizations and pre-built dashboards, isolating critical metrics for core cluster services such as Kafka reducing time to troubleshoot problems, and improving the level of service for cluster tenants.Ambari Metrics System (AMS) has also a new API to discover metrics. Only HDFS, YARN and HBASE default dashboards are included along with System metrics. Ambari 2.2.2 comes with built-in Grafana integration. Kafka default Grafana dashboard is targeted for Ambari 2.4. Until then, you can build a custom dashboard to see over 30 Kafka BrokerTopicMetrics using Grafana.Tutorial1. Install Ambari 2.2.2 or Upgrade to Ambari 2.2.2. On upgrade, to enable Grafana, follow instructions from Upgrade to Ambari 2.2.2.2. Automated Install of HDP 2.4 with Ambari 2.2.2, including Kafka 0.9.0.2.4. Grafana is added automatically on a new install.3. Explore Ambari Kafka Metrics that can be acces

Sometimes data in the system expires because it is no longer correct or the data was rented for a specific time period. One way to implement data expiration requirements is to delete the data after it is no longer valid. However you may also have another policy that requires retention of the data to track how decisions were made or for compliance with regulations. In addition deleting the data is more error prone to implement because an administrator must track a task in the future to delete the data after it expires. If the task is missed and the data is not deleted, expired data or illegal data could lead to incorrect decisions or lapses in compliance. This article shows an example of specifying the expiration date for a Hive table in Atlas and creating a tag based policy that prevents access of the table after the expiration date.Enabling Atlas in the Sandbox 1. Create a Hortonworks HDP 2.5 Sandbox You can use either a Virtual Machine or a host in the cloud. 2. In the brows

Use recommended mount options for all HDFS data disksThere are specific filesystem mount options that have proven to be more efficient for Hadoop clusters. Using these mount options will provide performance benefits. Since mount options are applied when mounting the filesystem, on system boot or remounting for example, changes to /etc/fstab alone are not enough for these settings to take effect. The recommended approach is to make the mount option changes and either manually remount the individual file systems or reboot the host for the settings to take effect.Use the following mount options for the respective file systems:ext4 —> "inode_readahead_blks=128","data=writeback","noatime","nodev"xfs —> "noatime"Configure HDFS block size for optimal performanceHaving optimal HDFS block size boosts NameNode performance as well as job execution performance.Make sure that the blocksize ('dfs.blocksize' in 'hdfs-site.xml') is within the recommended range of 134217728 to 1073741824 (excl

With HDCloud for AWS 1.16 and Cloudbreak 1.16.1, you can optionally use Hortonworks flex subscriptions to cover the controller and all clusters created. To do that you must: Obtain a flex subscription. For general information about the Hortonworks Flex Support Subscription, visit the Hortonworks Support page at https://hortonworks.com/services/support/enterprise/.Configure SmartSenseID and Telemetry when launching the cloud controller (HDCloud for AWS) or Cloudbreak.Register your Flex Subscription in the cloud controller or Cloudbreak web UI.When creating a cluster, select a flex subscription to use for the cluster.Configure and Manage Flex in HDCloud1. To configure flex in HDCloud, you must provide your SmartSense ID and Telemetry Opt In when launching the cloud controller. These options are available in the SmartSense Configuration section of the CloudFormation template:2. Once you log in to the cloud controller web UI, you can manage your flex subscriptions from the Settings > Ma

See Part 1: https://community.hortonworks.com/content/kbentry/107379/minifi-for-image-capture-and-ingestion-from-raspbe.html See Part 2: https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon-data-in.html We build a flow in Apache NiFi and then export the template. Using the MiniFi tool we convert this into a config.yaml file and send it to our device via scp. You can see this in Part 1. This simple flow calls a shell script that will run a Python script to get our sensor data. This flow will then send the data to our NiFi server via S2S over HTTP. What I have added in this part is use of the new Record and Schema paradigms and also the ability to SQL queries against incoming flow files using QueryRecord. This requires building an AVRO schema for our data, which is dead easy JSON definition. We set out port to connect the minifi agent to our server. Data quickly starts coming in. Receive the JSON Messages in NiFi via S2S

See Part 1: https://community.hortonworks.com/content/kbentry/107379/minifi-for-image-capture-and-ingestion-from-raspbe.htmlIn this article, we are ingesting beacon data using an ASUS Tinkerboard which is very similar to a Raspberry PI in form, form factor and features. It is newer and has less peripherals, but has 2 gig of RAM, ARM Mali GPU T760 MP4, Quad-core ARM Cortex-A17 1.8Ghz processor with a dual-channel DDR3 memory and BLE built-in (Bluetooth 4.0 with EDR).For beacons, I am using the basic Estimate three beacon collection. They have a really nice solid beacon in a silicon case, that looks nice and works well. They have a mobile app to manage, monitor and simulate as well as a cloud app.From our Apache NiFi server we have an Input Port that is receiving the messages from the Java MiniFi agent running on the ASUS Tinkerboard.As with most of the data, I want to ingest the data every minute.The ASUS Tinkerboard device has the read light lit up. The Estimate beacons are the

While traditional fraud detection systems have focused on looking for factors such as bad IP addresses or unusual login times based on business rules and events, the Connected Data Platform renovates such an approach by enabling machine learning capabilities at scale. The Credit Fraud Prevention Demo is an excellent example of a Modern Data Application running on the Hortonworks Connected Platform (HDF/HDP). Part I: Preparing the DemoFollow the instructions in the README to install the Credit Card Transaction Monitor Application on an HDP 2.4 Sandbox.Note: There is a known bug with the HAXM emulator used by the Android Studio for the Credit Card Transaction Monitor Mobile App that does not allow it to be launched while a Virtual Box virtual machine is running. To avoid this, I am using a HDP 2.4 Sandbox instance that I created in the Azure Cloud directly from the Microsoft Azure Marketplace. The install shell script handles the installation and configuration of all the application arti

DCloud for AWS 1.16 (released in June 2017) allows you to register a previously created RDS instance as a Hive or Druid metastore.In this tutorial, we will be: Launching an RDS instance and creating a database on it.Registering a database running on an RDS instance as a Hive metastore.Note that you can use these instructions for creating a Druid metastore.Let's get started!PrerequisitesAs a prerequisite to this tutorial, you need to have an HDCloud cloud controller running on AWS. If you need help with the steps required to meet this prerequisite, refer to the following tutorial: How to set up Hortonworks Data Cloud (HDCloud) for AWS. Launch an RDS Instance1. Navigate to the RDS Dashboard at https://console.aws.amazon.com/rds.2. In the top right corner, select the region in which you want to create your DB instance. Let’s create the RDS instance in the same region in which you've launched the cloud controller.3. In the RDS Dashboard navigation pane, click Instances, and then click Laun

Hortonworks Data Cloud for AWS (HDCloud for AWS) allows you to create on-demand ephemeral Hadoop clusters on AWS.In this tutorial, we will set up Hortonworks Data Cloud on AWS 1.16 (released in June 2017), including:Meeting the prerequisitesSubscribing to HDCloud services on AWS MarketplaceLaunching the cloud controller using the simple template Exploring AWS resources created Accessing the cloud controller UICreating a cluster Working with the cloud dashboardOpening additional portsCleaning up to avoid further chargesThis tutorial assumes no prior experience with AWS. Still, if you run into any issues, refer to the Troubleshooting documentation. Meet the Prerequisites1. Set up an AWS account: In order to launch HDCloud on AWS, you need to have an AWS account. You can set one up at https://aws.amazon.com/. Creating an AWS account is free, but you need to add a credit card that will be charged once you start running AWS services. Alternatively, you may want to contact your IT to find ou

The HDP 2.5 Sandbox VM for VirtualBox comes without the preconfigured port forwardings that allow you to map all the important Web UIs to ports on the host machine. Instead of going through the motions of entering all the port forwardings manually, I decided to copy them from the earlier Tech-Preview release using the VBoxManage CLI. The process has three steps: Obtain the new VM's SSID, which is a unique identifier that VirtualBox uses to address a VM.Read out the forwarding rules from the "old" (tech preview) VMFor each rule, configure a matching rule in the "new" VMHere is the code:VMID=$(VBoxManage list vms | perl -ne '/Docker.*{(.*)}/ && print $1')This bit gets the new VM's list entry (it is called "Hortonworks Docker Sandbox"), and parses the SSID, which is the bit that is enclosed by curly brackets. We need the SSID in the next step:VBoxManage showvminfo "Hortonworks Sandbox with HDP 2.5 Technical Preview" | perl -ne '/NIC.*?Rule.*name = (.*?),.*protocol = (.*?),.*host i

Installing SQuirrelDownload the SQuirrel jar from here, as per your operating system. Open the terminal and go to the directory where you have downloaded the file. Run the following command:java -jar squirrel-sql-3.7.1-MACOSX-install.jarOnce the installer dialog pops up, follow the instructions to install SQuirreL onto your system. You can choose to select optional installs if you like. I am choosing only the base and standard install. If you want other plugins, you can select them from the list given.Configuring SQuirrel and Phoenix Integration:1. Make sure your Sandbox and HBase is up and running.2. Open Additional Ports.You need all the HBase and Region Server ports forwarded, 16000, 16010, 16020, 16030. To do this, click on the Settings button for the VM instance. Then click the Network button in the pop up window. There is a port forwarding button at the bottom, click it. Click to add each of the following ports that you don't currently have.3. Copy the Phoenix client jar to SQuir

ISSUE:-/var/log/ambari-server/ambari-server.log18 May 2017 07:56:33,754 WARN [ambari-client-thread-26] PermissionHelper:78 - Error occurred when cluster or view is searched based on resource idjava.lang.NullPointerExceptionat org.apache.ambari.server.security.authorization.PermissionHelper.getPermissionLabels(PermissionHelper.java:74)18 May 2017 07:56:33,975 ERROR [ambari-client-thread-26] ContainerResponse:419 - The RuntimeException could not be mapped to a response, re-throwing to the HTTP containercom.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key 58.at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348)at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318)at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)18 May 2017 07:56:33,977 WARN [ambari-client-thread-26] ServletHandler:561 - Error Processing URI: /api/v1/users/admin - (com.google.common.cache.Cach

Symptoms:After recently upgrading SmartSense, you successfully log into Activity-Explorer and attempt to execute any pre-installed notebooks only to encounter one of the following errors:RpcRetryingCaller {globalStartTime=1473694857754, pause=100, retries=35}, org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to node1.abcxyz.com/192.168.1.1:61300 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to node1.abcxyz.com/192.168.1.1:61300 is closing. Call id=4, waitTime=2or192.168.1.1:61300 is closing. Call id=32, waitTime=4 RpcRetryingCaller{globalStartTime=1473456721387, pause=100, retries=35}, org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.io.IOException: Broken pipe RpcRetryingCaller{globalStartTime=1473456721387, pause=100, retries=35}, org.apache.hadoop.hbase.MasterNotRunningException

There are certain circumstances where the Ambari CA needs to be recreated. Maybe it was corrupted, maybe Ambari's CA certificate expired, etc.... This how-to, assumes that the certificates signed by the Ambari CA are replaceable - which is generally the case for certificates used by Ambari agents for 2-way SSL connections. In the end, the Ambari server and all the agents will be restarted, causing a new CA certificate to be created along with new SSL certificates for each of the Ambari agents. On the Ambari server: Stop the Ambari serverBackup /var/lib/ambari-server/keys and it child directoriesDelete the following files from /var/lib/ambari-server/keysca.keyca.csrca.crtpass.txtkeystore.p12*.csr*.crtDelete the following files from /var/lib/ambari-server/keys/dbindex.txt.oldindex.txt.attr.oldserial.oldTruncate the following files from /var/lib/ambari-server/keys/dbindex.txtindex.txt.attrEdit the following files from /var/lib/ambari-server/keys/dbserialset the contents to be exactly00De

Problem:Phoenix query using PERCENTILE_CONT fails with NullPointerExceptionFor example:CREATE TABLE IF NOT EXISTS P_C ( COL1 INTEGER NOT NULL PRIMARY KEY, COL2 INTEGER ); SELECT PERCENTILE_CONT (0.99) WITHIN GROUP (ORDER BY COL2 ASC) FROM P_C; java.lang.NullPointerException at org.apache.phoenix.expression.aggregator.PercentileClientAggregator.evaluate(PercentileClientAggregator.java:82) at org.apache.phoenix.schema.KeyValueSchema.toBytes(KeyValueSchema.java:112) at org.apache.phoenix.schema.KeyValueSchema.toBytes(KeyValueSchema.java:93) at org.apache.phoenix.expression.aggregator.Aggregators.toBytes(Aggregators.java:109) at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:44) at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:778) at sqlline.BufferedRows.<init>(BufferedRows.java:37) at sqlline.SqlLine.print(SqlLine.java:1650) at sqlline.Commands.execute(Commands.java:833) at sqlline.Com

Problem:hbase hbck fails with the following messageszookeeper.ClientCnxn: SASL configuration failed: javax.security.auth.login.LoginException: Zookeeper client cannot authenticate using the 'Client' section of the supplied JAAS configuration: '/usr/hdp/current/hbase-client/conf/hbase_regionserver_jaas.conf' because of a RuntimeException: java.lang.SecurityException: java.io.IOException: /usr/hdp/current/hbase-client/conf/hbase_regionserver_jaas.conf (No such file or directory) 017-06-11 10:11:36,293 ERROR [main] master.TableLockManager: Unexpected ZooKeeper error when listing childrenorg.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /hbase-secure/table-lock at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)Solution:Identify the location of the jaas file and set it in the HBASE_SERVER_JAAS_OPTS parameter, make sure it exists, then set the HBASE_SERVER_JAAS_OPTS

Problem: Running a query in Hive View in Ambari, rows 1 to 100 are shown, then the next page starts at 102.For example, first screen:Second screen:Resolution:This is bug AMBARI-19666 which is fixed in Ambari 2.4.3 and later.

Problem: Certain arithmetic operations return the wrong result in Phoenix. For example:0: jdbc:phoenix:> select 2/4 ;+----+| 0 |+----+| 0 |+----+1 row selected (0.014 seconds) Cause: This is because the result is returned as an int rather than a decimalSee PHOENIX-3312 Workaround: Provide decimal values within the SQLe.g.0: jdbc:phoenix:> select 2.0/4.0 ;+------+| 0.5 |+------+| 0.5 |+------+1 row selected (0.011 seconds)

Question: Are there any functions which allow date arithmetic with phoenix? Answer: this is done as a fraction of a day. e.g. to add 12 hours to a day: > select now(), now() + (0.5) ; +---------------------------------+---------------------------------+| DATE '2017-06-04 16:55:07.989' | DATE '2017-06-05 04:55:07.989' |+---------------------------------+---------------------------------+| 2017-06-04 16:55:07.989 | 2017-06-05 04:55:07.989 |+---------------------------------+---------------------------------+ To add 15 minutes you would use (15/1440). ie. 0: jdbc:phoenix:> select now(), now() + (0.010416666666667) ;+---------------------------------+---------------------------------+| DATE '2017-06-04 16:56:31.492' | DATE '2017-06-04 17:11:31.492' |+---------------------------------+---------------------------------+| 2017-06-04 16:56:31.492 | 2017-06-04 17:11:31.492 |+---------------------------------+---------------------------------+1 row

STEP 1Configure cluster for Tez ViewEnable ATS by : yarn.timeline-service.enabled=true Enable following yarn setting: yarn.resourcemanager.system-metrics-publisher.enabled=true yarn.timeline-service.webapp.address=IP:PORT of ATS Enable following core-site setting hadoop.proxyuser.ambari-server.groups=* hadoop.proxyuser.ambari-server.hosts=* Kerberize Ambari-server Create a principal in your KDC for the Ambari Server. For example, using kadmin:addprinc -randkey ambari-server@HADOOP.GCSKDC.CORP.APPLE.COM Generate a keytab for that principal.xst -k ambari.server.keytab ambari-server@HADOOP.GCSKDC.CORP.APPLE.COM Place that keytab on the Ambari Server host. Be sure to set the file permissions so the user running the Ambari Server daemon can access the keytab file./etc/security/keytabs/ambari.server.keytab Stop the ambari server.ambari-server stop Run the setup-security command.amb

Configure SSL for Ambari Server# On ambari-server host, take a back up of existing filesmkdir /root/ambari-ssl-backup; cp -r /var/lib/ambari-server/keys/* /root/ambari-ssl-backup/# Navigate to the folder where all certificates are stored.cd <dir># Extract certificates from p7b fileopenssl pkcs7 –in ambari-server-host-cert.p7b –print_certs > ambari-server-host-name.crt# Above .crt file contains full certificate chain, so separate server certificate from root chain. Copy the intermediate/root chain to ca.crt and delete intermediate/root chain from ambari-server-host-name.crt# Copy ca.crt, ambari-server-host-name.crt, ambari-server-host-privateKey.key files to /var/lib/ambari-server/keys/cp ca.crt /var/lib/ambari-server/keys/cp ambari-server-host-privateKey.key /var/lib/ambari-server/keys/cp ambari-server-host-name.crt /var/lib/ambari-server/keys/# Update password for the keystore in a file (Assuming it as ‘changeit’)echo "changeit" > pass.txt# Generate the keystore for ambar

This article describes how to enable Knox proxying for Zeppelin Notebook for Wire encrypted environment (i.e SSL is enabled for Knox and Zeppelin) Non Wire encrypted environments Configuring Knox proxying for Zeppelin Notebook in Wire encrypted Environments If you have already configured SSL for zeppelin, please proceed to section 2. If not, please read through section 1. Section 1 : Configuring SSL for Zeppelin Note : The steps mentioned in section 1 are just for example purpose, and the production setup may be different. Also the steps assume no client-side authentication. For client-side authentication, please follow the Zeppelin Component Guide in HDP release documents (HDP 2.6.1 Zeppelin Component Guide : https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_zeppelin-component-guide/content/config-ssl-zepp.html ) Create keystore file, truststore file and certificates for each host on the cluster by following these steps Navigate to the directory where you wa

This is a beginner's guide for debugging a running NiFi instance. It covers two important aspects of debugging: adjusting the log levels of what is written to the NiFi application logs and attaching a graphical debugger to a running NiFi instance.RequirementsApache NiFi source code (this tutorial uses NiFi 1.3.0) If you have never build and run NiFi from source, follow the instructions in the NiFi Development Quick Start Guide: https://nifi.apache.org/quickstart.htmlIn this tutorial, {NIFI_ROOT} refers to the location of your NiFi source directory, i.e., your git clone or uncompressed source directory.IntelliJ IDEA (this tutorial was made using IntelliJ IDEA Community 2107.01)Available from: https://www.jetbrains.com/idea/downloadIf you are using other version of Apache NiFi or IntelliJ IDEA, the steps should be very similar though some of the details and example screen shots might vary slightly.Enabling Debug LoggingBy default, the log level for NiFi is set to INFO, meaning log lines

Component AlignmentIf you have ever wanted to line up the components on your canvas, you now have an easy way to do it. Just select the desired components, right-click to see the context menu and select “Align vertically” or “Align horizontally” depending on your desired result."Align vertically" example:“Align horizontally” example: Component LinkingHave you ever wanted to go directly to a process group instead of having to find your way through other process groups to get to it? NiFi 1.2 supports the use of a hyperlink to navigate directly to a component (process group, processor, etc.). For example, the flow shown in the following screenshot is in process group “PG 4” which is embedded within three other process groups: Note that the NiFi URL in the browser now contains component information, specifically the Process Group ID for "PG 4”. The URL can now be shared with other users, and when pasted in their browsers, will take them directly to that process group on their canvas.

NiFi 1.2.0 has a nice, new little feature that will surely please those who may spend a bit of time – for some, perhaps A LOT of time – getting their flow to line up perfectly. The background grid lines can help with this, but the process can still be quite tedious with many components. Now there is a quick, easy way.Select the components on the canvas you want to align. You can do this by either Shift+clicking each component until they are all selected, or while holding Shift, click and (fully) drag around all the components you want to align.Next, simply right-click on any selected component and choose the desired "Align..." option.The next image shows the result after choosing the Align vertically option.If components are different sizes, they will align according to their center points.And that's all there is to it!