Community Articles

Scenarios: There are few scenarios where you may need to migrate ranger components from one host to another host in Ambari. Server that is hosting Ranger master components is faulty & down or need to be replaced for newer hardware. Reduce the administrative implementation of enabling Ranger HA with load balancer and still have some recovery mechanism to get ranger back online as quickly as possible should you encounter above sitaution. Design of Ranger has HA builtin where the policies are cached in every plugin such as namenode will have a copy of policies cached on disk in the event ranger admin is not available. If ranger admin/usersync is down administrators will only lose access to add/modify/delete policies and users. In many of the hadoop clusters, users may not need to have high availability access to Ranger UI. It may be ok to survive for few hours without ranger ui and usersync components as existing policies and users should continue to work. Method: Stop

This video covers the Default Zeppelin UI option and basic navigation demo. References: Cloudera Product document page: https://docs.cloudera.com/runtime/7.1...  Cloudera tutorials: https://www.cloudera.com/tutorials/ge...  Visit Apache Zeppelin Website: http://zeppelin.apache.org  Cloudera Community: https://community.cloudera.com/

This video covers zeppelin’s backend operations and overview on impersonation concepts in Zeppelin. References: Cloudera Product document page: https://docs.cloudera.com/runtime/7.1...  Cloudera tutorials: https://www.cloudera.com/tutorials/ge...  Visit Apache Zeppelin Website: http://zeppelin.apache.org  Cloudera Community: https://community.cloudera.com/

This video covers High level overview of Zeppelin’s architecture and Operational Flow. References: Cloudera Product document page: https://docs.cloudera.com/runtime/7.1... Cloudera tutorials: https://www.cloudera.com/tutorials/ge... Visit Apache Zeppelin Website: http://zeppelin.apache.org Cloudera Community: https://community.cloudera.com/

The Cloudera Data Platform (CDP) comes with many places to store your data, and it can be challenging to know which one to use. Though there is no formal decision tree, I hereby share the key considerations from my personal perspective. They can be visualized like this:  Explanation of each patha. Have large bulky files, that do not need to be queried > File and object storageThe exact kind of storage to be used will mostly be defined by your environment, in a classical cluster HDFS is available. In the public cloud, each provider object store will be leveraged, and on-premises Ozone will serve as the object-store.b. Have a table, either from large bulky files, or a set of messages > Hive for scale or Kudu for interactionIf you want to work with a table, and need to store it as such, it is clear you want to store your data as a table. Even if this may force you to think about how to implement the ingest in a sensible way. Kudu is great for fast insights, where hive tabl

Disclaimer: This article is based on my personal experience and knowledge. Don't take it as a standard guidelines, understand the concept and modify it for your environmental best practices and use case. Ambari unofficially supports a curl command which will help deleting the old stacks and packages from each of the host . the script is described in https://issues.apache.org/jira/browse/AMBARI-18435 and also in the article : https://community.hortonworks.com/articles/202904/how-to-remove-all-previous-version-hdp-directories.html But this above script doesn't work in ambari-2.7.3 version. Root cause : the script will fail with below exception : Traceback (most recent call last): File "/var/lib/ambari-agent/cache/custom_actions/scripts/remove_previous_stacks.py", line 119, in <module> RemovePreviousStacks().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/var/lib/ambari-agent/cache/custom

In this tutorial we will walkthrough the steps to enable YARN Node labels so that we can run LLAP on a specific set of nodes in the cluster. (Stack Version HDP 2.6.3)In the first step we will check if we have any NodeLabels created on the cluster and the command is as follows[$] yarn cluster --list-node-labels 18/02/05 10:32:29 INFO client.RMProxy: Connecting to ResourceManager at stocks.hdp/192.168.1.200:8050 18/02/05 10:32:31 INFO client.AHSProxy: Connecting to Application History server at stocks.hdp/192.168.1.200:10200 Node Labels: This shows that there are no node labels configured on the cluster. We now try to add a new node label, we will use this node label to assign to LLAP queue, so that LLAP runs on that specific node(s). For now we will keep the node label exclusive, so that only users in LLAP queue can access these nodes [$] yarn rmadmin -addToClusterNodeLabels "interactive(exclusive=true)” 18/02/05 10:33:25 INFO client.RMProxy: Connecting to ResourceManager at stocks.hdp/

I am capturing the steps to install supplementary spark version on your HDP version. Installing the version not shipped by ambari is unsupported and not recommended, however they are some times customer need it for testing purposes.Please find the steps belowHere are the steps :-1. Create the spark user on all nodes. Add it to the hdfs group.    useradd -G hdfs spark 2. Create conf and log directorymkdir -p /etc/spark2.4/confmkdir -p /var/log/spark2.4chmod 755 /var/log/spark2.4/chown spark:hadoop /var/log/spark2.4mkdir -p /var/run/spark2.4chown spark:hadoop /var/run/spark2.4chmod 775 /var/run/spark2.4mkdir /var/lib/spark2/chown spark:spark /var/lib/spark2/3. Create a directory in /usr/hdp and cd to the dirmkdir -p /usr/hdp/<your hdp version>/spark2.4 (root user)cd /usr/hdp/<your hdp version>/spark2.44. Download the tar file from location http://apache.claz.org/spark/spark-2.4.0/ wget http://apache.claz.org/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.7.tgz5. Extract

There will be multiple form factors available in the future, for now, I will assume you have an environment that contains 1 datahub with NiFi, and 1 Data Hub with both Impala and Kudu. (The answer still works if all are on the same datahub).   Prerequisites Data Hubs with NiFi and Impala+Kudu Permission to access these (e.g. add a processor, create table)  Know your Workload User Name (CDP Management Console > Your name (bottom left) > Profile) You should have set your Workload Password in the same location Steps to write data from NiFi to Kudu in CDP Public Cloud Unless mentioned otherwise, I have kept everything to its default settings. In Kudu Data Hub Cluster: Gather the FQDN links of the brokers and the used ports. Go to the Data Hub > Click Kudu Master > Click Masters. Combine the RPC addresses together in the following format: host1:port,host2:port,host3:port Example:```master1fqdn.abc:7051,master2fqdn.abc:7051,master3fqdn.abc:7051```​ In HUE

The Cloudera Data Platform (CDP) comes with a wide variety of tools that move data, these are the same in any cloud as well as on-premises.Though there is no formal decision tree, I will summarize the key considerations from my personal perspective. In short, it can be visualized like this:   Steps for finding the right tool to move your dataStaying within Hive and SQL queries suffice? > Hive otherwiseNo complex operations (e.g. joins) > Nifi otherwiseBatch > Spark otherwiseAlready have Kafka Streams/Spark Streaming in use? > Kakfa Streams/Spark Streaming otherwiseFlinkSome notes:If you can use Nifi or a more complex solution, use NifiUse Flink as your streaming engine, unless there is a good reason not to. It is the latest generation of streaming engines.Currently, I do not recommend using Flink for batch processing yet, but that will likely soon changeI did not include tools like Impala, Hbase/Phoenix, Druid as their main purpose is accessing dataThis is a

When running the YARN service check in a Resource Manager HA environment you see that the service check fails - all other functionality is working correctly (restart of services, running jobs etc)When you run the service check you see:stderr: /var/lib/ambari-agent/data/errors-392.txtTraceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py", line 159, in <module> ServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py", line 130, in service_check info_app_url = params.scheme + "://" + rm_webapp_address + "/ws/v1/cluster/apps/" + application_name File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__ raise Fail("Configuration parameter '"

Co-authored by Balazs Gaspar and @steffen.maerkl - more coming soon   The purpose of this article is to describe how to set up Keycloak as a SAML 2.0 Identity Provider for CDP Public Cloud, enabling any user in your organisation to use single sign-on to access CDP and its services, such as Cloudera Machine Learning or Cloudera Data Warehouse.   Keycloak is a popular open source Identity and Access Management (IAM) solution but of course there are many more such solutions, so see this as one example. In an upcoming Cloudera Community post we plan to provide a similar guide for configuring CDP Public Cloud with other popular SSO solutions.   In the examples below we will create users and groups directly in Keycloak. In a real world deployment it is more likely that you would use Keycloak only for SSO and rely on a separate provider (such as Active Directory) to store users and groups via LDAP federation. If you have not been exposed to SAML 2.0 previously, Okta

This article contains Questions & Answers on Cloudera Operational Database (COD). What is the relationship between Data Hub & Data Lake?Data Lake houses SDX (governance & authorization). Data Hub is the actual service that hosts the workload, in this case, the Operational DB. When should I use Cloudera OpDB vs a template in Data Hub?For new apps use Cloudera OpDB (COD) that is self-tuning and auto-improves performance over time. And for replicating on-prem environments to the cloud via lift and shift or for disaster recovery use Data Hub templates. How does Apache Phoenix relate to Apache HBase?Phoenix is an OLTP SQL engine for OpDB. It adds relational capabilities on top of HBase. Phoenix provides a much more familiar programming paradigm and allows our customers to reach production faster. Think of Phoenix as a SQL persona and HBase as a NoSQL persona. Should I use HBase/Phoenix or Apache Kudu for an operational data store (ODS) / Operational database?

This article contains Questions & Answers on Cloudera Data Warehouse (CDW) demos. Which data warehousing engines are available in CDW?Hive for EDW and complex report building and dashboarding, Impala for interactive SQL and ad hoc exploration, Kudu for time-series and Druid for log analytics. How do you create data warehouses?Step 1, create your CDP environment. Step 2, activate the CDW service. Step 3, create your virtual warehouse. Step 4, define tables, load data, run queries, integrate your BI tool, etc. What’s the relationship between a Database Catalog and Virtual Warehouse?For each database catalog there can be one or more virtual warehouses. But each virtual warehouse is isolated from other warehouses while they share the same data and metadata. What’s the CDW query performance in the cloud when using remote storage?CDW uses different levels of caching to offset object storage access latency. This includes a data cache on each query execution node, a que

This article contains Questions & Answers on Cloudera AI. Is it possible to run CML on premises?Yes, Cloudera Machine Learning is available on both CDP Public Cloud as well as CDP Private Cloud. How is the deployment of models managed by CML Private Cloud vs CML Public Cloud?Model deployment functions similarly across both form factors of CML; the models are built into containerized images, and then deployed on top of Kubernetes pods for production-level serving. Is there a level of programming required for a data scientist to use this platform? What languages can developers use?CML enables data scientists to write code in Python, R, or Scala in their editor of choice. Therefore, beginner data scientists can easily run sample code in the workbench, and more experienced data scientists can leverage open source libraries for more complex workloads. Can you run SQL-like queries? E.g. with Spark SQL?Yes, Spark SQL can be run from CML Do pre-built models come ou

This article contains Questions & Answers on Cloudera DataFlow (CDF). What is the difference between Cloudera’s NiFi and Apache NiFi?Cloudera Flow Management (CFM) is based on Apache NiFi but comes with all the additional platform integration that you’ve just seen in the demo. We make sure it works with CDP’s identity management, integrates with Apache Ranger and Apache Atlas. The original creators of Apache NiFi work for Cloudera. Does NiFi come with CDP public cloud or is it an add on?Any CDP Public Cloud customer can start using NiFi by creating Flow Management clusters in CDP Data Hub. You don’t need additional licenses and you will be charged based on how many instance hours your clusters consume What types of read/write data does NiFi support?NiFi supports 400+ processors with many sources/destinations. Do you support cloud-native data sources and sinks?Yes, we support a lot of cloud-native sources and sinks with dedicated processors for AWS, Azure, and GC

This article contains Questions & Answers on CDP Security and Governance (SDX). Does SDX provide governance for all the data in the cluster that’s in the cloud?Yes. Any cluster that has been built with CDP will have governance applied to it, regardless of whether it’s deployed to a public or private cloud. How do we enable end to end governance on CDP?  Is it different for public cloud and private cloud?End-to-end governance is enabled by default in CDP public cloud and private cloud.  Under the hood, Apache Ranger and Apache Atlas are out-of-the-box wired into all the processing engines in CDP (e.g. Apache Hive, Impala, NiFi, Spark et al.). How do you set up SDX?SDX installation is completed when you provision an environment with wire and at-rest encryption preconfigured. Technical Metadata management functionalities are also set up automatically. Business Metadata and data policies must be implemented according to the customer’s context and requirements.&

This article contains Questions & Answers on Cloudera Data Platform (CDP) - private or public cloud. CLOUDERA DATA PLATFORM Which clouds is CDP public supported on?AWS, Azure and, soon, GCP. What’s the difference between Cloudera’s product and cloud providers?Cloudera Data Platform (CDP) is a Platform-as-a-Service (PaaS) that is cloud infrastructure agnostic and easily portable between multiple cloud providers including private solutions such as OpenShift. CDP is both hybrid and multi-cloud from the ground up which means one platform can serve all data lifecycle use cases, independent of location or cloud, with a unified security and governance model.  How do CDP experiences compare to solutions from other cloud service providers?CDP has an SDX layer that stores all policies and metadata for security and governance. This preservation of state is the big differentiating factor, especially when running transient workloads and a variety of experiences. The SDX

To put Kerberos DEBUG logging for HDF managed Schema Registry ( SR ) clusters, do the following depending on your environment: Navigate to Ambari schema registry > configs > advanced > registry-env template Add the below line:export REGISTRY_OPTS="$REGISTRY_OPTS -Dsun.security.krb5.debug=true"Example: Restart SR. You will then be able to view Kerberos DEBUG messages under /var/log/registry (default).     

In this article, we cover the following:  Demonstrate how hive sessions work Clarify some misunderstanding about hive sessions behavior (most of the people, including myself until recently, believe hive sessions and queue allocation work different than the way they actually work) Propose a solution for small-medium environments (up to 200 nodes) that allows concurrent users + multi-tenancy + soft allocation + preemption   Assumptions: Tested with HDP 2.4.0 I'm using configuration suggested in best-practices and docs related to security and hive performance guide, which include following configurations: hive engine (hive.execution.engine) = tez hive do-as (hive.server2.enable.doAs) = false hive default queues (hive.server2.tez.default.queues) = (queue-name1,queue-name2,etc) hive number of sessions (hive.server2.tez.sessions.per.default.queue) = 1 (or up to 4) hive start sessions (hive.server2.tez.initialize.default.sessions) = true hive pre-warm containers (hive.prewarm.en

When NiFi is secured for TLS server authentication, at UI login time first it tries to use TLS certificates if loaded on the browser and then, it tries to use SPNEGO authentication, and finally, it falls back to your configured login provider. If you KERBERISE the cluster via AMBARI and want to use login-providers like LDAP or KERBEROS, it automatically sets the following properties which enable SPNEGO authentication. nifi.kerberos.spnego.keytab.location nifi.kerberos.spnego.principal    Furthermore, SPNEGO properties through AMBARI are greyed out for: COMMAND: From your Amabari manager host, change the setting for NiFi, where the text in red is tailored to your unique environment: nifi.kerberos.spnego.keytab.location to be blank: ./configs.py -a set -s http -l c2288-node1.squadron.support.hortonworks.com -t 8080 -n c2288 -u admin -p AdminPassword -c nifi-properties -k 'nifi.kerberos.spnego.keytab.location' -v '' nifi.kerberos.spnego.principal to be bl

See the webinar for this Prototype here: https://www.cloudera.com/about/events/webinars/building-diverse-anomaly-detection-capabilities-for-business-with-cdp.html  We now have our second Applied Machine Learning Prototypes available (Go here for the previous Prototype - Customer Churn). These are prototypes (or templates as they were called) that will help you build a fully working machine learning example in CML. The Templates will include source data, and walk through various steps:Ingest data into a useful place in CDP (e.g. a Hive Table)Explore the data setCreate a plan to build a modelTrain the modelDeploy the modelBuild and deploy an applicationOnce you have deployed the template and all the CML artifacts that go with it, you can unpick and work it backward to map the process to your own data in your own environment. Our latest Prototype - Fraud Detection - is now available. To get up and running with it, do the following:Log in to your CML workspace and creat

Issues with Global IndexesIn Part 1 and Part 2 of this article series we discussed Index internals and some frequently faced issues. In this article we will cover few more index issues in the form of scenarios.Scenario 4 : Index writes are failing, client retries exhaused, handler pool saturated while Index table regions are in transition.Here client is trying to write to data table on server1 which is triggering index update for server2 (via server-to-server RPC) . If Index region is stuck in transition, the index update RPC would also hang and eventually timeout and because of this the RPCs between client and server1 also gets stuck and timed out. Since client would make several retries to write this mutation, it would again lead to handler saturation on region server one causing another “deadlock” like situation.This should be fixed in two steps: Fix all Index RITs first as without this no client index maintenance or server side index rebuild would succeed.As a holistic tuning, keep

If we are working on some cloud environment like "OpenStack" then we might encounter an issue in which after installing the ambari agents we will see that some of the Quick Links in the ambari console is not having the expected hostname. For example if we navigate to Ambari --> HDFS --> "Resource Links" then we may see the incorrect hostnames. Actually those are detected as "public_host_name" by the ambari "/usr/lib/python2.6/site-packages/ambari_agent/hostname.py" script during every restart.So in case of OpenStack it is possible that the "public_host_name" might be different from the "host_name" (`hostname -f`)Here if we see that the Resource Link will show the links with the hostname "ambari1.example.com" (which is `hostname -f` but it is showing "ambari1.novalocal" hence the links are not working.In such case we might find that the ambari "hosts" table in the ambari database has different values for the "public_host_name" and the "host_name"select public_host_name, host_name

This is a quick how-to on integrating spark on zeppelin with kerberised kafka. To do this first we will enable impersonation support in zeppelin for spark. By Impersonation, i mean jobs will be executed / submitted to spark by zeppelin as the user logged in to zeppelin and not the default zeppelin user. In my case i will be using my user "Karthik"Now to enable impersonation while submitting to spark, i would recommend using the livy server. The livy server and the interpreter natively support impersonation. Spark interpreter impersonation is possible, but you may have to configure SSh keys or sudo capability for zeppelin which may not be recommended. You can livy to hdp by going to a host and installing "Spark client" , followed by the livy server.Restart Zeppelin , to enable the livy interpreter. Ensure that livy impersonation and livy proxy permissions are setup correctly in ambari. This is done when you install livy server by default.Log in to zeppelin using your credentials.click t