Q&A

When I enable the Ranger Service from HIVE in Cloudera Manager, it's getting an error like:"Hive Impersonation is enabled for Hive Server2 role 'HiveServer2 (node1)'. Hive Impersonation should be disabled to enable Hive authorization using Ranger"I want to access control from Ranger, But I cant Anybody, please help me.

It says a problem with the current version of jquery. URL :  "http://<myip>:8042/static/jquery/jquery-3.3.1.min.js" Installed version : 3.3.1 Fixed version : 3.5.0Vulnerability link:" https://www.tenable.com/plugins/nessus/136929"Finding description: JQuery 1.2 > 3.50 XSS Is there like a workaround to overcome this vulnerability ?  or is necessary to upgrade to a newer version of HDP?Would be possible to upgrade jquery version only ? Thanks in advance..! 

HI, I am trying to install a custom service on Cloudera. This service requires some libs for it to run.I have created the CSD sdl and control.sh file and created a jar out of it. The issue is that I am not getting where to put service libs in Cloudera. Should these be bundled in CSD jar itself? or should these be installed as parcel. If as parcel, how to install it. I do not see much doc on installing/deploying local parcel on Cloudera manager. Please help. 

I have a NIFI image running in openshift and a postgres in the cloud "owned" by another department.   They sent to us a certificate(".crt") and a key(".key"), since the log-in is made trough client certificate, instead of username and password. I have succeeded to log-in in pgAdmin 4, but not to connect the NIFI to the Postgres with certificate and key.   I have uploaded the certificate and the key to the image (using a secret and mounting it) so if I go to the pod terminal I can access it.   But, when I pass to a DBCPConnectionPoll service the connection string bellow and activates an ExecuteSQL processor, I receive an exception that the certificate is not valid, as follows:   ERRORExecuteSQL[id=...] Unable to execute SQL query <...>;due to java.sql.SQLException: Cannot create a PoolableConnectionFactory (FATAL: connection requires a valid client certificate).No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLEx

Hi All,I am new on Ambari. I use Rhel7 and i installed Ambari server and have been configured it succefully on my server.So, i want to install Airflow on this existing ambari server. I've tried to install it with the command "ambari-server install-mpack --mpack=/opt/airflow-service-mpack.tar.gz"and i got this python message : Using python /usr/bin/python Usage: {start|stop|restart|setup|setup-jce|upgrade|status|upgradestack|setup-ldap|sync-ldap|set-current|setup-security|setup-sso|refresh-stack-hash|backup|restore|update-host-names|check-database|db-cleanup|enable-stack} [options] Use --help to get details on options available. Or, simply invoke ambari-server.py --help to print the options. Need help Please. Thanks, Math

Hi,I need to setup a 5 node cluster with Hadoop 3.1.0 and Spark 2.4.5 . Someone recommended to use Ambari to do so. I checked Ambari but it seems Ambari can be used only to install HDP and latest HDP do not support Spark 2.4.5 version.  Please suggest in this aspect, what will be the best way to setup the required big data cluster.

Hi, do you know how can I have a replication between two clusters? for example, I want my cdh cluster 1 to be replicated with cdh cluster 2 in real time, so once this cluster is down, we still have the other cluster to work on with

Hi, I have been trying to setup nifi site to site via our web proxy, but am having issues.  I have reviewed the documentation, but can't get past the files we want to transfer sitting in the queue.  My client side setup is relatively simple:-A GETFILE processor that monitors a local directory on the server-A Nifi Flow via a Remote Process Group connection to the remote nifi. The Remote processor group is configured to hit our proxy using http on port 8080.  When configured, it picks up the Input of the remote connection in the Remote Group at  http://remoteserver.org:9090/nifi So the connectivity seems to be there, but the files simply sit in the queued state and never try to use the remote connection.   Am I missing a step on the client side (maybe an output processor)?  Is there something I can have the remote side admin check to enable data flow? Sorry for the beginner question, but I just can't seem to find the answer afte

Hello,I have a question for CDH Upgrade.I have a problem with CPU scheduling in RHEL7.4, so I need to upgrade to RHEL7.7 * Information<AS-IS>- CDH: 5.14.4- OS: RHEL 7.4(3.10.0-693-e17.x86_64) <TO-BE>- CDH: 5.16.2- OS: RHEL 7.7(3.10.0-1062-e17.x86_64) ISSUE: No Internet Environment (Separate Network Env) Q1) Is there any problem with system library reference?Q2) Is there a library linkage problem between CDH 5.14.4 and CDH5.16.2?Q3) If the 3 servers are RHEL 7.4 and the 1 server is RHEL7.7, and you upgrade the CDH version from 5.14.4 to 5.16.2, can you proceed with finding the problem in advance?   

How can I download CDH 6.3.3 Community Edition?

Hi,How can i convert integer value like 20190202 into Date format like 2019-02-02 in the NiFi flowfile content?

My Metata data are available in PostgreSQL Database . I need to Migrate the meta data to MySql . what are the steps to be followed 

Hi Experts,   Can you please share me the link to download binaries for HDF version 3.3.0 and HDF Management pack version 3.3.0 for SLES 12 SP3 ?   Support matrix always says SLES 12 SP3 is supported with HDF 3.3.0. But if you open release notes of HDF 3.3.0, i dont see any option to download the HDF and HDF management pack for SLES 12 SP3.  Release Notes contains link to download for SLES 12 SP1 only.   Can you please share me the link to download binaries for HDF version 3.3.0 and HDF Management pack version 3.3.0 for SLES 12 SP3 ? Thanks  

Hi,Trying to add to a CDH 6.3.2 cluster (managed with CM 6.3.1) a CDSW 1.7.2 trial on two new hosts (master and worker). The CDSW parcel is downloaded, distributed and activated. Unfortunately the CDSW service is unavailable in the cluster's "Add Service" list.Would appreciate any assistance in troubleshooting.

Hi, I'm getting the JSON error when I'm making the post request. Please note that it works perfectly fine in Postman. Can some please help me out thanks. Post body Template   { "format": "CSV", "filter": { "createdAt": { "startAt": 2020-06-20 16:33:19.780Z, "endAt": 2020-06-21 16:33:19.780Z }, "activityTypeIds": [ 1 ] }}   Error code : "Invalid JSON. Unexpected character ('-' (code 45)): was expecting comma to separate Object entries"}]}

Hi, I'm trying to use Cloudera quickstart on my windows through docker. I downloaded docker and did the below steps. 1) Open windows power shell, docker pull cloudera/quickstart:latest2) docker images (Able to see the quickstart image)3) docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 7180:7180 -p 80:80 -p 8888:8888 -p 7187:7187 docker.io/cloudera/quickstart:latest /usr/bin/docker-quickstart When running command 3, I'm unable to see the quickstart launching up, rather I'm seeing the container status as 'Exited with code 39'. I'm unable to get past this step. Kindly let me know if I'm missing anything from my end or if there are any similar open/closed issues on windows as such. Thanks,Venkatesh Raman  

Hello, I downloaded and imported an HDP 3.01 sandbox into Virtualbox 6.1. When I try to write to HDFS, I can connect to the Namenode, but I'm creating zero-byte files because my client can't directly access the Namenode in the HDP Sandbox from my Windows client (could only be replicated to 0 nodes instead of minReplication (=1)) This is happening because the Namenode creates a filehandle when I try to write the file, returns the URI of the Datanode to my client, but my client can't access the URI. I read this related article on how to resolve this for HDP 2.5 and determined the changes I needed to do for HDP 3.0.1 (adding a port for the Datanode 50010) as follows:cd /sandbox/proxyvi proxy-deploy.sh So I understand that I need to create a route from my Windows host to the docker container in the Sandbox. I've done that - but the problem I'm running into is that the docker container's network is DOWN (#5) in the list below: Questions:(1) Should the docker network

We're in the process of upgrading from CDH 5 to CDH 6.3.2. We have a management application written in java which uses SparkLauncher to start spark jobs on yarn. We've updated libraries in both the management application and the spark applications from spark 1.x to 2.x When a spark job is started on yarn, it fails due to missing Hive libraries. Adding these libraries via SparkLauncher.addJar() results in the classes being unavailable from the desired classloader.   Code which adds libraries:     final String JAR_DIR = "/opt/cloudera/parcels/CDH/jars/";File jarDir = new File(JAR_DIR);for (String filename : jarDir.list()) {String jar = JAR_DIR + filename;log.info("adding jar: " + jar); launcher.addJar(jar);}     Resulting exception:     2020-05-20 00:21:01 [,,,] [] INFO ApplicationMaster:57 - Final app status: FAILED, exitCode: 15, (reason: User class threw exception: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgum

Update: template for flow titled CountryCodeLookup.xml is in GitHub: https://github.com/johndcal/NiFi-Tutorial-5-FilesHello,Can anyone help me with this?  Below is the data that I am trying to parse and the schema being used to parse it.  Should I be looking at something else?  Will provide more info if needed.Thanks,jc Data:name,loc_id,launch,norad_number,classificationNOAA 19,US,2009-02-06,33591,WeatherGOES 13,US,2006-05-24,29155,TVSAUDISAT 1C,SA,2002-12-20,27607,EducationMETEOR M2,RU,2014-07-08,40069,WeatherMEASAT 3,MY,2006-12-11,29648,TV Schema:{"type": "record","namespace": "satellites","name": "satdata_in","fields": [{ "name": "name", "type": "string" },{ "name": "loc_id", "type": "string" },{ "name": "launch", "type": "string" },{ "name": "norad_number", "type": "int" },{ "name": "classification", "type": "string" }]}

Hello. I have a HDF 3.2 cluster shared for different developing groups. It is running correctly but it's quickly growing bigger and bigger and we've decided to set up different isolated groups of NiFi nodes in order to prevent that a collapse on one project affects all the projects. Our first idea was to install new HDF clusters but we want to manage all the resources with one unique Ambari console and also we want to share other resources (Kafka, NiFi Registry) between all the groups, but I've read that older versions of Ambari doesn't support managing multiple clusters (I don't know if this is still true). What would be the recommended approach? It is possible to manage isolated groups in one unique cluster or multiple clusters managed by different ambari each is the only solution? PD: If upgrading to CDF last version is necessary, that's something we're open to (we're already planning to do this in the near future). Many thanks for your help 

I want to create roles for my hive database management in Hue, but when I try to create roles using Sentry in Hue, I keep on getting this message : Unable to obtain groups in user :I've already input my admin user to Sentry admin user

Hello guys!   I'm trying to run the following commands in Hive: use dataengsandbox;hive.mapred.supports.subdirectories=true;insert overwrite directory '/tmp/hmb/test' select max(id) from nodename;   If I remove the text marked in orange it works without issues, but I want my output to be written into a HDFS directory.   Here is the error that I've received: Error while compiling statement: FAILED: SemanticException No valid privileges User lrm0613 does not have privileges for QUERY The required privileges: Server=server1->URI=hdfs://nameservice1/tmp/hmb/test->action=*;

Hi all  I am running the following ambari API in order to find the CPU count on data node machine  curl -H "X-Requested-By: ambari" -u admin:admin -X GET http://master:8080/api/v1/clusters/hdp_test/hosts?fields=Hosts/host_name,Hosts/public_host_name,Hosts/cpu_count&host_compnents/HostRoles/component_name.in(DATANODE) we get the error about:  {  "status" : 400,  "message" : "The properties [host_compnents/HostRoles/component_name] specified in the request or predicate are not supported for the resource type Host."  The target is to find the number of cpu on data node machine as we can do it on datanode linux machine nproc16

Hi,I've got an INPUT-json that looks like this:{"vendor" : "myINC","beginTime" : "2020-01-21T10:00:00+08:00","userLabel" : "DB","Types" : [ {"Name" : "type001","subTypes" : [ {"subtype003" : {"counter1" : "0","counter5" : "0","counter54" : "0"}},{"subtype001": {"counter002": "0","counter5": "0","counter65": "0"}} ]}, {"Name" : "type002","subTypes" : [ {"subtype006" : {"counter1" : "0","counter5" : "0","counter54" : "0"}},{"subtype009" : {"counter002": "0","counter5": "0","counter65": "0"}}]} ]} and would need a OUTPUT-json like this: {"vendor" : "myINC","beginTime" : "2020-01-21T10:00:00+08:00","userLabel" : "DG","Types" : [ {"Name" : "type001","subType" : [ {"Name": "subtype003","Counters": [{"name": "counter1","value": "0"},{"name": "counter5","value": "0"},{"name": "counter54","value": "0"} ]},{"Name": "subtype001","Counters": [{"name": "counter002","value": "0"},{"name": "counter5","value": "0"},{"name": "counter65","value": "0"}]} ]}, {"Name" : "type002","subType" : [ {"

I have the following input formatINPUT.JSON  [ { "name": "adam", "age": 12, "address": { "city": "delhi", "country": "india", "zip": "110011" } }, { "name": "louis", "age": 23, "address": { "city": "goa", "country": "india", "zip": "110022" } }]  After applying jolt transformation i want to get the following outputDESIRED OUTPUT.JSON  [ { "name": "adam", "age": 12, "address": { "current_city": "delhi", //change here "current_country": "india", //change here "zipode": "110011" //change here } }, { "name": "louis", "age": 23, "address": { "current_city": "goa", //change here "current_country": "india", //change here "zipode": "110022"