Solved

Hi friends,Im in CDH 6.2 and hence trying to check jps command through cmd prompt.Not sure why it is not showing up hadoop services on issuing it and hence returning with below error [root@cloudera bin]# jpsbash: jps: command not found... Kindly help me out with this.Regards,GTA

I am trying to create a very basic Custom HiveMetastore Event Listener using the following blog. I have successfully created the JAR file and have executed the following steps on beeline but the event listener logs are still not getting displayed in /var/log/hive/hivemetastore.logFollowed Tutorial=> https://towardsdatascience.com/apache-hive-hooks-and-metastore-listeners-a-tale-of-your-metadata-903b751ee99fGit Repository=> https://github.com/mesmacosta/hive-custom-metastore-listenerStep 1: Added JAR to hdfshadoop fs -put custom-metastore-listener.jar /user/data/Step 2: Added JAR from HDFS to the hive beeline> add jar hdfs:///user/data/custom-metastore-listener.jar; Step3: Set event listenerbeeline> set hive.metastore.event.listeners=com.medium.hive.metastore.listerner.CustomListener;Step4: Run create table command and check hivemetastore logs.beeline> CREATE TABLE test(name string) STORED AS ORC;  But the log does not show anything from a custom

Hi,Would appreciate any advice, how to solve a problem with terminal access from a CDSW session. Let me highlight that I can launch a session, however within a session I am unable to access the terminal – please see the attached screenshot with HTTP ERROR 401.The networking requirements are met, in particular:IPv6 is enabledCDSW hosts are within the same subnet as the CDH clusterDNS is configured with the relevant A record for domain name, CNAME record for wildcard domain, and a reverse PTR domain record (please see the enclosed response to ping command, where the DNS resolves the terminal's FQDN to CDSW master node's IP)No iptables rules were enabledSElinux is disabled[cloud-user@cdh-control-01 ~]$ ping -c1 tty-jidv65sd8630btx4.cdsw.<intranetdomain>PING cdsw.<intranetdomain> (10.133.210.200) 56(84) bytes of data.64 bytes from cdsw.<intranetdomain> (10.133.210.200): icmp_seq=1 ttl=60 time=0.884 ms--- cdsw.<intranetdomain> ping statistics ---1 pack

Hi,I am looking for Cloudera CDP, Hive with Kerberos security and encryption to setup a basic development environment. What kind of license policy is required? Thanks,Tulasi

Hi All,   I am trying to generate separate UUID for JSON array of objects. I tried to use ${UUID()} in JOLT specification, but this is generating same UUID for all the elements.   e.g. Input: =============== {  "aliases": [    "Alias1",    " Alias2"   ]}   JOLT Specification ===================== [{"operation": "shift","spec": {"aliases": {"*": "alias[&0].name"}}},{"operation": "modify-default-beta","spec": {"alias": {"*": {"id": "${UUID()}"}}}}]Current Output: ===================== "alias" : [ {"name" : "Alias1","id" : "2948446b-6213-44ba-b81f-c4e5f7dab2eb"}, {"name" : " Alias2","id" : "2948446b-6213-44ba-b81f-c4e5f7dab2eb"} ]   Any pointers will be greatly appreciated.   Thanks, Biswa 

I have designed flow in nifi for XML to JSON converter. And for that, I have used GenerateFlow File and JoltTransformRecord(not JoltTransform JSON). But I am getting null value in the JSON file with the schema(output in screen shot).Please find my jolt specification:[{"operation":"shift","spec":{"to":"Receiver","from":"Sender","heading":"Heading","body":"Message"}}] Jolt Transformation DSL is chain. As I am trying to convert XML to JSON and make a change in element name in one go through JoltTransformRecord. I think JoltTransformRecord accepts any form of payload and then converts it into JSON format and then applies the change in elements of the JSON file. OutputInput file  Please help why this happens. thanks

Hello I installed CDP 7.1.1 because I wanted to try Ozone. I played a little with apache Ozone in the past and wanted to see how it is in Cloudera. After installation, I went ahead and tried to create a volume using this command (which complies with the documentation):ozone sh volume create --quota=1TB --user=hdfs /tests I got this error message:"Service ID or host name must not be omitted when ozone.om.service.ids is defined." I searched the internet for the error and did not find anything useful. I looked for ozone-site.xml file and found that ozone.om.service.id was "ozone1". But I do not know where in the command should I specify it. the command syntax does not take any service id or host name. Why am I getting this error ? how can I specify service id or host name so the volume will be created ? Thanks Guy

I am installing Cloudera 7 Data Center on AWS m5.xlarge (as a single node cluster) and ran into this problem - logs from /var/logs/cloudera-scm-server 020-04-26 16:26:28,355 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Failed to download manifest. Status code: 401 URI: https://archive.cloudera.com/p/cdh7/7.0/parcels/manifest.json2020-04-26 16:26:28,355 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Could not retrieve repository info for repo https://archive.cloudera.com/p/cdh7/7.0/parcels/. Got HTTP response code 4012020-04-26 16:26:28,454 WARN ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Invalid parcel in manifest : CDH-7.0.3-1.cdh7.0.3.p0.1635019-stretch.parcel2020-04-26 16:26:34,884 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands2020-04-26 16:26:34,884 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThre

When I enable the Ranger Service from HIVE in Cloudera Manager, it's getting an error like:"Hive Impersonation is enabled for Hive Server2 role 'HiveServer2 (node1)'. Hive Impersonation should be disabled to enable Hive authorization using Ranger"I want to access control from Ranger, But I cant Anybody, please help me.

It says a problem with the current version of jquery. URL :  "http://<myip>:8042/static/jquery/jquery-3.3.1.min.js" Installed version : 3.3.1 Fixed version : 3.5.0Vulnerability link:" https://www.tenable.com/plugins/nessus/136929"Finding description: JQuery 1.2 > 3.50 XSS Is there like a workaround to overcome this vulnerability ?  or is necessary to upgrade to a newer version of HDP?Would be possible to upgrade jquery version only ? Thanks in advance..! 

HI, I am trying to install a custom service on Cloudera. This service requires some libs for it to run.I have created the CSD sdl and control.sh file and created a jar out of it. The issue is that I am not getting where to put service libs in Cloudera. Should these be bundled in CSD jar itself? or should these be installed as parcel. If as parcel, how to install it. I do not see much doc on installing/deploying local parcel on Cloudera manager. Please help. 

I have a NIFI image running in openshift and a postgres in the cloud "owned" by another department.   They sent to us a certificate(".crt") and a key(".key"), since the log-in is made trough client certificate, instead of username and password. I have succeeded to log-in in pgAdmin 4, but not to connect the NIFI to the Postgres with certificate and key.   I have uploaded the certificate and the key to the image (using a secret and mounting it) so if I go to the pod terminal I can access it.   But, when I pass to a DBCPConnectionPoll service the connection string bellow and activates an ExecuteSQL processor, I receive an exception that the certificate is not valid, as follows:   ERRORExecuteSQL[id=...] Unable to execute SQL query <...>;due to java.sql.SQLException: Cannot create a PoolableConnectionFactory (FATAL: connection requires a valid client certificate).No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLEx

Hi All,I am new on Ambari. I use Rhel7 and i installed Ambari server and have been configured it succefully on my server.So, i want to install Airflow on this existing ambari server. I've tried to install it with the command "ambari-server install-mpack --mpack=/opt/airflow-service-mpack.tar.gz"and i got this python message : Using python /usr/bin/python Usage: {start|stop|restart|setup|setup-jce|upgrade|status|upgradestack|setup-ldap|sync-ldap|set-current|setup-security|setup-sso|refresh-stack-hash|backup|restore|update-host-names|check-database|db-cleanup|enable-stack} [options] Use --help to get details on options available. Or, simply invoke ambari-server.py --help to print the options. Need help Please. Thanks, Math

Hi,I need to setup a 5 node cluster with Hadoop 3.1.0 and Spark 2.4.5 . Someone recommended to use Ambari to do so. I checked Ambari but it seems Ambari can be used only to install HDP and latest HDP do not support Spark 2.4.5 version.  Please suggest in this aspect, what will be the best way to setup the required big data cluster.

Hi, do you know how can I have a replication between two clusters? for example, I want my cdh cluster 1 to be replicated with cdh cluster 2 in real time, so once this cluster is down, we still have the other cluster to work on with

Hi, I have been trying to setup nifi site to site via our web proxy, but am having issues.  I have reviewed the documentation, but can't get past the files we want to transfer sitting in the queue.  My client side setup is relatively simple:-A GETFILE processor that monitors a local directory on the server-A Nifi Flow via a Remote Process Group connection to the remote nifi. The Remote processor group is configured to hit our proxy using http on port 8080.  When configured, it picks up the Input of the remote connection in the Remote Group at  http://remoteserver.org:9090/nifi So the connectivity seems to be there, but the files simply sit in the queued state and never try to use the remote connection.   Am I missing a step on the client side (maybe an output processor)?  Is there something I can have the remote side admin check to enable data flow? Sorry for the beginner question, but I just can't seem to find the answer afte

Hello,I have a question for CDH Upgrade.I have a problem with CPU scheduling in RHEL7.4, so I need to upgrade to RHEL7.7 * Information<AS-IS>- CDH: 5.14.4- OS: RHEL 7.4(3.10.0-693-e17.x86_64) <TO-BE>- CDH: 5.16.2- OS: RHEL 7.7(3.10.0-1062-e17.x86_64) ISSUE: No Internet Environment (Separate Network Env) Q1) Is there any problem with system library reference?Q2) Is there a library linkage problem between CDH 5.14.4 and CDH5.16.2?Q3) If the 3 servers are RHEL 7.4 and the 1 server is RHEL7.7, and you upgrade the CDH version from 5.14.4 to 5.16.2, can you proceed with finding the problem in advance?   

How can I download CDH 6.3.3 Community Edition?

Hi,How can i convert integer value like 20190202 into Date format like 2019-02-02 in the NiFi flowfile content?

My Metata data are available in PostgreSQL Database . I need to Migrate the meta data to MySql . what are the steps to be followed 

Hi Experts,   Can you please share me the link to download binaries for HDF version 3.3.0 and HDF Management pack version 3.3.0 for SLES 12 SP3 ?   Support matrix always says SLES 12 SP3 is supported with HDF 3.3.0. But if you open release notes of HDF 3.3.0, i dont see any option to download the HDF and HDF management pack for SLES 12 SP3.  Release Notes contains link to download for SLES 12 SP1 only.   Can you please share me the link to download binaries for HDF version 3.3.0 and HDF Management pack version 3.3.0 for SLES 12 SP3 ? Thanks  

Hi,Trying to add to a CDH 6.3.2 cluster (managed with CM 6.3.1) a CDSW 1.7.2 trial on two new hosts (master and worker). The CDSW parcel is downloaded, distributed and activated. Unfortunately the CDSW service is unavailable in the cluster's "Add Service" list.Would appreciate any assistance in troubleshooting.

Hi, I'm getting the JSON error when I'm making the post request. Please note that it works perfectly fine in Postman. Can some please help me out thanks. Post body Template   { "format": "CSV", "filter": { "createdAt": { "startAt": 2020-06-20 16:33:19.780Z, "endAt": 2020-06-21 16:33:19.780Z }, "activityTypeIds": [ 1 ] }}   Error code : "Invalid JSON. Unexpected character ('-' (code 45)): was expecting comma to separate Object entries"}]}

Hi, I'm trying to use Cloudera quickstart on my windows through docker. I downloaded docker and did the below steps. 1) Open windows power shell, docker pull cloudera/quickstart:latest2) docker images (Able to see the quickstart image)3) docker run --hostname=quickstart.cloudera --privileged=true -t -i -p 7180:7180 -p 80:80 -p 8888:8888 -p 7187:7187 docker.io/cloudera/quickstart:latest /usr/bin/docker-quickstart When running command 3, I'm unable to see the quickstart launching up, rather I'm seeing the container status as 'Exited with code 39'. I'm unable to get past this step. Kindly let me know if I'm missing anything from my end or if there are any similar open/closed issues on windows as such. Thanks,Venkatesh Raman  

Hello, I downloaded and imported an HDP 3.01 sandbox into Virtualbox 6.1. When I try to write to HDFS, I can connect to the Namenode, but I'm creating zero-byte files because my client can't directly access the Namenode in the HDP Sandbox from my Windows client (could only be replicated to 0 nodes instead of minReplication (=1)) This is happening because the Namenode creates a filehandle when I try to write the file, returns the URI of the Datanode to my client, but my client can't access the URI. I read this related article on how to resolve this for HDP 2.5 and determined the changes I needed to do for HDP 3.0.1 (adding a port for the Datanode 50010) as follows:cd /sandbox/proxyvi proxy-deploy.sh So I understand that I need to create a route from my Windows host to the docker container in the Sandbox. I've done that - but the problem I'm running into is that the docker container's network is DOWN (#5) in the list below: Questions:(1) Should the docker network