Solved

Hello,I am facing an issue with Impala queries, though the query status says 'finished' on Impala status page it still shows 'executing'. I was following Cloudera Article : https://my.cloudera.com/knowledge/Finished-Queries-show-as-Executing-in-the-Cloudera-Manager?id=71576the difference  in my case is queries were run through impala shell via jdbc connection.would the resolution outlined in the article be applicable, for queries submitted through impala jdbc connection? and secondly what would be the fix for such queries which are in hung state. RegardsAmn

Hello Team,     We were planning to implement Role-Based-Access-Control(RBAC) for our impala tables we tried Sentry but there is a prerequisite of installing Kerberos which we don't want to use. So Apache Ranger is the other option we have but the latest CDH version 6.3 has Impala 3.2 which does not support Ranger. So just wanted to if there is going to be release of new CDH bundle which will have Impala 3.3.0 and support for ranger? What will be approximate release date of the new CDH bundle?       Regards Parth

  Hello  i need to connect Nifi to Oracle DB but i faced a problem in the connection. i copy Ojdbc.jar to lib directory     i choose QueryDatabaseTable processor ( i am not sure if it's correct )  -  Database Connection URL :  jdbc:oracle:thin:@//HostName:XXXX/XXXXXXX -  the Database Driver Class Name : oracle.jdbc.driver.OracleDriver -  Database Driver Location : C:\Users\d1156\Desktop\nifi-1.10.0\lib\ojdbc8.jar   after i start the processor this error appears : unable to execute SQL select query - cannot create JDBC driver of class ' Oracle.jdbc.driver.oracleDriver'    Thank you .  

I am trying to use Ranger for Authorization with Hive and HDFS. But somehow Ranger doesn't recognize Hive or HDFS. I don't see the default policies in Ranger for Hive and HDFS. I activated the Ranger Plugins for Hive and HDFS and checked the Advanced ranger-hive-plugin-properties and Advanced ranger-hdfs-plugin-properties. For me everything seems fine, but the services don't appear in Ranger. I also activated Ranger as authorization method for Hive. But that also didn't helpThis is how my configuration looks:Ranger Services:Ranger Plugins:Advanced ranger-hdfs-plugin-properties.  Advanced ranger-hive-plugin-properties What am I missing here? I would be happy about hints!Thanks

I have a problem with the HandleHTTPRequest Processor, I'm using it to receive Data from Sensors via a path and a port, the problem is that even If sensors send Data and when using wireshark I see Data coming from sensors to my Integration server in the port assigned , but the processor doesn't show anything until I restart NIFI , Data reach the processor for some minutes or sometime for 2 days and after it stop again , don't know what is the problem , because restarting Nifi every time is not a good solution for me , someone can help me on that ?

Hi,  I am wondering if it is possible to get Service-Wide configurations via read_config method of the RoleConfigGroupsResourceApi class. https://archive.cloudera.com/cm6/6.3.0/generic/jar/cm_api/swagger-html-sdk-docs/python/docs/RoleConfigGroupsResourceApi.html#read_config   The read_roles method of RolesResourceAPI returns theses roles:   CD-HDFS-eHtEMKVf-DATANODE-BASECD-HDFS-eHtEMKVf-SECONDARYNAMENODE-BASECD-HDFS-eHtEMKVf-HTTPFS-BASECD-HDFS-eHtEMKVf-DATANODE-BASECD-HDFS-eHtEMKVf-DATANODE-BASECD-HDFS-eHtEMKVf-NAMENODE-BASE    But when I query all these roles, I cannot find the Service-Wide property of Advanced configuration for core-site.xml   Reading configuration for CD-HDFS-eHtEMKVf-DATANODE-BASE{'items': [{'default': None, 'description': 'For advanced use only, key-value pairs (one on ' "each line) to be inserted into a role's " 'environment. Applies to configurations of this '

CDH6.2.1imapla 3.2.0-CDH6.2.1 the below errors happened every day,  i also find some similar issue on issue.apache.org.  it seems a bug .  here is some issue list :https://issues.apache.org/jira/browse/IMPALA-8561https://github.com/cloudera/Impala/commit/9e4bf4c99f48a29e1c1d46d49c5f8452b777e2db it seems this issue also happened on Impala3.3, not juse impala 3.2, but it's fixed in 3.3. so, Cloudera support, how to fix this issue on imapla-3.2( CDH6.2.1), this issue is so critical cause many users encounter this issue and ask me what's happening, and i just can tell them this is a bug and i can't do anything. may i ask another question ? if Impala3.2 can't resolve this issue ,may i update imapla 3,2 to impala 3.3 , i mean i don't update Cloudera Manager  and CDH other components , just update impala . i hope Cloudera Supporter can give me advises , thank you very much .

hi all   we are using the following HDP cluster with ambari ,   list of nodes and their RHEL version   3 masters machines ( with namenode & resource manager ) , installed on RHEL 7.2 312 DATA-NODES machines , installed on RHEL 7.2 5 kafka machines , installed on RHEL 7.2   now we want to add the following machines to the cluster but with RHEL 7.5   85 DATA-NODES machines , should be installed on RHEL 7.5 version 2 kafka brokers machines , should be installed on RHEL 7.5 version   so my question is   can we mix in HDP cluster RHEL 7.2 with RHEL  7.5 version ?          

I would like to analyze a big data (0.9 TB after unzipping) in a cluster with 14 nodes and 39 cores (Azure HDInsight/Kafka). But it's very slow. Here what I do:   1. Data is downloaded from here. 2. val data = spark.read.json(path) ---- it crashes. Data are stored in HDFS.  3. val rdd = sc.textFile(path) ... then rdd.count() .... also crashes 4. rdd.take(10) , ... these are ok 5. It was not possible to unzip the file; I read it with data.json.gz Any suggestion? How I can read it with json reader? Thanks

I am installing Cloudera Enterprise 6.3.x and I am at step 5 (https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/prepare_cm_database.html#scm_prepare_syntax) .   I could not find /opt/cloudera/cm/schema/scm_prepare_database.sh file in my installation   Please help

Hi Cloudera,  the cloudera altus director cannot create a database(s) for CM and throws this error, even if it has a root user access to the external AWS RDS posgre database:       org.postgresql.util.PSQLException: ERROR: must be member of role "cmadmin_dkndwlxo"     I could not find any hint in the docs what this exact role means and why the root user must be member of this role. Postgre error log:     2019-11-25 11:07:06 UTC:10.150.1.7(43878):dbroot@postgres:[12215]:ERROR: must be member of role "cmadmin_dkndwlxo"2019-11-25 11:07:06 UTC:10.150.1.7(43878):dbroot@postgres:[12215]:STATEMENT: create database scm_75ilop0jdikuhinujsfs7l5m1n owner cmadmin_dkndwlxo encoding 'UTF8'2019-11-25 11:07:17 UTC:10.150.1.7(43880):dbroot@postgres:[12313]:ERROR: must be member of role "cmadmin_wrhjespw"2019-11-25 11:07:17 UTC:10.150.1.7(43880):dbroot@postgres:[12313]:STATEMENT: create database scm_38kegs9qab7j5l6hgqo069h3am owner cmadmin_wrhjespw encoding 'UTF8'

Hello,   We recently did an upgrade on our Hadoop cluster from 2.6.4 to 3.1.4 - everything went decently - a few random issues that we managed to solve ourselves.But this one - it's for sure something we don't understand.We have a partitioned table with around 5b rows (1Tb+) and it's partitioned into 256 almost even partitions (and is also distributed by a certain, single column).       PARTITIONED BY ( `partfunc` string) CLUSTERED BY ( event_name) SORTED BY ( event_time ASC) INTO 10 BUCKETS ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT

Hi,   I am having issues with the zookeeper (zookeeper-3.4.13) with latest nifi version of Nifi-1.10.0 , below is the stack trace. I am upgrading from nifi-1.9.0 .  let me know if latest version of Nifi needs different version of zookeeper ? or the issue is somewhere else.   Not : With Java 8 and nifi-1.9.0 everything works   Actual Logs using Java 11  2019-11-15 03:40:09,202 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED2019-11-15 03:40:09,241 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting2019-11-15 03:40:09,355 INFO [main] o.apache.nifi.controller.FlowController It appears that no Cluster Coordinator has been Elected yet. Registering for Cluster Coordinator Role.2019-11-15 03:40:09,356 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager CuratorLeaderElectionManager[stopped=true] Registered new Leader Selector for role Cluster Coordinator; this node is an active participant i

Hello experts! I try to use Apache Phoenix on my Docker-based Cloudera Quickstart, for training purposes. I found some amazing blog about this : http://crazyadmins.com/install-and-configure-apache-phoenix-on-cloudera-hadoop-cdh5/ However it doesn't work at me... My current Hbase version is 1.2.0. So if I refer to Apache Phoenix website, I can get the 4.14 version but not a too recent one. Here's what I did :1. wget http://archive.apache.org/dist/phoenix/apache-phoenix-4.14.0-cdh5.13.2/bin/apache-phoenix-4.14.0-cdh5.13.2-bin.tar.gz2. tar -xvf apache-phoenix-4.14.0-cdh5.13.2-bin.tar.gz3. I copy all the content of this extracted tar.gz into my /usr/lib/hbase/lib/ directory (using : cp -a /phoenix/apache-phoenix-4.14.0-cdh5.13.2-bin/. /usr/lib/hbase/lib/apache-phoenix)4. I restart HBase (to be sure, I stop my docker container and I start it again)5. I should be able to use that kind of command : ./sqlline.py localhostBut it doesn't work. I received the following er

I wrote a Kafka producer, that sends some simulated data to a Kafka stream (replication-factor 3, one partition).   Now, I want to access this data by using Hive and/or Spark Streaming.    First approach: Using an external Hive table with KafkaStorageHandler: CREATE EXTERNAL TABLE mydb.kafka_timeseriestest ( description string, version int, ts timestamp, varname string, varvalue float)STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'TBLPROPERTIES ( "kafka.topic" = "testtopic", "kafka.bootstrap.servers"="server1:6667,server2:6667,server3:6667");-- e.g. SELECT max(varvalue) from mydb.kafka_timeseriestest; -- takes too long, and only one Tez task is running   Second approach: Writing a Spark Streaming app, that accesses the Kafka topic: // started with 10 executors, but only one executor is active...JavaInputDStream<ConsumerRecord<String, String>> stream = KafkaUtils.createDirectStream(jssc, LocationStrategies.PreferConsistent(), Consumer

Sqoop import fails while importing tables that has space and special characters in the table name.     command used: sqoop import --connect <>  --username <> --password <> --table "employee details information" --as-parquetfile --delete-target-dir --target-dir  /hdfs/dir/ --m 1 

Will Node labels be supported in the new CDP ?

Hi, I am getting the below error when trying to escape the apostrophe character in HUE Impala.   select * from hi_problem_view where state like 'Won''t fix'   won't fix is a value that I am trying to filter by for records.     Error: AnalysisException: Syntax error in line 3:undefined: state like 'Won''t fix' ^ Encountered: STRING LITERAL Expected: AND, BETWEEN, DIV, GROUP, HAVING, ILIKE, IN, IREGEXP, IS, LIKE, LIMIT, NOT, OFFSET, OR, ORDER, REGEXP, RLIKE, UNION CAUSED BY: Exception: Syntax error

Hi,    I registered to Cloudera and posted 2 questions earlier. Now whenever I login to Cloudera I get the following annoying popup our form submission has failed.This may have been caused by one of the following:Your request timed outA plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page." I removed all my extensions in Chrome. It still kept coming. Then I uninstalled and reinstalled Chrome. No success. I installed Firefox and I get the popup. I don't know what I should do Any suggestions? ThanksGanesh

Hello community: I need you help please. You Know where exactly are all repository from Hadoop(2.6.x and 2.7.x and 3.x) and Ambari 2.6.x and 2.7.x). All URLs are broken. Thank for avance.

Good morning guys,    I have a very strong problem that I am struggling to figure out.   I installed Cloudera 6.1.1. on a set of AWS (5 hosts in total).   My /etc/hosts is the following:   127.0.0.1 localhost.localdomain localhost13.48.140.49 master.sysdatadigital.it master13.48.181.38 slave1.sysdatadigital.it slave113.48.185.39 slave2.sysdatadigital.it slave213.53.62.160 slave3.sysdatadigital.it slave313.48.18.0 slave4.sysdatadigital.it slave4     I am receiving the following error from the Master by all 4 hosts in my cluster:     2019-11-22 08:22:41,355 WARN New I/O boss #15:com.cloudera.server.cmf.HeartbeatRequester: Error requesting heartbeat of host id 072beea9-4ba3-4018-8b7b-fa11fd9eac25java.net.ConnectException: Connection refused: ec2-13-48-18-0.eu-north-1.compute.amazonaws.com/172.31.16.104:9000 at com.ning.http.client.providers.netty.request.NettyConnectListener.onFutureFailure(NettyConnectListener.java:133) at com.ni

I have secured my Nifi instance with help of below tutorials.   https://community.hortonworks.com/content/kbentry/170966/setting-up-a-secure-apache-nifi-registry.html   https://community.hortonworks.com/articles/171173/setting-up-a-secure-nifi-to-integrate-with-a-secur.html     All is working fine.   But now I need to remove HTTPS access and replace it with regular HTTP.   This is not as simple as I thought.   I tried changing web.http and web.https in nifi.properties and restarting nifi.   From     # web properties #nifi.web.war.directory=./libnifi.web.http.host=nifi.web.http.port=nifi.web.http.network.interface.default=nifi.web.https.host=0.0.0.nifi.web.https.port=9443       Back to     # web properties #nifi.web.war.directory=./libnifi.web.http.host=nifi.web.http.port=8888nifi.web.http.network.interface.default=nifi.web.https.host=nifi.web.https.port=       But nifi won't start, saying remote H

I am trying to write a new article, but when I put code boxes into the article, every save changes any urls to <a link tags.  Then another save does same thing to the link text, and ends up trashing the content. Can anyone help?

Hi,We are currently using HDP 3.0 with ambari and we installed 2 nifi nodes. We made some config changes on nifi node01 without restarting both nodes (i only restarted the node01 and not node02). The changes were not working properly so we decided to roll back to the previous configs but whenever i try to start node01 i am getting the following error:Failed to connect node to cluster because local flow is different than cluster flow.  My guess would be that both nodes are out of synch.... How can we fix this issue? Thank you for your help.

Hello, I'm  developing a use case using flume to save data in kafka topic. I'm traing to tail  files from a local data source  directory and  save them in the same  partition of a kafka topic. I want to use as a key of the message the absolute path of the file. I have configured a TailDir source that save the absolute path of a file but I can't configure the kafka sink in order to assign the path as the key of the message in order to save all the events from the same file in the same kafka topic partition. In more details, when I consume the message with the kafka-console-consumer, I can see that the key is 'null' and the absolute path is in the body message. In the flume documentation I didn't find much information on how to deal with this issue, could someone halps me? tnks 🙂 sourceagent3a.sources.source3a.type = TAILDIRagent3a.sources.source3a.filegroups= f1agent3a.sources.source3a.filegroups.f1 = /path/local/data/source/agent3a.sources.source3a.channels= c