Q&A

I am using the parameter contest in that their is parameter, which is a sensitive parameter and i need to call in routeOnattribute for our requirement. how can i achieve this calling the sensitive parameter from parameterContests.Please help me out. Thanks

NiFi Version: 1.16.1Java 11 I noticed the logback.xml is the setting for the nifi logs. However, the file size of the nifi-app.log is exceeding the maxFileSize set on logback.xml. This is my logback.xm file:  

 is TEZ UI working on CDP 7.1.7 ?if yes, how can we enable it ? Regards ; 

Hi Team,    We had lost the master passphrase for one of the navencrypt server. Can you help us on resetting the master passphrase to new one.Thanks,Ashok

in zeppelin i run show databases; and show tables works but when pyspark to select * from tablesget error permission denied user = anonymous  java.security.AccessControlException: Permission denied: user=anonymous, access=READ, inode="/warehouse/tablespace/managed/hive when i check getfacl in /warehouse/tablespace/managed/hive user zeppelin can rwx the directory why does zeppelin read as user anonymous ?  anybody can help ? thanks 

Hello Experts, I have custom processor code where I am using 2 controller services as input property.DBCPService (jdbc connection pool) works fine without any issue,but when I try to take input as MongoDBClientService (mongodb connection pool controller service) I get MongoDBClientService NoClassDefFoundError error during project build.  public class TrackRequestStatus extends AbstractProcessor { public static final PropertyDescriptor DBDC_SERVICE = new PropertyDescriptor.Builder() .name("JDBC Connection Pool") .description("JDBC Connection Pool") .required(false) .dynamic(true) .identifiesControllerService(DBCPService.class) //.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build(); static final PropertyDescriptor CLIENT_SERVICE = new PropertyDescriptor.Builder() .name("mongo-client-service") .displayName("MongoDB Client Service") .description("If configured, this p

I set up a Haproxy service:listen ranger_admin    bind 0.0.0.0:6080    mode tcp    server ranger_node1:6080     server ranger_node2:6080 Then I refer to the document，https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/fault-tolerance/content/configuring_ranger_admin_ha_with_ssl.htmlfrom 32 steps to the end. I found the Ranger authorization invalid.Namenode log error:WARN client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(182)) - Error getting policies. secureMode=true, user=nn/node1@TEST.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=Captain_hadoopWARN client.RangerAdminRESTClient (RangerAdminRESTClient.java:getServiceTagsIfUpdated(365)) - Error getting tags. secureMode=true, user=nn/node1@TEST.COM (auth:KERBEROS), response={"httpStatusCode":401,"statusCode":0}, serviceName=Captain_hadoop Ranger admin access log error:"GET /service/plugin

Hello, I would like to write a log file to /var/log/cdp/hdfs/audit/solr/spool directory.I added the following settings to Cloudera Manger's HDFS Advanced ranger-solr-audit.xml.xasecure.audit.destination.solr.batch.filespool.enabled=truexasecure.audit.destination.solr.batch.filespool.dir=/var/log/cdp/hdfs/audit/solr/spoolxasecure.audit.destination.solr.batch.filespool.archive.dir=/var/log/cdp/hdfs/audit/solr/spool/archivexasecure.audit.destination.solr.batch.filespool.archive.max.files=50/var/log/cdp/hdfs/audit/solr/spool directory was missing, creating spool and archive directories.Should I modify install.properties or ranger-hdfs-plugin/install/conf.templates/enable/ranger-hdfs-audit.xml directly?Or, I would like to know if there are any additional parts that need to be set up in Cloudera Manager. Any help would be highly appreciated.

Hi All,I am running 4 node Nifi cluster with each node having 8 core and 32 GB or RAM. it was running fine until I introduced QueyRecord processor to process huge metric data (14 GB per min). the CPU now consistently remaining above 85% and memory above 90% (buffering too much). Can someone let me know how to diagnose the issue and find the root cause.  Thanks in advance 

Hello, We're looking for some advice on how to identify possible improvements on our current NiFi flow that is a copy of what we had implemented in Flume, both running on AWS with CDP Private Cloud. Our use case begins with an HTTPS listener that in its simplest form receives GET requests and processes them and writes events into Kafka. Between those two ends, we perform some validations and transformations:- we convert the request parameters into JSON format (using 5 processors with AttributesToJSON and ReplaceText)- convert JSON format to rename properties using a JoltTransformJSON- validations using a couple of RouteOnAttribute + ReplaceText to confirm mandatory fields and transform null values- mapping of numeric values into strings using reference data (using UpdateAttribute, RouteOnAttribute and 2 LookupAttribute)- convert JSON into Avro format with ConvertRecord- returning the HTTP response + publishing each individual record to Kafka In total, we have around

How can I download Cloudera Manager 7.x Installer

I am fetching data from table for employees and each employees has maximum 7 Records. I am splitting data based on the IDs and week number column. I need to merge the data in such a way that each flow file contains 100 records and all the records of each employees comes into that flow filefor exampleMy flow file already has 98 records and next employee has 7 records so, that employee's data should not be the part of that flow file. It should come in next flow file likewise.....  While merging the FFs order is also important.How can I do that. I am not familiar with all the processors of NiFi

Hello, I am unable to launch Yarn Queue Manager Web UI and get error - Failed to load the Yarn Queue Manager UI. Reload the UI after fixing the issue. Reason - No active RM Found. We have 2 RM (Active/ Standby) yet facing this error.  Reques any help / guidance to resolve this. CM 7.6.1Cloudera Runtime - 7.1.7

Hi guys, I am facing a problem with Nifi I have about 80G data in queues that make backlogs and I need the data to be ingested faster and but I can't clear the queues in case of missing files, Please help me solve this issue, thanks a lot. NIFI full queues:((((

I read a Document guide of Cloudera on this Schedule Job link. The problem is I don't have access to "Cloudera Data Platform (CDP) management console" (which looks like below):. I only have access to Cloudera Web server UI on xxx.xxx.xxx.xxx:7180, which looks like this:Please note that we run Cloudera on a host machine that runs centos, and I have to ssh to that machine there isn't any UI like the first picture, there is only a black window with line commands. I only have the webserver. Same problem for many other guides on the website, they're always require you have access to CDP management console UI

Hi AllWe have our hive database on Postgress and wanted to migrate it to MySQL,Is there any easy way to do it? Thanks

we want to find the **approach / test /cli / API** that gives us the results about HeartBeat Lost between Ambari agent to Ambari serverHeartBeat Lost could be as results of poor connection between Ambari agent to Ambari server or because Ambari server was down for along time , etc Note - from Ambari GUI the machine with **HeartBeat Lost** state usually colored by yellow state    clarification:  the case as described here appears when `ambari-agent status` is in running state as the followingambari-agent statusFound ambari-agent PID: 119315ambari-agent running.Agent PID at: /run/ambari-agent/ambari-agent.pidAgent out at: /var/log/ambari-agent/ambari-agent.outAgent log at: /var/log/ambari-agent/ambari-agent.log

Hi Team,Can someone please help with best practices for tuning queue manager in CDP? Thanks!

Hello,  I am trying to use hbase-spark in order to query over Hbase with spark-sql, but I am stuck with some of these exceptions: java.lang.NullPointerExceptionor java.lang.NoSuchMethodError: org.apache.hadoop.hbase.util.ByteStringer.wrap([B)Lcom/google/protobuf/ByteString; Contents : 1- Details on the platform 2- Details on the problem3- Description of the attached content 1- A Cloudera cluster running CDH-6.3.4 2- I run a (POC) Java application with spark-submit in yarn cluster mode. What the application does is to sequencially  :  - Create a Hbase table and populate it using the Java API - Use a spark-session to read the hbase table (using .format("org.apache.hadoop.hbase.spark") ) - Perform some queries on the Dataframe.  For now, I only got partial success on the last step. I can show the contents of the dataframe with this for example : Dataset<Row> sqlDF1 = sqlContext.sql("SELECT * FR

Hello.I would like to help about check the history log of sql query command of each account on Impala and Hive. I have been check on `/var/log/hive` (the Hive log) but not expected.Can you figure out for me where is the log save on ? And any have another way to check it ?.Thank you for you help.

ERROR orm.CompilationManager: Sqoop requires a JDK that can compile Java code cloudera

It looks like CDH 7.1.7 SP1 is vulnerable to CVE-2022-42889.   Here is the announcement from Apache which indicates the mitigation is to "Upgrade to Apache Commons Text 1.10.0".   https://lists.apache.org/thread/n2bd4vdsgkqh2tm14l1wyc3jyol7s1om   There was another community thread about Text4Shell in NiFi, but CVE-2022-42889 is NOT just a NiFi issue.   CDH 7.1.7 SP1 (even p1057) includes the vulnerable common-jars 1.6/1.7 and 1.9.   /opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1057.32088321/jars/commons-text-1.6.jar/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1057.32088321/jars/commons-text-1.7.jar/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1057.32088321/jars/commons-text-1.9.jar.../opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1057.32088321/lib/solr/server/solr-webapp/webapp/WEB-INF/lib/commons-text-1.6.jar/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1057.32088321/lib/streams_messaging_manager/libs/commons-text-1.9.jar/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.

Hi Team, Spark job is hang or struck due to below error. can any one please help here.. 22/12/05 22:29:55 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x5e29988e0x0, quorum=localhost:2181, baseZNode=/hbase22/12/05 22:29:55 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)22/12/05 22:29:55 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnectjava.net.ConnectException: Connection refusedat sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1126).... 22/12/05 22:30:12 INFO zookeeper.ClientCnxn: Op

Hi guys, do you know how to get the version mapping such as hadoop between cdh and apache community? Just like how to get the difference of hadoop between hadoop-cdh5.16.2 and 2.6.0-cdh5.5.0.

Starting Apache NIFI 1.16.0 on windows 10 , with jdk 1.8.0_45 installed is failing to start with the following error in nidi-app.log:Caused by: java.lang.IllegalStateException: Flow controller TLS configuration is invalid              at org.apache.nifi.controller.FlowController.<init>(FlowController.java:501)              at org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:403)              at org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:86)              at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:169)         &n