Q&A

we need to know which is the responsible file that keeps the zookeeper settings, which one would it be?we need to validate if the "maxClientCnxns" option is the same as what was defined by the CDM in the zookeeper service.

Hello everyoneI cant see pending containers count on Yarn resource pool. How can enable or solve this problem? any suggestions?thanks 

Hello,Is it possible to send dynamic values using some external commands like "curl" so that values will be used in the process?For example we will be send date dynamically then it can be used in SQL query.

I am facing the following error on the cmd:   command: grunt> A =  LOAD 'C:\data\myfile.txt'; or  grunt> a = load 'c:\data\myfile.txt' using PigStorage(' ') as (f1:int,f2:int, f3:int); error:<line 1, column 14>  Unexpected character 'D'2020-03-15 20:56:50,153 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <line 1, column 14>  Unexpected character 'D'Details at logfile: C:\deploy\logs\pig_1584281243336.log     the error trace is as follows-  Pig Stack Trace---------------ERROR 1200: <line 1, column 14>  Unexpected character 'D' Failed to parse: <line 1, column 14>  Unexpected character 'D' at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:246) at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182) at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1792) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1765

hi,  is it possible to  add workload user as host user ?  (just need it for rstudio server) thanks

I have been searching all over and can't find any solution that works.  I use Kudu and Parquet(hdfs) all searchable via Impala.  I want to use Grafana to visualize these data sources, but all I'm finding is performance metrics of the Cloudera components, not how to get the the massive amounts of data in the lake it self.   Any suggestions?

We are running rebalance on HDFS and we are getting the error below: "WARN balancer.Dispatcher: Failed to move blk_1314197946_240487461 with size=134217728 from 10.10.10.109:1019:DISK to 10.10.10.183:1019:DISK through 10.10.10.99:1019: Got error, status message opReplaceBlock BP-1707938289-10.10.10.96-1520510791093:blk_1314197946_240487461 received exception java.io.IOException: Got error, status message Not able to copy block 1314197946 to /10.10.10.183:51650 because threads quota is exceeded., copy block BP-1707938289-10.10.10.96-1520510791093:blk_1314197946_240487461 from /10.10.10.99:1019, block move is failed" we would like to know why we are getting these errors? would it be because the network interfaces are 1GB? if so, how can we resolve these errors? increasing the speed of network interfaces? PS: The hosts have 1GB network interfacesPS: We are using Ambari Server version 2.6.2.2.

Previous issue was Py4JJavaError: An error occurred while calling oxxxx.collectToPython. 

Hi,Nifi is not capable of spinning up a new node if one goes down, some of the threads in community says, autoscaling is possible in AWS . AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain a steady, predictable predefined number of a node if a specifically configured alert is triggered eg Bad health or lost heartbeat.   We have a setup in our 3 fixed node cluster. To established authorised nodes communication within the cluster we need to provide host names in authorizers.xml file. So I need to know how we can autoscale our NIFI clusters, where we need to update our authorizers.xml file in each and every instance. Any help would be highly appreciated. Thanks Abhishek  

When I am connecting via Cloudera ODBC Driver the process fails due to following error: [Cloudera][Support] (50170) Creation of new thread failed.  Does anyone know what the issue is and can help here? I did not find anything concerning this error.

I'm having a problem trying to create a nifi cluster with kubernetes.Error displayed: "The specified run.as user ec2-user does not exist. Exiting."The error appears when I use /opt/nifi/nifi-current/conf folder mappingIf I remove this mapping from the nifi folder, the cluster is created without problems.But I need this folder to keep the data flows if the container dies, come back with the datathe folder in the instance is in /home/ec2-user/user/1.16I already gave permission 777 on the folder to test and it still didn't workI already changed the user to upload with root user and nifi (pid 0 and 1000) and it didn't work

Hello Community, I'm running into a problem when I submit a job in Hue editor, from Hue home page I navigate to Query > Editor > Java. I point to the path of my jar file and then add the class name called Hortonwork.SparkTutorial.Main. Then after I hit run I get this error   "Caused by: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf".   I imagine it has to with not having the jar file SparkConf, I'm using IntelliJ for this project and I've added some jars to my project, I've taken some screen shots of what the jar files look like and have attached them to the post. Curious if any of these jars look familiar to anyone, I may not even be adding the right ones.    Thank you all for the taking the time to read my post and for your help.    Full list error is:    >>> Invoking Main class now >>>INFO: loading log4j config file log4j.properties.INFO: log4j config file log4j.properties loaded successfully.Setting [

Tried to install 7.4.4 on Ubuntu 2004 on a VMWe do have a license for private useOn starting server (Base) we have follwing log 2022-03-31 16:25:44,839 INFO main:com.cloudera.server.cmf.Main: ================================================================================2022-03-31 16:25:44,852 INFO main:com.cloudera.server.cmf.Main: Starting SCM Server. JVM Args: [-Dlog4j.configuration=file:/etc/cloudera-scm-server/log4j.properties, -Dfile.encoding=UTF-8, -Dcmf.root.logger=INFO,LOGFILE, -Dcmf.log.dir=/var/log/cloudera-scm-server, -Dcmf.log.file=cloudera-scm-server.log, -Dcmf.jetty.threshhold=WARN, -Dcmf.schema.dir=/opt/cloudera/cm/schema, -Djava.awt.headless=true, -Djava.net.preferIPv4Stack=true, -Dpython.home=/opt/cloudera/cm/python, -XX:+HeapDumpOnOutOfMemoryError, -Xmx4G, -XX:MaxPermSize=256m, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/tmp, -XX:+UseConcMarkSweepGC, -XX:+UseParNewGC, -XX:OnOutOfMemoryError={CMF_PATH}/bin/killparent-cm-server], Args: [], Version: 7.4.4 (

I have literally used the VM out of the box. Trying to train on materials and can't get anything working. Been trying to resolve this issue for a  week. Just finished a Codementor session and they couldn't get it either. if the password is not default (and I literally haven't touched anything) what is next?  Hive access denied.  Someone suggested changing something in ranger. I don't know if that's true or not. Asked a guy on Codementor. Just trying to get data into the table I created.Delete this VM and try again or?  I have logged in and ran hive as root, maria_dev, hive and hdfs. (btw, don't know what I am doing, not being trained properly, have zero tech support and am expected to learn this for work clients and am not supposed to have any problems that need solving. (shrugs)  

Hi, I am trying to perform a scripted start / stop of HDP services and its components. I need to get this done service by service, because there are few components like Hive Interactive / Spark Thrift server / Kafka broker, which does not get started / stopped in proper time. Most of the times it is observed that even if HSInteractive is still starting (visible in the background operations panel on Ambari), command moves to next service to start / stop and ultimately HSI fails. Hence, I also want to ensure that previous service is completely stopped / started, before attempting to stop / start next service in the list. To achieve, below is the script I have written (this is to stop, I have similar for start). However, when it reaches to the point for Hive or Spark or Kafka services, even though the the internal components like Hive Interactive or Spark Thrift server or Kafka broker are not started / stopped, it moves to the next service. Most of the times the start of Spark T

Hi, I am currently new to NIFI and have been working on a project that requires communicating to a secured RabbitMQ server to consume data. To make an SSL handshake I am using StandardSSLContextService controller service by adding the required Truststore information. And the Truststore actually contains the client certificate to communicate to RabbitMQ server from NIFI. But I am facing the below issue while running the ConsumeAMQP processor. 17:36:40 UTC ERROR nifi.svc.cluster.local:9443 ConsumeAMQP[id=76fe890c-4c44-3a3e-933b-4f4e5c4e4539] Failed to initialize AMQP client: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_requiredcauses: com.rabbitmq.client.ShutdownSignalException: connection errorcauses: java.io.IOExceptioncauses: java.lang.IllegalStateException: Failed to establish connection with AMQP Broker: com.rabbitmq.client.ConnectionFactory@72bc529717:36:40 UTCERROR76fe890c-4c44-3a3e-933b-4f4e5c4e4539 nifi.svc.cluster.local:94

Hi Everyone!   I've been receiving the following error on my Cloudera Manager.   "603.1 MiB of swap memory is being used by this role's process. Warning threshold: 200.0 MiB."     I have tried removing Solr collections by going to the following location and running the following query.   http://<HOSTNAME>:8993/solr/#/ranger_audits/documents<delete><query>evtTime:[* TO NOW-0DAYS]</query></delete>   But it's not having any effect on the Swap memory usage.      P.S. Swappiness is set to 1 as when I run  cat /proc/sys/vm/swappiness The output generated is 1.      Cluster Details :  Solr: 8.4.1 Hadoop 3.1.1.7.1.6.0-297 CDH-7.1.6

Hi everyone   I want to add 2 servers with KTS and 2 servers with KMS to my cluster. The question is : Do I need licenses for each of this additional servers?   Thanks

We faced an issue in our Impala environment:Refresh to any of the impala tables were getting failed while when we query any tables like select count * , it was working.We found that the catalogd lost its connection to statestored. The catalog logs show:Connection with statestore lost, entering recovery mode.Trying to re-register with statestore, attempt: 1TSocket::write_partial() send() <Host: ipaddress Port: 53164>Broken pipe While the statestored logs shows:Missed subscriber (catalog-server@servername:26000) heartbeat deadline by 8971ms, consider increasing --statestore_heartbeat_frequency_ms (currently 1000) Can someone advise what could be the issue.Both Statstore and catalog daemons are on the same node.

Hello, all,   I am trying to connect to our Hadoop 3.0 Unix server using the 64-bot Hortonworks ODBC driver.  SSL is enabled on this server.  I am able to connect using the Cloudera 64-bit ODBC driver, but, for reasons that I won't go into here, we would prefer to use the Hortonworks driver instead.  The error message I keep getting from the Hortonworks connection dialog from odbcadm32.exe is "SSL v3 alert: handshake error."  No other info is available.  I know relatively little regarding security, so this is not something I can solve on my own.  Was wondering if anyone else has experienced this, and, if so, what you did to rectify it.   Thank you,   Sincerely,  

HI,  after i have integrated CDH with Openldap, I found there is  a WARNING in container log like below, try to get password file localjecks and permission denied.   2022-03-31 00:53:13,420 WARN [main] org.apache.hadoop.security.LdapGroupsMapping: Exception while trying to get password for alias hadoop.security.group.mapping.ldap.ssl.keystore.password: java.io.IOException: Configuration problem with provider path. at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2118) at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2037) at org.apache.hadoop.security.LdapGroupsMapping.getPassword(LdapGroupsMapping.java:528) at org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:473) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.security.Groups.<init>(Gro

I am using Nifi with Local setup . I am using 'consumeKafka' ProcessorIn configuration I am specifyingKafka broker : Client_specified_broker ,Security Protocol : SASL_SSL ,SASL Mechanism : Plain ,Username : SSL username ,Password : SSL password ,SSL Context Service : StandardSSLContextService ,Topic : Topic From which data is consumed ,Group_id : testI dont see any keystore or truststore credentials with Confluent Kafka broker I am using . How to configure ConsumeKafka Processor and also please let me know if there are any setting to be done with Nifi so that it could read messages from SSL encrypted Kafka broker 

We would like to understand two behaviors that Kudu is showing. 1. When restarting the Kudu Tablet, it takes between 5 and 10min for this Kudu Tablet to communicate with the Kudu Master again, why is this happening? 2. There are 5 Kudu Tablets and two of those Kudu Tablets is using more than 90% of the memory that was set in "memory_limit_hard_bytes", why is this happening?

Hi all,  Maybe I didn't find some entries or articles about this "issue". Is there any limitations about the web upload for hue?I only found a really old ticket from 2016 (https://issues.cloudera.org/browse/HUE-2782).  Currently we are facing some issues during the upload via web frontend. I think it is due to the file size, but it would be nice to get a confirmation about this.  GreetingsHeiner

recently i have set up a new CDH cluster with all SSD disk. after this cluster goes live , i found the namenode log always output some WARNING log, as below:  2022-03-26 06:00:57,688 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 3 to reach 3 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD], creationFallbacks=[DISK], replicationFallbacks=[DISK]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy and org.apache.hadoop.net.NetworkTopology2022-03-26 06:00:57,688 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 3 to reach 3 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD], creationFallbacks=[DISK], replicationFallbacks=[DISK]}, newBlock=true) For more information, please en