Q&A

Hello,  I have just upgraded the JDK version of my Cloudera 5.10 cluster from 1.7 to 1.8 and everything starts ok but when I try to run a job from Talend the following error appears in HDFS Name Node log: 2021-03-22 13:11:32,711 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8020: readAndProcess from client 69.117.42.187 threw exception [javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: DES CBC mode with MD5 encryption type not in permitted_enctypes list)]] If I go back to JDK 1.7 it runs perfectly.I have tried with 1.8.201 and also with 1.8.91 (since this was one of the tested ones).I think I have JCE and java.security correctly configured. We do not want to allow DES CBC in the KDC since it a weak algorith. Has it happened to anyone? Thanks in advance, Ángel

I am trying to transport a python environment with a PySpark according to: https://community.cloudera.com/t5/Community-Articles/Running-PySpark-with-Conda-Env/ta-p/247551https://henning.kropponline.de/2014/07/18/virtualenv-hadoop-streaming/https://community.cloudera.com/t5/Community-Articles/Using-VirtualEnv-with-PySpark/ta-p/245932 Last try I issued the following commands:   #Creating my relocatable environment:conda create -y -n py35 python=3.5 numpy pandas scikit-learnconda activate py35cd /opt/cloudera/parcels/Anaconda/envs/zip -r py35.zip py35#Runing the job:PYSPARK_DRIVER_PYTHON=/opt/cloudera/parcels/Anaconda/envs/py35/bin/python \PYSPARK_PYTHON=/opt/cloudera/parcels/Anaconda/envs/py35/bin/python \pyspark2 \--conf spark.yarn.appMasterEnv.PYSPARK_PYTHON=/opt/cloudera/parcels/Anaconda/envs/py35/bin/python \--master yarn \--deploy-mode client \--archives /opt/cloudera/parcels/Anaconda/envs/py35.zip#py35 test.py   test.py:  

Hello, in our local filesystem we see thousands of subdirectories like "????????-????-????-????-????????????_resources" older than 5 days (and this is for guarantee nothing from any still running process).Additionally there are thousands of files and hundreds of subdirectories in "/tmp/hive" older than 5 days.Where do those relicts come from?How to get rid of them in an automated way? $ find -O1 /tmp -type d -name "????????-????-????-????-????????????_resources" -mtime +5 | wc -l26263$ find /tmp/hive -type d -mtime +5 | wc -l538$ find /tmp/hive -type f -mtime +5 | wc -l5784 Best RegardsCarsten

Hi all, I am getting error while executing the phoenix query. Error: Can't get the location for replica 0 (state=08000,code=101)org.apache.phoenix.exception.PhoenixIOException: Can't get the location for replica 0at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:117)at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1126)at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1418)at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2190)at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:872)at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:194)at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:343)at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:331)at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)at org

I am seeing frequent Cloudera Manager Service Monitor outages: SERVICE_MONITOR_PAUSE_DURATION has become bad: Average time spent paused was 39.5 second(s) (65.76%) per minute over the previous 5 minute(s). despite increasing the heap size to 7g and the 'off-heap' size to 24g. The machine often sees a high load (a NodeManager is also on the same machine), like 90 on a 24 core machine, so I suspect it might be starved of cpu when doing aggregation. The process regularly has +700 files open.  I have motivation to fix this issue since it causes data loss in the time series because SM pulls data  and misses data points for +15 minutes at times.  The WARN: AggregatingTimeSeriesStore: run duration exceeded desired periodis correlated with the above.  Is there a documented procedure to move Service Monitor to another machine while keeping existing data?  Perhaps like:0. stop SM to quiesce changes to /var/lib/cloudera-service-moni

I am trying to build a new maven spark project but I cannot seem to pull the spark maven dependencies. I keep getting the following error:  Could not resolve dependencies for project za.co.sanlam.custhub.spark:party-matching-rules:jar:1.0.0-SNAPSHOT: Failed to collect dependencies at org.apache.spark:spark-core_2.11:jar:2.4.0.7.1.4.9-1 -> org.apache.avro:avro:jar:1.8.2.7.1.4.9-1: Failed to read artifact descriptor for org.apache.avro:avro:jar:1.8.2.7.1.4.9-1: Could not transfer artifact org.apache.avro:avro:pom:1.8.2.7.1.4.9-1 from/to cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/): Transfer failed for https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/avro/avro/1.8.2.7.1.4.9-1/avro-1.8.2.7.1.4.9-1.pom ProxyInfo{host='10.0.0.132', userName='null', port=8080, type='http', nonProxyHosts='null'}: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested

I have a NiFi flow [on a 2 processor machine using 8/5 thread counts] where the InvokeHTTP block is no longer processing files from the queue [shown below]:As you can see there are 8 processes running but these seem to be stuck for hours and the only way out of it is manually stopping and terminating the block. This results in the following log message:2020-05-18 08:31:06,228 ERROR [Timer-Driven Process Thread-12 <Terminated Task>] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=f2f13d29-016b-1000-293d-c0cb04835881] Routing to Failure due to exception: java.net.SocketTimeoutException: timeout: java.net.SocketTimeoutException: timeoutjava.net.SocketTimeoutException: timeout at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:593) at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:601) at okhttp3.internal.http2.Http2Stream.takeResponseHeaders(Http2Stream.java:146) at okhttp3.internal.http2.Http2Code

Hi,We need help with respect to order of records in Nifi. We have couple of custom processors which work in batch.There is processor which reads the data from source (Processor A) & another processor which writes the data to SFTP (processor B).Processor A outputs the data in form of multiple flow files (100 records from source are written in one flow file). Assume if there are 600 records in source, we get 6 FFs out of processor A.Process A & Processor B are directly connected to each other & entire execution happens in Single node only. However, we are finding that one some occasions the Process B is processing flow files out of order. For example, third flow gets processed first & then the second flow file. Understood the flow fill prioritizer needs to be chosen explicitly in the connection between Process A & Process B. However, confused up on FirstInFirstOut prioritizer & OlderFlowFilePrioritizer. Kindly provide pointers to guarantee the

We run multiple NiFi clusters for dev, test, integration, and failover.  After recently upgrading from NiFi 1.0.8 to 1.11.4, we've found that NiFi now registers *all* processor state changes as Registry changes.  Since these processors must be in different states between the various clusters, it's become very problematic. Is there any way to configure NiFi and/or the Registry to ignore processor states like it used to?  I've looked through the properties files and admin docs for both, and haven't found anything configuration that supports that.

Hello, I have a processor scheduling setup as below - Run Schedule - 30 secRun Duration  - 0 msConcurrent Tasks - 1 In this case, assuming that each task takes more than 30 secs to complete once it starts, can 2 (or more than 2) processor tasks run at the same time?  0s - Task1 starts 30s - Task2 starts (Task 1 is still running) Is there any way to prevent overlapping task runs i,e Task 2 should start at 30s only if Task1 that started at 0s is complete? Thanks!

@ApacheNifi I have this flow that take files from Azure Blob Storage (with ListAzureBlobStorage processor).Now.. is there some trigger or notification mechanism that indicates that new files to be processed have been uploaded to the Azure blob??Thank you so much 

Hi everyone, I noticed that after CDP 7.1.2.1, you aren't exposing the yum repositories anymore, at least in the URLs that I know. For example, in https://archive.cloudera.com/p/cdh7/7.1.2.1/redhat7/yum/ we have access to packages like hadoop, but in https://archive.cloudera.com/p/cdh7/7.1.6.0/ we don't have access to yum path anymore and it becomes impossible to install the packages that you distribute with CDP 7.1.6 using yum. Do you provide access to the yum repository in any other location or are we forced to install cloudera manager if we want to have access to some packages that could be installed previously with yum but now only using parcels? Thanks in advance.

I am doing the coursera course in big data. I followed the instruction and did the installation. I realised then i had to enable the Virtual Technology in BIOS (which was enabled anyway). However, i still get the same error: 'Failed to Open a session for the virtual machine cloudera-quickstart-vm-5.4.2-0-virtualbox' The virtual machine 'cloudera-quickstart-vm-5.4.2-0-virtualbox' has terminated unexpectedly during startup with exit code 1 (0x1). More details may be available in 'C:\Users\badao\VirtualBox VMs\cloudera-quickstart-vm-5.4.2-0-virtualbox\Logs\VBoxHardening.log'.Result Code: E_FAIL (0x80004005)Component: MachineWrapInterface: IMachine {b2547866-a0a1-4391-8b86-6952d82efaa0} Please help me with a comment

I am integrating Apache NiFi 1.9.2 (secure cluster) with HDP 3.1.4. HDP contains Zookeeper 3.4.6 with SASL auth (Kerberos). NiFi nodes successfully connect to this Zookeeper, sync flow and log heartbeats.Meanwhile, NiFi processors using Zookeeper are not able to connect. GenerateTableFetch throws: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 uri from ZooKeeper)  ListHDFS fails attempting to write its state to ZooKeeper: o.apache.nifi.processors.hadoop.ListHDFS ListHDFS Failed to retrieve timestamp of last listing from the State Manager. Will not perform listing until this is accomplished. But GetHDFS (which doesn't communicate with zk) works OK.Enable HiveConnector works without errors.Thank you in before for any ideas and support.

Hi all.I'm facing a CM configuration error I can't understand.I've kerberized a Cloudera 5.16 cluster which authenticates against an AD/DC controller.I've set aes256-cts and aes256-cts-hmac-sha1-96 encryption types.CM reports a configuration issue:I've a separate cluster, configured with the same values and authenticating to the same AD/DC controller, that have no errors. I'm struggling on that error but can't solve. Any idea?  

Hello all, I tried to set up Ambari  for a research project, however I get  "ERROR 403:  FORBIDDEN" (Access denied ) when trying to download the repositories from  Hortonworks & I don't know what I can do know or how to install Ambari and all its HDP services manually. Is anyone else here encountering this problem or does anyone have a solution for it?I tried it with multiple versions already. Thanks in advance!

Hi all. First of all, let me wish you a happy and prosperous new year.I am trying to install HDP via Ambari. Based on the latest docs (https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/access_ambari_paywall.html). According to it, starting from HDP 3.1.5 / Ambari 2.7.5, accessing Ambari repositories requires a special set of credentials. Does anybody has an idea of how to get such credentials? Unfortunately, instructions on the specific documentation site are not intuitive.Thanks in advance!

Hello Team, This is Vaibhav Waghmode from Pune, India. it is regarding downloading the cloudera manager CHD5. also getting below mentioned page while downloading.could you please suggest me on this this? 

Hello all,I'm trying to kerberize the Ambari 2.7.3 cluster. However, during the setup, I get the following error:Caused by: org.apache.ambari.server.serveraction.kerberos.KerberosOperationException: Unexpected error condition executing the kadmin command. STDERR: kadmin: Matching credential not found (filename: /tmp/ambari_krb_142308985016794830cc) while initializing kadmin interface at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.invokeKAdmin(MITKerberosOperationHandler.java:323) at org.apache.ambari.server.serveraction.kerberos.MITKerberosOperationHandler.principalExists(MITKerberosOperationHandler.java:123) at org.apache.ambari.server.serveraction.kerberos.KerberosOperationHandler.testAdministratorCredentials(KerberosOperationHandler.java:314) at org.apache.ambari.server.controller.KerberosHelperImpl.validateKDCCredentials(KerberosHelperImpl.java:2133)All of the authentication settings are okay, because I am able to kinit and

Hi ,I have created 2 flows -1] in this flow i have used GenerateFlowFile->UpdateAttribute->InvokeHttpGenerateFlowFile config- Dataformat = textCustomText = Science is magic$$@@ i love Data ## UpdateAttribute config -mime.type = application/x-www-form-urlencodedInvokeHttp config - HTTP method = POSTand this works super fine . But when i use this with gettwitter it fails as shown in below -  1] GetTwitter is working fine - 2] EvaluateJsonPath - 3] AttributeTOJson- 4] UpdateAttribute- 5] ReplaceText -   6] InvokeHTTP -     this is the error i am getting -  but if i set empty text at this from invokehttp processor - then this error goes away but the api responds with missing api_key and text -  Somebody pls help, this is eating my head up . thanks in advance   

Hi everyone! I have a file that contains a Json array of json objects. My first step is to split this flowfile into many flowfiles and ach one of them has a json object using SplitJson Processor. Then I'm trying to set the attributes of json to the flowfile attributes of each flowfile so I can use it after to build a query using Evaluatejsonpath processor. Everything worked fine at the beginning but then I noticed some flowfiles go to failure queue of Evaluatejsonpath.   I checked logs and the error is Invalid Json but when I went to see json files in failure queue the json has no problem and valid. Then I tried to change the processor that I'm using so I used ExecuteScript Processor. I used a script that does the some job I wanted, but still some problem some number of files go the failure queue. Help!         Evaluatejsonpath Processor:         SplitJson Processor:    

Hello,Need some assistance / guidance on how we can reduce Non-HDFS Space. We see Non-HDFS Space of around 270 used, as we are facing space crunch, we would explore possibilities for getting non-hdfs space reduced.I have cleared all Yarn logs for the applications which were killed/ failed etc (our /data mountpoint houses dfs, yarn, kudu, impala), yet this does not solve our issue. Any assistance / guidance is much appreciated. Thanks Amn

Hi all,I just discover NIFI few weeks ago and I tried to replicate a simple curl command with the processor InvokeHTTP :   I tried this curl command in the VM where my NIFI server is installed and it works well, I received all the needed information. So what I did in the invokeHTTP processor is the folowing: in properties1) HTTP Method =GET2) Remote URL = "URL used in the curl command"then I added a new property Authorisation = my_login:token But the processor stay invalid. I know that there is plenty of other properties and settings but I'am a little bit lost, if someone can help me it will be greatThanks  

Hi community, We run Spark 2.3.2 on Hadoop 3.1.1. We use external ORC tables stored on HDFS. We are encountering an issue on a job run under CRON when issuing the command `sql("msck repair table db.some_table")`. The table is partitioned and the issue is the following: 21/03/22 22:44:13 WARN HiveConf: HiveConf of name hive.heapsize does not exist21/03/22 22:44:13 WARN HiveConf: HiveConf of name hive.stats.fetch.partition.stats does not exist21/03/22 22:44:13 WARN HiveConf: HiveConf of name hive.plan.serialization.format does not existHive Session ID = 2625af79-e021-4b57-9435-e0fea4f0080321/03/22 22:44:13 INFO SessionState: Hive Session ID = 2625af79-e021-4b57-9435-e0fea4f0080321/03/22 22:44:13 ERROR ApplicationMaster: User class threw exception: org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: java.io.IOException: Unable to create directory /tmp/hive/2625af79-e021-4b57-9435-e0fea4f00803_resources;org.apache.spark.sql.AnalysisException: java.lang.R

Hello, How do we balance data which are stored on individual disk on a particular datanode, we have 5 disks on a single node and one of the disk is 90% full, running balancer is not fixing the issue.Would like to get some suggestions/ comments to fix this issue.I was going through this article and it says it is not possible to balance disks within a single node, so what other options can we use to fix this until we upgrade to CDH 6.3(https://community.cloudera.com/t5/Community-Articles/HDFS-Balancer-Balancing-Data-Between-Disks-on-a-DataNode/ta-p/244650) CM & CDH - 5.16.3 Appreciate all inputs. ThanksWert