Solved

I have a problem very similar to this one which has been marked as solved. I have followed the instructions which did not work. I am trying to upload a Nifi XML template that I have previously exported, but am getting a 403.I am running Nifi version 1.17.0 in a docker container.The docker image is the latest, which should be 1.14.0It is run using docker compose.I am using Postman to send the REST requests. The steps I have taken are: Generate an access token using: POST https://localhost:8443/nifi-api/access/token with an x-www.form-urlencoded body containing the username and password set in the .env file. This token is then used as the bearer tokenGet the root process-group id using: GET https://localhost:8443/nifi-api/flow/process-group/root then taking the id from processGroupFlow:idFailing to import an XML template using: POST https://localhost:8443/nifi-api/process-groups/{id}/templates/import where the body is raw XML, containing the previo

Hi All,Could you please help to resolve below concern as - We are executing below script and getting ERROR. I have enclosed the same for referene. hive -e "sethive.merge.tezfiles=true;set hive.merge.mapfiles=true;set hive.merge.mapredfiles=true;set tez.queue.name=BIAdv;set hive.execution.engine=tez;set hive.vectorized.execution.enabled =true;set hive.vectorized.execution.reduce.enabled ==true;set hive.exec.dynamic.partition=true;set hive.exec.dynamic.partition.mode=nonstrict;set hive.exec.max.dynamic.partitions.pernode=20000;set hive.exec.max.dynamic.partitions=100000;set hive.merge.size.per.task=134217724;set hive.merge.smallfiles.avgsize=134217724;set tez.grouping.split-count=1;INSERT overwrite TABLE <table-name1> partition (reported_date,last_usage_date)SELECT * from <table-name1> where reported_date='2022-11-16' and last_usage_date='2022-04-10';" ERROR - Log :----------------------------------------------------------------------------------------------ERR

Hi community,  I have found an issue on CDH 6 with hive log rotation. In this CDH version, hive log4j changes from rotation by size (configured by cloudera manager) to rotation by size and day (RFA to DRFA). ISSUEThis change results in:Cloudera can not remove old logs (and the FS can be filled), property "Maximum Log File Backups" does not work properly.The rotated log name changed from index to date, If we produced more logs than "Max Log Size", we only have one file per day, so we are overwriting this file with the new produced logs, losing the old logs on same day, and we will have X logs per day, but probably losing logs every day (huge problem when resolving issues). I'm completely sure that is a new issue, because hive log4j configuration changes on this version. SOLUTIONThe solution for me was changing the "appender.DRFA.rollingPolicy.FileNamePattern" property through Cloudera Safety Valve in Hive Server 2,  "Logging Advanced Configur

Whether to consider ambari supporting CentOS8?

I am trying to remove oozie from the cluster but when i am trying to do, it its showing dependency of hue. But when I removed the dependency of hue and trying to restart the cluster then I  am unable to restart the cluster with following error, Command failed to run because service Hue has an invalid configuration. Review and correct its configuration. First error: Missing required value: Oozie Service 

Hi, I have the below data: Record 6|Invalid values for Entry Date.Record 7|Processing failed due to error in record: 6Record 8|Processing failed due to error in record: 6Record 9|Processing failed due to error in record: 6Record 10|Processing failed due to error in record: 6Record 11|Duplicate of record Job id  : , data id : , first_name: , last_name: , Date: 2021, Entry Date: 2021.Record 12|Processing failed due to error in record: 11Record 13|Processing failed due to error in record: 11Record 14|Processing failed due to error in record: 11Record 15|Processing failed due to error in record: 11 I want to convert it like: [{"Record 6":"Invalid values for Entry Date."},{"Record 7":"Processing failed due to error in record: 6"},{"Record 8":"Processing failed due to error in record: 6"},{"Record 9":"Processing failed due to error in record: 6"},{"Record 10":"Processing failed due to error in record: 6"},{"Record 11":"Duplicate of record Job id  : , data id : ,

Hello,In my 3Masters cluster, one Kudu Master is starting and stopping all the time, this is the Log detail from Cloudera Manager: Time Log Level Source Log Message10:14:41.417 AM WARN cc:288Found duplicates in --master_addresses: the unique set of addresses is Master1:7051, Master2:7051, Master3:705110:15:11.823 AM WARN cc:254Call kudu.consensus.ConsensusService.RequestConsensusVote from 10.157.136.55:55402 (request call id 0) took 4542 ms (4.54 s). Client timeout 1775 ms (1.78 s)10:15:11.823 AM WARN cc:254Call kudu.consensus.ConsensusService.RequestConsensusVote from 10.157.136.37:59796 (request call id 0) took 30215 ms (30.2 s). Client timeout 9654 ms (9.65 s)10:15:11.823 AM WARN cc:260Trace:1112 10:15:07.281146 (+ 0us) service_pool.cc:169] Inserting onto call queue1112 10:15:07.281169 (+ 23us) service_pool.cc:228] Handling call1112 10:15:11.823245 (+4542076us) inbound_call.cc:171] Queueing success responseMetrics: {"spinlock_wait_cycles":384}10:15:11.823 AM WARN cc:260Trace:11

APAI am about to upgrade from cdh to cdp and I have some questions regarding new version of Hive.Until now I used to have hive as etl service because it is more stable but slower than impala. My tables that bi users see are in impala.My questions are:1) Is hive 3 fast enough to compete impala ?2) In case of bi use is it more appropriate to point hive or impala(I read that hive 3 uses cache and makes bi repeated requests faster)?3) In case of kafka flow, is it appropriate to create an acid table in hive 3 and store the fetched data live ?

Hello experts, I am using below api to create and update variable registry variables../nifi-api/process-groups/" + processGroupId + "/variable-registry/update-requests But I am unable to delete any already created variable from specific processor group variable registry using nifi api.Which api I am suppose to use to delete a variable registry variables? Thanks,Mahendra 

Where can I see a list of all the processors that will load NiFi's content into memory?

I'm looking at the Hbase Web UI and when I click on one of the region servers listed under base stats I see a list of region names.   Example:   img,0006943d-20150504220458043384375D00000002-00093,1527295748538.7b45a9f6f5584fc50b3152d41a5323a2.   My question is, what part of this is the actual region name.  My understanding is, img is the table name, 0006943d-20150504220458043384375D00000002-00093 is the beginning rowkey.  Is 1527295748538.7b45a9f6f5584fc50b3152d41a5323a2. the actual region name? 

I WAS TRYING TO INSTALL CDP THE OTHER DAY. I GOT A WIRED PROBLEM. CM AUTOMATICALLY  SKIPPED THE DATABASE SETTING AFTER I ASSIGNED ALL THE ROLES TO THE COMPONENTS. IS THERE ANY STEPS THAT I MISSED? CAN ANYONE HELP PLEASE?

Hive Fails to install kerberos authentication hive. But after uninstall kerberos , caused by an error occurs during the hive startup Unable to read HiveServer2 configs from Zookeeper

Hi Team, We have 3 DN's and Data Directories mount point size is around 1TB on each Node , total Data Directories size was 3TB. I deleted 500GB data from HDFS but space got release from only DN3 Data Directory not from other DN's. Please advise.. 

hello cloudera community, we are trying to use pyspark to interact with kudu We were able to successfully execute the command below: import kudu but when running the command below: from kudu.client import Partitioning returns the error ImportError: libkudu_client.so.0: cannot open shared object file: No such file or directory ps: we are using cdp, cdh 7.1.7 and cm 7.6.1

I was trying to control apache nifi processors to start/stop by calling apache nifi-api. I am using Postman to make the call, for example to start a GenerateFlowFile processor:https://localhost:8443/processors/{processor id} (PUT Request)But I got the response status 403(forbidden) as shown in the image below.                I was able to make a get request. I have also a client self-signed certificate generated using nifi-toolkit by following this https://kylo.readthedocs.io/en/v0.10.0/security/ConfigureNiFiWithSSL.html from nifi.properties file:ifi.security.autoreload.enabled=falsenifi.security.autoreload.interval=10 secsnifi.security.keystore=./conf/kylo/localhost/keystore.jksnifi.security.keystoreType=jksnifi.security.keystorePasswd=.............nifi.security.keyPasswd=.............nifi.security.truststore=./conf/kylo/localhost/truststore.jksnifi.security.truststoreType=jksnifi.secur

After restarting all the services trough ambari, only two services don't start and are facing the same error : ranger and infrasolr /usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__ raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")resource_management.core.exceptions.Fail: Configuration parameter 'kerberos-env' was not found in configurations dictionary!  Any idea ?Thanks

Hi Team, Can you please explain me how to interpret  'Cluster CPU' graph from CM.  

We have an MYSQL HA cluster,In case of failover, for the secondary master to take the place of the failed master, the hive metastore connection string configuration would need to be changed to point to the new master, and the services restarted. is it possible to have true HA on the Hive Metastore without changing the connection string manually ? 

How to resove given ERROR - ERROR - Unknown Exception:07<class 'py4j.protocol.Py4JJavaError'>An error occurred while calling o1140.insertInto.: org.apache.spark.SparkException: Job aborted.

Hello team, my workflow has been failing for several days in SSH actions randomly.the error found in the SSH logs is:   Causé par : GSSException : aucune information d'identification valide fournie (niveau de mécanisme : TGT a été révoqué (20)) sur sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:772) sur sun.security.jgss.GSSContextImpl.initSecContext (GSSContextImpl.java:248) sur sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) sur com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 45 plus Causé par : KrbException : TGT a été révoqué (20) sur sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) sur sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:226) sur sun .security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:237)à sun.security.krb5.internal.CredentialsUtil.serviceCredsSingle(CredentialsUtil.java:482) à sun.security.krb5.internal.CredentialsUtil.serviceCreds(C

i am using hive and tez.whenever i am performing insert query its returns following error: execution error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.teztask 

I understand NiFi stores its FlowFile attributes in memory and large attributes are not recommended because it takes up space in the JVM. What I am having trouble grasping is the consequences. Can someone explain different use cases what could happen and also, how long those attributes are stored in memory for?

Hello everyone,  I'm tinkering with a test cluster. Test cluster has the trial license and so far I have installed HDFS, Oozie, Yarn, Spark and Zookeeper (all services are on one node). I tried to install Yarn Queue Manager, but when starting the YARN Queue Manager Webapp service, it errors out with   CM Internal user credentials are not available as system environment variables. Please make sure that both CM_USER and CM_PASSWORD are present as environment variables for the application to work correctly.  Currently I have Cloudera Manager:Version: Cloudera Enterprise Trial 7.1.4Java VM Name: OpenJDK 64-Bit Server VMJava Version: 1.8.0_232 and when installing the cluster, I could only choose from Cloudera Runtime 7.1.8, no other choices. Are there any possible fixes for the issue?

Hello and thanks for reading  I have an issue with hadoop hbase while trying to find the issue I can see below memstore flush logs for all tables and its non stop MemstoreFlusherChore requesting flush of table 1. because K has an old edit so flush to free WALs after random delay 65889ms 022-10-29 12:22:13,176 INFO [regionserver/uhn-hw-fores201b:16020.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of table 1. because K has an old edit so flush to free WALs after random delay 65889ms2022-10-29 12:22:13,177 INFO [regionserver/uhn-hw-fores201b:16020.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of tabl 2 . because K has an old edit so flush to free WALs after random delay 181952ms2022-10-29 12:22:13,177 INFO [regionserver/uhn-hw-fores201b:16020.Chore.1] regionserver.HRegionServer: MemstoreFlusherChore requesting flush of table 3 . because K has an old edit so flush to free WALs after random delay 150462ms2