Solved

Hello Team, I have an issue with setting the Knox authentication with PAM. I have the default login in /etc/pam.d/   $ cat /etc/pam.d/login#%PAM-1.0auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.soauth substack system-authauth include postloginaccount required pam_nologin.soaccount include system-authpassword include system-auth# pam_selinux.so close should be the first session rulesession required pam_selinux.so closesession required pam_loginuid.sosession optional pam_console.so# pam_selinux.so open should only be followed by sessions to be executed in the user contextsession required pam_selinux.so opensession required pam_namespace.sosession optional pam_keyinit.so force revokesession include system-authsession include postlogin-session optional pam_ck_connector.so  Knox-sso looks as following (the default one) 

Good evening everyone, I have a problem on ranger, the users have access to the databases and everything works, but on the ranger web ui if I try to search for users belonging to the groups the following screen is shown. Even if checking in Unix the group contains several users:I saw that there is a KB for the same problem after upgrade but I already upgraded to 7.1.7sp1. Can anyone help me? Thank you

I have a parquet table named 'cats' holding 2 columns (day STRING, number INT). Statistics are generated both for the table and the 2 columns (e.g. max, min, distinct can be seen in DESCRIBE FORMATTED default.cats number).Still, when running an explain command for a select max, it looks like the statistics aren't being used. Why is this? How can Hive be configured to use the stats?All the following settings are true:set hive.cbo.enable;set hive.stats.autogather;set hive.compute.query.using.stats;set hive.stats.fetch.column.stats;set hive.stats.fetch.partition.stats;explain select max(number) from default.cats; 

Hi All, I would like to replace first element of an array value  i.e would like to replace with first net val with nifi attribute input{"iH": [{"DN": "711449","iLineList": [{"DNnumber": "711449","DNLineNumber": 2},{"DNNumber": "711449","DNLineNumber": 3}],"netval": "22.09"},{"DN": "711450","iLineList": [{"DNnumber": "711450","DNLineNumber": 2},{"DNNumber": "711450","DNLineNumber": 3}],"netval": "11.09"}]}defined SPEC like :[{"operation": "shift","spec": {"*": {"0": {"netVal": "iH[&1].netVal[0]","*": "iH[&1].&"},"*": "&1"}}},{"operation": "modify-overwrite-beta","spec": {"*": {// "invoiceHeader": {"0": {"netVal": "${netVal}"}// }}}}]Expected output:{"iH" : [ {"DN" : "711449","iLineList" : [ {"DNnumber" : "711449","DNLineNumber" : 2}, {"DNNumber" : "711449","DNLineNumber" : 3} ],"netval" : "${netval}"}, {"DN" : "711450","iLineList" : [ {"DNnumber" : "711450","DNLineNumber" : 2}, {"DNNumber" : "711450","DNLineNumber" : 3} ],"netval" : "11.09"} ]} 

Hi everybody,Testing CDP 7.1.9, looks like we hit a bug in Hive . The "UNBOUNDED FOLLOWING" actually behaves as "CURENT ROW". But only when the other side of the window is "UNBOUNDED PRECEEDING". Did someone hit this one ?If we change the PRECEEDING, it's okThanks in advance

>>[21/Feb/2024 16:06:55 +0800] 2712551 MainThread agent ERROR Heartbeating to master:7182 failedI am getting this error, so I cant continue installation.I am using RHEL 8 btw.

Hi Team, We have a usescase wherein we want to import the templates directly into nifi canvas using the rest apis provided by nifi we are hitting with the below error. If anyone faced this issue previously please assisst.we are able to upload, download but import is not working To get the tokenbearer=$(curl 'https://nifi-host:nifi-port/nifi-api/access/token' --data 'username=username&password=password' --compressed --insecure)To import the template curl -i -X -k POST -H "Content-Type: application/xml" -H "Authorization: Bearer $bearer" -F "template=@/root/tmp/template.xml" https://nifi-host:nifi-port/nifi-api/process-groups/processor-group-id/templates/import Nifi version - 1.8.0.3.3.1.0-10error - HTTP 400 Bad RequestThanks 

Our team is working on a client solution involving Cloudera, and we need to develop a Proof of Concept (POC). Is there a way to access Cloudera for free to set up a development environment? Specifically, we need HDFS and Impala integration; later, we will connect Impala to Tableau for visualization. Any guidance on accessing these services without cost would be greatly appreciated.

Hi all,I tried to connect to Impala in Windows Server 2012 through ODBC v2.7.0.1002 and get error FAILED![Cloudera][DriverSupport] (1170) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct.This is log error from odbcERROR 4196 Simba::ThriftExtension::TETSaslClientTransport::SaslLoggingCallback: ::LogUsername: GetUsernameEx(NameUserPrincipal) failed: 1332But I tried in my local laptop with same settings and ODBC version it can connect.I check from windows server can telnet to Impala IP and port.Is there any certificates that need to install in windows server? when I check in certificates console my laptop got much certificate than serverMy laptopWindows server that can't connect to ImpalaPlease suggest.  

Hello,I am a beginner starting to learn Kafka and Cloudera. I am running a simple java code for producer,and am using Cloudera SMM through Docker Desktop. On running my java code,I keep getting this error-"Disconnecting from node 1001 due to socket connection setup timeout". There are no issues in my code or pom. I am using Java Maven in VSCode. Please help me to fix this issue

It was observed that our application generates an error message that includes sensitive information about its environment, files, or associated data such as internal system path.is there a way to put a custome error page in nifiBanner Disclosureit was observed that the application is disclosing information about internal components details through response.

Hi. I want to change the LDAP user  as currently we are using a Domain Admin account. We want to use the service account for LDAP configuration. I  have found below properties in hdfs configuration to change.hadoop.security.group.mapping.ldap.bind.userhadoop.security.group.mapping.ldap.bind.passwordhadoop.security.group.mapping.ldap.basehadoop.security.group.mapping.ldap.search.attr.group.nameCould you please guide me how can I change these and is there any document for this?I am using cloudera 5.10 version.

How we can use  AttributesToJSON process to reformat the n1,n2,n3 parameters to json string as below ?{  "n1": 20,  "n2": {    "n3": "1400000474"  }}      we try to do that but the output of json string coming with \" , we want to pass the output of AttributesToJSON process to postHttp process{"n1" : "20""n2" : "{\"n3\":\"8001001922\"}",}

Hello experts,We are using Nifi for data ingestion via http - using 'HandleHttpRequest' processor.Data is getting ingested continuously but once in a while when incoming volume is little high we get 'Service Unavailable - 503' error from nifi end point.Below is the log we see in nifi app logs -  2024-03-04 11:20:25,558 WARN [StandardHttpContextMap-c2ffb1fd-2c09-3a90-0e5e-18384499605f] o.a.nifi.http.StandardHttpContextMap StandardHttpContextMap[id=c2ffb1fd-2c09-3a90-0e5e-18384499605f] Request from 10.13.2.204 timed out; responding with SERVICE_UNAVAILABLE Imp configs :[HandleHttpRequest processor]Container Queue Size : 2000   [StandardHttpContextMap controller service]Maximum Outstanding Requests : 8000Request Expiration : 3min nifi.web.jetty.threads=200 (in nifi.properties)Maximum Timer Driven Thread Count = 200 Total number of running processors on this env : 4200+This nifi is running on 8 core machine (ec2 instance) When we look at the CPU

I'm new to Apache Nifi Community and I got stuck at one point - please help me for the below problem :In 'Validate CSV' processor I'm giving below schema(schema_used) to validate the CSV input. But, processor is showing error(err_screenshot).Validate CSV -Schema_Used - StrNotNullOrEmpty(), ParseDate("MM/dd/yyyy"), StrNotNullOrEmpty(), StrNotNullOrEmpty(), Optional(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), StrNotNullOrEmpty(), Optional(), Optional(), StrNotNullOrEmpty()The above schema is showing below error -err_screenshot -Note: The Schema is working totally fine, if I remove 'Optional()'.Problem - Please tell me how can I use 'Validate CSV' processor in Apache Nifi and validate Mandatory/Optional Fields of CSV. Also, suggest/share any script(groovy/python) if that can validate the CSV.

I have a simple question, Is CDP (CDH 7.x) open source?   when I try to download the cm7 from this URL https://archive.cloudera.com/p/cm7/ Its ask for password.   I am following this URL https://docs.cloudera.com/cdpdc/7.0/installation/topics/cdpdc-configure-repository.html   Thanks

Hi Everyone,I am parsing data from JIRA portal, after all cleanup I am pushing the data in database. However my mapping running fine when I am loading the small amount of data, but its get failed when I am loading two months data which is around 800 rows (360 only loaded). however I am encountering an error java.lang.ClassCastException: null and in failure, I am not able to find the rows and reason which causing this issue.  2024-03-02 21:07:48,707 ERROR [Timer-Driven Process Thread-10] o.a.n.p.standard.PutDatabaseRecord PutDatabaseRecord[id=f10515e5-3b10-1c3c-802a-83b2ee68649a] Failed to put Records to database for StandardFlowFileRecord[uuid=0e2ec0fc-ebc5-4734-8e59-868a2aa0e6d5,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1709399268684-21833, container=default, section=329], offset=4, length=115143],offset=0,name=a016a8e7-82ae-4ed4-a51d-934d976fd173,size=115143]. Routing to failure.java.lang.ClassCastException: null2024-03-02 21:07:50,049 INFO [Cl

Hi.I'm trying to use Python extensions with Nifi.I've uncommented the line   nifi.python.command=python3   in nifi.properties but Nifi always crash when I try to run it. When I re-comment the line, Nifi starts and works fine. I've read this thread  https://community.cloudera.com/t5/Support-Questions/Nifi-2-0-0-M1-Installation-error-with-python/m-p/381430and tried to follow the instructions in the solutions.It still crashes. I'm posting here because the problem is on Windows instead of Linux. Thanks for your help! OSWindows 10 home editionNifi versionnifi-2.0.0-M2Java installed version21.0.2Python installed version3.11.8   nifi.propertiesIn   nifi.properties   I changed the line from   nifi.python.command=python3   to   nifi.python.command=python VenvI made sure the python venv package is installed by issuing this command   python -m venvwhich gave this messa

Hi everyone, I would like some clarification. Thank you.If a customer is already using active OCP (OpenShift) and wants to use CDP Private Cloud Data Services, will this impact other services performed on OCP? If so, which services of CDP Private Cloud Data Services will affect OCP?For example,If CDP Private Cloud Data Services are installed, the scheduler will default to using Yunikorn. So, if the settings are not changed, it won't use the default scheduler.For instance, if I apply a pod and want to use the default scheduler, one method is to adjust the Yunikorn ConfigMap settings to bypass the admission controller.

I'm just installing Apache Nifi on linux server with version 1.25 .this is my nifi.properties :#nifi.security=none#nifi.security.autoreload.enabled=false#nifi.security.autoreload.interval=10 secs#nifi.security.keystore=#nifi.security.keystoreType=PKCS12#nifi.security.keystorePasswd=#nifi.security.keyPasswd=#nifi.security.truststore=#nifi.security.truststoreType=PKCS12#nifi.security.truststorePasswd=#nifi.security.user.authorizer=nifi.security.allow.anonymous.authentication=falsenifi.security.user.login.identity.provider=file-login-provider#nifi.security.user.jws.key.rotation.period=PT1H#nifi.security.ocsp.responder.url=#nifi.security.ocsp.responder.certificate=this is my users.xml :<?xml version="1.0" encoding="UTF-8" standalone="yes"?><tenants> <users> <!-- Add more users as needed --> <user identifier="ronald"> <identity>ronald</identity> <!-- Replace CREDENTIAL_VALUE with the hashed password -->

hello cloudera community, do I need to insert a new datanode in the hdfs and with that I need to know how much size will be used from this new datanode after performing a rebalance on the hdfs? for example, I have the following scenario: datanode 1 has 36tb total to use, 30tb is being used, containing about 2.5 million blocksdatanode 2 has 36tb total to use, 30tb is being used, containing about 2.5 million blocksdatanode 3 has 36tb total to use, 30tb is being used, containing about 2.5 million blocksdatanode 4 has 36tb total to use, 30tb is being used, containing about 2.5 million blocksdatanode 5 has 36tb total to use, 30tb is being used, containing about 2.5 million blocksdatanode 6 has 36tb total to use, 30tb is being used, containing about 2.5 million blocks i will insert the "datanode 7" which has 36tb total, after performing the rebalance in hdfs, how much will this new datanode 7 receive data?

Hello,I am trying to add key: value in a json after SplitJSON. I was using ReplaceText processor for achieving so. But problem my regex expr is coming twice my json object which causing the invalid json. What would be the appropriate way to do so ?Current Structure:Replace Text Processor looks like :Any suggestions will be of great help!!

Hi everyone, We faced some issue when using Flink version 1.16.2.CSA.1.11.0.x, this is the error message2024-02-19 16:52:42java.lang.IllegalStateException: Failed to commit KafkaCommittable{producerId=1077051, epoch=0, transactionalId=payload_migs-TEST_mig_2-0-3} at org.apache.flink.streaming.runtime.operators.sink.committables.CommitRequestImpl.signalFailedWithUnknownReason(CommitRequestImpl.java:77) at org.apache.flink.connector.kafka.sink.KafkaCommitter.commit(KafkaCommitter.java:119) at org.apache.flink.streaming.runtime.operators.sink.committables.CheckpointCommittableManagerImpl.commit(CheckpointCommittableManagerImpl.java:126) at org.apache.flink.streaming.runtime.operators.sink.CommitterOperator.commitAndEmit(CommitterOperator.java:176) at org.apache.flink.streaming.runtime.operators.sink.CommitterOperator.commitAndEmitCheckpoints(CommitterOperator.java:160) at org.apache.flink.streaming.runtime.operators.sink.CommitterOperator.initializeState(CommitterOperator.java:

Hey there, we're running a CDP PC Base Cluster in our datacenter. In the last months we saw serveral disks failing due to theire age. The Problem we had, is that, when a disk fails, HDFS DN and YARN NM creating directories in the root-vg of the node. During normal operations the DN has it directories on the grids (/grid/[0-16]/*).Is there a parameter to prevent them of writing into the root-vg, when a grid points to the root-vg and not it's physical device under /dev/sd*?Regards, Timo

Hello Team, I have one column and did some calculation on it and wanted to make another copy of same column to create flag to check whether records are empty or not and retain the same column also.I tired to using pulling another processor and  wrote calculation New_field , Old_column which did not work.However, this approach did not yield the desired outcome.