Q&A

Hey Team,I am want to connect  hive with SSL and  Kerberos.Getting error with trying to  connect via beeline and got the following error:root@ip-172-31-13-77:~# beeline -u "jdbc:hive2://ec2-52-66-58-15.ap-south-1.compute.amazonaws.com:10000/default;principal=hive/ec2-52-66-58-15.ap-south-1.compute.amazonaws.com@AP-SOUTH-1.COMPUTE.AMAZONAWS.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=******************"SLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/log4j-slf4j-impl-2.13.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.sl

Hello,I am unable to access Hive using Beeline connection string in the edge node. The JDBC String that I am using is as follows:beeline -u "jdbc:hive2://Example.com:10000/default;principal=hive/Example.com@EXAMPLE.COM" Error:Unknown HS2 problem when communicating with Thrift server.Error:jdbc:hive2://Example.com:10000/default;principal=hive/Example.com@EXAMPLE.COM: Invalid status 21 (state=08S01,code=0)Help is required to fix this issue. Thank you. 

Hi, I am trying to group webCategoryId and item with yes/no flag with some conditions inside array.1 - if  webCategoryId variable is present and not empty value then only use webCategoryId, ignore item value if present inside array.2 - if  webCategoryId variable is not present or present with empty value then use item if item value is present inside array.Could you please advice. Input   { "correlationId": "3242a7b4-7929-43b3-9fb6-0d7e173373af", "sourceSystemId": "PCS", "messageTimestamp": "2024-05-03T00:00:00", "id": 1234, "promotionId": 2303, "offerId": 2303, "description": "Members Get Tees $15", "customerDescription": "Members Get Tees $15", "levelCode": 0, "typeCode": 3, "templateId": 4, "couponCode": null, "isCouponCodeRequired": 0, "startDatetime": "2024-06-20T00:00:00", "endDatetime": "2024-06-24T23:59:00", "cancelationDatetime": null, "comments": null, "distributionRuleCode": null, "exclusiveDi

I'm trying to connect to Impala on a new mac (M2 Chip). I followed all the steps here for unixODBC. I confirmed that my unixODBC install is working and that the Cloudera driver installed properly.When I run `odbcinst -j` I get:unixODBC 2.3.12DRIVERS............: /opt/homebrew/etc/odbcinst.iniSYSTEM DATA SOURCES: /opt/homebrew/etc/odbc.iniFILE DATA SOURCES..: /opt/homebrew/etc/ODBCDataSourcesUSER DATA SOURCES..: /Users/willhipson/.odbc.iniSQLULEN Size.......: 8SQLLEN Size........: 8SQLSETPOSIROW Size.: 8Here's my odbcinst.ini [ODBC Drivers]Cloudera ODBC Driver for Impala=Installed[Impala]Description=Cloudera ODBC Driver for ImpalaDriver=/opt/cloudera/impalaodbc/lib/universal/libclouderaimpalaodbc.dylib  And here's the first few lines of my odbc.ini [ODBC][ODBC Data Sources]Impala=Cloudera ODBC Driver for Impala[Impala]Description=Cloudera Impala ODBC Driver DSNDriver=Impala I have confirmed that the driver exists in `/opt/cloudera/impalaodbc/lib/univer

Hi,I am using NiFi 1.26.0 and having a flow with combination of ListFile and FetchFile.Basically i have three files in my local directory (test_1.txt, test_2.txt,test_3.txt). I have to fetch only one file (ex:test_1.txt) for further process.Based on regex listFile list's all three files.But facing issue in fetch file. Instead of fetching only one file, it tried to fetch all three files listed by listFile processor  and giving output as three files instead of one file, but content of all three files are same as content of file mentioned in fetchFile configuration (test_1.txt)In data provenance could see it is trying to fetch the files which is not mentioned in fetchFile configuration and after fetching the files it modifies the content.How can i fix this. Please help.

Just for context, our Hadoop cluster is kerberized and is backed by Active Directory that also hosts the other accounts running our .NET apps on Windows.One of our .NET apps opens a connection to Impala using the .NET OdbcConnection class. The OdbcConnection instance is supplied with an ODBC connection string that specifies the use of the Impala ODBC driver (2.6.7) and points to our Impala daemon nodes. We want to run our Impala queries using a different account from the one that is running the .NET app, so we do WindowsIdentity.RunImpersonated (WindowsIdentity.RunImpersonated Method (System.Security.Principal) | Microsoft Learn) to run the ODBC connection code as the user that we want. Here's a simple example of how the code looks:  WindowsIdentity.RunImpersonated(safeAccessTokenHandle, () => { var connectionString = "Driver=Cloudera ODBC Driver for Impala;...;AuthMech=Kerberos;SSL=1;UseSystemTrustStore=1;"; OdbcCommand command = new OdbcCommand("SELECT 1"); using (OdbcConn

I recently downloaded the latest version of NiFi (nifi-2.0.0-M4) and installed it by placing the folder in /opt/nifi-2.0.0-M4. The only modifications I made were in the nifi-env.sh and nifi.properties files as follows:nifi-env.shexport JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64nifi.properties# Site to Site propertiesnifi.remote.input.host=10.37.10.199nifi.remote.input.secure=falsenifi.remote.input.socket.port=10000nifi.remote.input.http.enabled=truenifi.remote.input.http.transaction.ttl=60 secnifi.remote.contents.cache.expiration=60 secs# web properties ############################################### For security, NiFi will present the UI on 127.0.0.1 and only be accessible through this loopback interface.# Be aware that changing these properties may affect how your instance can be accessed without any restriction.# We recommend configuring HTTPS instead. The administrators guide provides instructions on how to do this.nifi.web.http.host= 0.0.0.0 # Here goes the IP of my virtual mach

Hello community,I want to get the from hdfs to confluent cloud kafka is there any suggestion how to do that.

"result" : "SUCCESS","correlationID" : "X8058296","generationTimestamp" : "1354","meHostName" : "rc01;rik2jc","nccVersion" : "SPS__R1", This is my input in my nifi and i want to do changes in meHostName means i want mehostName to have 1st value like in this case rc01 and after that creating 1 more field with name devicename having second value means rik2jc I acheived the 1st part means i configured in UpdateRecord processor  for meHostName and it is printing 1st value means it is running fine ,now please help me in acheiving devicename to have 2nd value of meHostName which is rik2jc . i have tried split(/meHostName, ';')[1] but its not working

My solr service is up and runing, but still all the ranger audits for atlas service are being loaded into /var/log/atlas/audit/spool/archive which ideally is done when the solr-sink is down for that service. It is causing a lot of space consumption rapidly please help!

I have created a three node NIFI cluster and the embedded zookeeper is set up and all the three nodes are communicating with each other through heartbeats but when I try to access the NIFI UI I am getting the below error:Unauthorized error="invalid_token", error_description="An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found", error_uri="https://tools.ietf.org/html/rfc6750#section-3.1"How do I fix this issue?

Hi,regarding the disks of an HDFS datanode with for example 10 disks, what are good values ​​for the read/write throughput for each JBOD disk? Are there any benchmarks for a production environment? (e.g. 200 MB/sec).Thank you.

Hello Community,I would like to rise a big inconsistency about requirements.Starting by the Cloudera Support Matrix that says: CDP Private Cloud Base7.1.8 > will need Cloudera Manager 7.7.1 and OS: RHEL8.4 will work.Now, regarding to NTP, based on teh OS:"In RHEL 8, the NTP protocol is implemented only by the chronyd daemon, provided by the chrony package. The ntp daemon is no longer available. If you used ntp on your RHEL 7 system, you might need to migrate to chrony"in the other hand:https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/kudu-management/topics/kudu-server-management-limitations.html"Kudu releases have only been tested with NTP. Other time synchronization providers such as Chrony may not work."Could someone from Kudu team give some consistency here? RHEL8 not having ntpd anymore but mandatory for KUDU...Please help.

Hi , In My nifi flow i have two dataFlows i want to same same data to two different HTTP endPoints , for both data flow data is being consumed from same kaflka topic (TOPIC A) both flows has different kafka groupID and pushing data to different HPPT end points (endPoint 1 and endPoint 2). If there is any failure data is being produced into retryTopic R which is common for both data flow . There is separate retryData flow for both configured with seperate HttpEndPoints , but both consuming from same retryTopic .What issue coming is , due to same retry topic , dataFlow A can consume data of dataFlowB . How should i avoid this situation ? retryTopic used should be same for both. Here is a sample for the dataFlow .

hey all I was just wondering what the best way is to merge 2 JSON flowfiles together. I have 2 processes which generate pretty much identical data but with a different source (so can’t do it as one process) and I’d like to merge them together into one JSON file if possible. Example data is something like... Process 1:  [ { "code": "ABC123", "product": "My Product", "prices": [ { "valid_from": "20180913 09:00:08", "name": "pricelist1", "abc": "14.74" }, { "valid_from": "20180913 09:00:08", "name": "pricelist2", "abc": "10.70" } ] }]  Process 2:  [ { "code": "ABC123", "product": "My Product", "prices": [ { "valid_from": "20180913 09:00:08", "name": "pricelist1", "xyz": "53.74" }, { "valid_from": "20180913 09:00:08", "name": "pricelist2", "xyz": "21.70" } ] }]  Desired Outcome: &n

Hi all,I'm trying to get our CDSW 1.10.5 Instance to recognize the GPUs, according to this documentation: https://docs.cloudera.com/cdsw/1.10.5/gpu/topics/cdsw-gpu.htmlBasically: nvidia-smi shows the GPUs are present on the host but after activating GPU-Support in Cloudera Manager the GPUs do not show up in the CDSW interface. We tried to follow this advice, since we also run an AirGapped Setup: https://community.cloudera.com/t5/Support-Questions/CDSW-1-6-does-not-recognize-NVIDA-GPUs/td-p/280207but unfortunatelly retagging the image did not solve the issue. How can we obtain additional info to diagnose the issue?

Hi allI was looking for some help regarding moving a NiFi Registry set up from one server to another.At the moment we set up a kind of "QA" system which was running well and so we have now created the production server if you like which we're going to use for all of our NiFi connections. The current registry is backed by a Git Repo using BitBucket which all works as expected.We have set up the new server, installed registry and connected the same repo and all seems to be fine in terms of the NiFi servers can connect and I can see registry on the new server.What I want to do now though, is move everything we had in the "QA" registry server, into the production server if this is possible (so that means the buckets, versions and flows etc) but JUST for the registry (we will then pull these fresh into the NiFi server(s)).I think I read somewhere that the information in registry is kinda "linked" to a registry server via a UUID or something but I'm not sure how that works (if not, my assump

Hi Everyone,I'm configuring Kerberos for our environment and aiming to ensure high availability. I've set up a master KDC and a slave KDC on EC2 instances but getting the below error message while propagating.here is my setup on ec2 instances for master kdc and slave kdc to propagate.Install Kerberos server on both the KDC's using:sudo apt install krb5-{admin-server,kdc}edited the below files. 1.sudo cat /etc/krb5.conf[libdefaults]default_realm = EXAMPLE.COM# The following krb5.conf variables are only for MIT Kerberos.kdc_timesync = 1ccache_type = 4forwardable = trueproxiable = truerdns = false[realms]EXAMPLE.COM = {kdc = kdc01.example.comkdc = kdc02.example.comadmin_server = kdc01.example.com}[domain_realm].example.com = EXAMPLE.COMexample.com = EXAMPLE.COM[logging]kdc = FILE:/var/log/kerberos/krb5kdc.logadmin_server = FILE:/var/log/kerberos/kadmin.logdefault = FILE:/var/log/kerberos/krb5lib.log2.sudo cat /etc/krb5kdc/kdc.conf[kdcdefaults]kdc_ports = 750,88[realms]EXAMPLE.COM = {

We enabled Hbase read replica - However we bump into this situtation   we observed that read replica cluster is still doing compaction and Primary cluster is reporting a FileNotFound Exception for multiple regions.What could be the reason for this ? Is there any specific configuration parameter that is enabled ? 

Hi all, I am practicing spark. When using pyspark to query table in Hive, I can retrieve the data from an external table but query a internal table. Here is the error message:>>> spark.read.table("exams").count()23/03/30 22:28:50 WARN conf.HiveConf: HiveConf of name hive.masking.algo does not existHive Session ID = eb0a9583-da34-4c85-9a1b-db790d126fb1Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/lib/spark/python/pyspark/sql/readwriter.py", line 301, in table return self._df(self._jreader.table(tableName)) File "/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", File "/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/lib/spark/python/pyspark/sql/utils.py", line 69, in deco raise AnalysisException(s.split(': ', 1)[1], stackTrace)pyspark.sql.utils.AnalysisException: u'\nSpark has no access to

Hey!I recently stumbled upon a medium-risk security vulnerability. I emailed Cloudera's security contact, but its been weeks with no reply or patch. I attempted to open a support ticket but the support page says my account doesn't have the necessary permissions. I was wondering if anybody can connect me to some employee who can help fix this? Thanks!

Hello team,My main work is dependent on HBase Data.I have Source (RHEL 7 CDH cluster) and Destination (RHEL 9 Open-Source Cluster self-managed cluster)RHEL 9 Open-Source Cluster Conf:Apache Services Configured and running:ZookeeperHadoopYarn is Default confHBaseMigration Approach:Export - Import approach:Did the export on the source side.Tried doing distcp to source which won't work because of authentication mismatch and non-presence of marred because it was not configured in the source side. So I did the winSCP to destination and tried import which didn't work because the MapRed was not configured because it was not in the requirement.So I skipped this approach.Snapshot approach:I took snapshot on source side and tried doing snapshot-export. It gave an error saying that the destination is a 'SIMPLE AUTH' but the source is Kerberos Authenticated. So, it didn't work.Manual Approach: Took snapshots on source.Hadoop Get: Used get command to copy the snapshots file, /hbase/data/D

Update Attribute processor Not working after upgrade from version 1.8 to version 1.9. We recently upgraded our Nifi Cluster from version 1.8 to version 1.9 and we were surprised to see the update attribute processor all going invalid for our existing flows. We had to manually fix the existing flow but this ideally should have been handled with the upgrade .Is there a way to overcome this ?

Hello everybody, I have a ODBC (DSN=IMPALA_PRD) setup to connect to Impala server through kerberos authentication.I have created the ticket in MIT with success.I've tested the connection with success.However when using the same ODBC connection on a python script or a PowerBI dashboard, it returns the error: ODBC: ERROR [HY000] [Cloudera][DriverSupport] (1110) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct and confirm if SSL should be enabled for the connection.\r\nERROR [HY000] [Cloudera][DriverSupport] (1110) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct and confirm if SSL should be enabled for the connection. For quick test i'm using this simple script:import pyodbcconn_impala = pyodbc.connect(DSN="IMPALA_PRD", autocommit = True)cursor = conn_impala.cursor()Could you please help me solve this riddle?Best regards, 

After installing the CDH Hadoop cluster using cloudera manager, everything was fine, then adding kerberos verification using the cm interface, and then. hive performs some slightly more complex HQL. The following error will appear：