Ask the Community

DearsI am an admin for a CDP Private Cloud Base customer who is struggling to raise support tickets on their respective myCloudera account. I did confirm that they purchased the highest level of support which they have not utilized ever since the cluster was deployed.Recently there has been several incidents on the cluster which required cloudera support but due to not having a support account we had to make due with help from cloudera partners. Whenever I access the support portal and try to raise a ticket I am faced with "This page is restricted please contact your admin"I thought that once I make an account with the company's email support will be active straight away but for some reason it is not. Has anyone faced this before and how was it resolved?

I am usinng Apache Nifi 1.28 and Hortonwrks HDP 2.5.0, while putting the data in hdfs using puthdfs processsor , i am able to get the following error: `2024-11-02 09:56:58,321 ERROR [Timer-Driven Process Thread-10] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=eb392407-0192-1000-f474-559950bfef43] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowedjava.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowedat java.base/javax.security.auth.Subject.getSubject(Subject.java:347)at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:338)at org.apache.hadoop.hdfs.DFSClient.<init>(DFSC

Hi,Currently we are using ConsumeAMQP processor to fetch messages from Rabbitmq queues.There are some use cases in our system where Rabbitmq streams seem to be a good fit.I have tested ConsumeAMQP to consume messages from Rabbitmq streams queue, but it doesn't work as expected.   I believe the new functionality related to OFFSET management is not available in ConsumeAMQP processor.Any insights on this would be helpful.  Am I missing something?  If functionality is not part of current processor and would like it to be enhanced, how should I request?Thanks in advance.  

All, I have two separate NiFi instance and I want to push provenance events from one instance to another. Source NiFi's hostname is nifi8443 and Target NiFi's hostname is nifi8444.Both of these instances are running over HTTPS. On the "Target NiFi" I set up an input port called Test. On source NiFi, here's what my SiteToSiteProvenanceReportingTask looks like. Here's a couple things I did. I realized that the reporting task on NiFi8443 is essentially acting like an client and NiFi8444 is acting as the server. I added the NiFi8444 server certificate into a truststore.p12 file on NiFi8443 and created the corresponding SSLContextService to point to that Truststore file. I resolved the SSL issue and then I was reading on the target node (ie. NiFi8444) I had to make an identity for the full DN of the source (ie NiFi8443) certificate. So on NiFI8444, I created a new user called this - CN=nifi8443, O=NiFi Server, L=San Francisco, ST=California, C=USThen I added th

Hello, I have a question about the NiFi Python extension. I tried to build my own processor with Hatch and created a NAR file, but when I copy it into the lib folder in NiFi, I encounter this error:"2024-07-22 10:43:47,322 WARN [main] org.apache.nifi.nar.NarClassLoader .\work\nar\extensions\nifi_python_extensions-2.0.0.dev0.nar-unpacked does not contain NAR-INF/bundled-dependencies!"Is there any help available to resolve this?

"result" : "SUCCESS","correlationID" : "X8058296","generationTimestamp" : "1354","meHostName" : "rc01;rik2jc","nccVersion" : "SPS__R1",   This is my input in my nifi and i want to do changes in meHostName means i want mehostName to have 1st value like in this case rc01 and after that creating 1 more field with name devicename having second value means rik2jc   I acheived the 1st part means i configured in UpdateRecord processor  for meHostName and it is printing 1st value means it is running fine ,now please help me in acheiving devicename to have 2nd value of meHostName which is rik2jc . i have tried split(/meHostName, ';')[1] but its not working          

Hello All,Actually, I'm getting one error from my post call in nifi and in that I'm making post call with form data which contains file, JSon body but even after properly configuring the Httpinvoke it's giving me this errorjava.lang.IllegalStateException: closed Can you please help me in this these are my configuration which I have donethis preview which i got And the configuration:-and the Entire pipeline 

Hi, Could be please help me why categoryList is combining all values inside one list instead of 2nd index of paths object , like productList is working fine only difference is that i added one more condition for categoryList that is causing the issue. could you please suggest what i am missing here ?Input { "correlationId": "4c7f8c69-36eb-4f65-bded-6ed08503631a", "sourceSystemId": "PCS", "messageTimestamp": "2024-10-29T13:10:54.297-04:00", "id": 2173, "promotionId": 569, "offerId": 2173, "description": "Price Rule wIncl and Excl - HCO UK", "levelCode": 0, "typeCode": 0, "templateId": 4, "isCouponCodeRequired": 0, "startDateTime": "2024-10-30T00:00:00.000Z", "endDateTime": "2024-10-31T23:59:00.000Z", "exclusiveDiscountInd": 0, "tiers": [ { "conditions": [ { "conditionId": 374, "buySpendType": 0, "buySpendValue": 1, "buyUom": "EA", "conditionsMerch": [ { "merchandiseLevel": 5

I have requirement where I need to append and prepend the attribute with "$$$". how ever, Nifi expression language is appending or prepending with "\$" For example: if attribute named "myattr" value is "This is Test" when I do ${myattr:prepend('$$$'):append('$$$')} I am expecting "$$$This is Test$$$" but the output is "\$This is Test\$" Is there any way to append or prepend the attribute with "$$$"

I have the AES decryption working in a Nodejs script using crypto locally on my computer, with the following parts:Text to decrypt: c6 c7 4b 49 0d cf 5c 20 87 0a e0 cd c4 a7 bf 94 d8Key: 3E 9B 26 FE 46 4F 6D 2D 2F 69 5D 87 8A 07 93 74Initialization Vector (IV): 2d 2c 83 42 00 74 1b 16 20 c0 7d 13 20 00 00 00Correct result: 14 25 79 ed a8 ff a7 00 00 e5 03 00 00 be 03 00 00So I know both key and IV are correct. I am using AES128, CTR and NoPaddingNow, I need to do it in Apache Nifi, and I am trying to use the DecryptContent processor. I am confused about how to make it happen and have tried quite a few things, the latest being:Cipher Algorithm Mode is set to CTRCipher Algorithm Padding is set to NoPaddingKey Specification Format: RAWI have set the content of the incoming flowfile content toc6c74b490dcf5c20870ae0cdc4a7bf94d84E69466949562d2c834200741b1620c07d1320000000(With 4E6946694956 as the NiFiIV delimiter)This results in an error from the DecryptContent processor: "Wrong IV length: m

I would like to know how to obtain the values of total_io_mb, mapreduce_inputBytes, and mapreduce_outputBytes for each application within a certain period of time.I believe that these values are written to the yarn log file. So, I would like to know SME's advice on how to do this, that is, how to configure these values to be written to the log.By Cloudera Manager, we are not able to get but the rate(ex. yarn_application_hdfs_bytes_read_rate). However, alternative ideas are welcome.The purpose is to evaluate the disk IO of the entire CDH cluster, which is made up of several hundred bare metal machines, and to estimate the disk IO required for the next generation of bare metal and/or storage nodes.

I am running into an issue where I am unable to successfully use a PutSQL or ExecuteSQL after returning from a Batch Processor group.The general scenario is this: I need to watch a folder for files that will be created with information to be uploaded to a PostgreSQL database. The files will have multiple lines, each line is to be inserted as a separate record into a temporary/staging table. When all the records from that file have been uploaded, I need to call a stored procedure that will process all those records and update other permanent tables all at once.I came across the concept of using a processor group set with 'Process Group Outbound Policy' set to 'Batch Output'.So, I am able to use GetFile to get each file as it is produced, then pass it into my batch processor group, where it splits the lines into individual flow files, and performs the inserts into my staging table. That all works perfectly.I then use a MergeContent processor to reduce those split files back into a single

Dears ,Kindly note that I am trying to send InvokeHttp request as Get with the following body=   {    "Domain": [        ["write_date",">=","2024-07-01 04:00:00.000"],        ["write_date","<=","2024-07-12 04:00:00.000"]    ]}  I have done some investigations and found no solution for my problem    

I am using Apache NIFI 1.28  version and i have successfully configured the  core-site.xml and hdfs-site.xml correctly. but when i run the processor it is giving the following error:• 13:00:11 PKT ERROR PutHDFS[id=d2b7ad77-0192-1000-e937-9d1797c4e0fd] Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.lang.NoClassDefFoundError: org/apache/commons/collections/map/UnmodifiableMap: java.lang.NoClassDefFoundError: org/apache/commons/collections/map/UnmodifiableMap - Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.map.UnmodifiableMapI have included all possible combination of different jar files but all in vain, i am also attaching the lib folder as well   

I am trying to remove the punctuation from a single column using the UpdateRecord processor.  But I am unable to get it right. I attached screenshots of my configurations and properties.   

Both Nifi and Nifi Registry services are configured to use LDAP, the user authentication in both of them are working just fine. The communication between Nifi and Registry is made by calling IP and Port and I use the same addresses to access their UI and login with my LDAP user.But when I try to start version control in one of the Nifi access groups it can connects to the Nifi Registry but returns "No available buckets" even with the buckets already been created in Nifi Registry.This is the Registry logs when I use the "Start version control" option inside Nifi and it can't find the buckets:  2024-10-17 17:39:54,289 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.x.X509CertificateExtractor No client certificate found in request.2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.IdentityFilter Attempt

Issue: The lineage tab in the UI disappears when creating my own DataSet and Process types.Details: Because I don’t want to inherit all the relationship attributes from DataSet, I would like to create my own custom, My_DataSet that inherits directly from Asset, just the same the Atlas’ buil-in one.Also, because of the relationships defined between DataSet and Process, I also created my own process type, say My_Process and therefore linked the new types through the new, my_dataset_process_inputs and my_process_dataset_outputs.These relationships, just like the buil-in Atlas’ ones tie together the relationships attributes My_DataSet.inputToProcesses, My_DataSet.inputFromProcesses to My_Process.inputs and My_Process.outputs. So just like Atlas, the only difference, less relationship attributes in the My_DataSet definition, as mentioned before.Back to the issue, now I don’t see the “Lineage” tab in the UI. The relationships between actual entities, when created, are shown properl

Hi, experts,I need to enrich flowfile with data from database table, but before this table should be filtered based on attribute value. Is it possible to filter data within lookup processor? If not, please advise any solution for my task.Best regards,Alexey  

CDICO2024E: Batch of 5 records could not be written to the data source: [SQLSTATE HY000] c error: [IBM][Hive JDBC Driver][Hive]Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (error code: DATA_IO_ERROR)  

Hello,We have done this before but now we are trying to make a change on a table column and we are getting the follow error:Error while compiling statement: FAILED: Execution Error, return code 40013 from org.apache.hadoop.hive.ql.ddl.DDLTask. Unable to alter table. Cannot change stats state for a transactional table pro_con.dmcrp001_th_importe_consolidado_contrato without providing the transactional write state for verification (new write ID 327, valid write IDs null; current state null; new state {}The statement that we are executing is:alter table `pro_con`.`dmcrp001_th_importe_consolidado_contrato` change column `fecha_movimiento` `fecha_proceso_dia` timestamp comment 'Fecha de referencia de los datos.'; 

Hello, We have HDF 3.5 cluster, where we have Kafka and Nifi. We are using postgres12 for Ambari, Ranger, Schema Registry. Now we are asked to migrate to postgres 14 or 15. Can you please suggest if we can migrate? Our concern is on HDF3.5 compatibility with postgres 14 or 15. If it is OK to migrate, any configuration updates required at cluster.Need your inputs here please.Thanks & Regards,Anil

Hello,after enabling Kerberos, I created a user in AD and on host machine which is part of a group of superusers in CM:than I set permissions in Ranger like so:and after doing kinit and hdfs dfs -ls / I have an error:WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]ls: DestHost:destPort FQDN:8020 , LocalHost:localPort FQDN/X.X.X.220:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]Could someone help please? 

Hello Cloudera Community,I'm encountering an issue with my NiFi deployment where users are getting frequent "Session Expired" messages approximately every 30 minutes. When this happens, they have to refresh their browser, which causes them to lose any unsaved modifications in the process.Setup details:NiFi is integrated with Azure Active Directory .Below are the relevant configurations from my nifi.properties file (please let me know if any more details are needed). nifi.security.user.oidc.additional.scopes=profilenifi.security.user.oidc.claim.identifying.user=upnnifi.security.user.oidc.client.id=********************nifi.security.user.oidc.client.secret=***************nifi.security.user.oidc.connect.timeout=180 secsnifi.security.user.oidc.discovery.url=https://login.microsoftonline.com/************nifi.security.user.oidc.preferred.jwsalgorithm=nifi.security.user.oidc.read.timeout=180 secsnifi.web.war.directory= ./libnifi.web.proxy.host= nifi.**************.comnifi.web.https.p

I am facing below issue in my flow.. flowfile is currently penalized and data cannot be processed at this time .Any idea how to solve this issue .Thanks in advance . 

Hi Folks,This is likely a multipart problem, but I've been trying to get the following sample JSON to distribute the "id" value from the root of the JSON object into a flattened array of objects with the "id" value and the "code" value from each category (or subcategory in the nested structure).  For example, given the following JSON payload: { "id": "1234", "categories": [ { "code": "A", "name": "Alpha", "subcategories": [ { "code": "AA", "name": "Alpha Alpha" }, { "code": "AB", "name": "Alpha Bravo" } ] }, { "code": "B", "name": "Bravo" }, { "code": "C", "name": "Charlie" }, { "code": "D", "name": "Delta" } ]} The Desired Output would be:  [ { "individualId": "1234", "categoryCode": "A", "categoryName": "Alpha" }, { "individualId": "1234", "categoryCode": "AA", "categoryName": "Alpha Alpha" }, {