Q&A

Hello, everytime I make a new version of or copy 4-5 big process groups, the Nifi UI start throwing 503 errors and sometimes It dies for 15-30 minutes. I tried changing the Garbage Collector algorythm but it did not work out, and I know that adding up memory heap will be useless. What else can I do? Thank you

Hello everyone,during the snapshot for an operational database, we had issues regarding the storage address/location. We were only able to place the snapshot in the same Azure storage container as the HBase service. Otherwise, we got errors for insufficient rights. Are there any official docs helping in this topic?Can Hadoop be modified to allow the use of a different container for storing these snapshots?or how can we give HBase the right to access different storage locations or different storage container?But then, what backups are stored in the default location? Only the cluster backups?Regards(Our Post leading to these questions: https://community.cloudera.com/t5/Support-Questions/HBase-Snapshotting-Failed-because-of-Failure-to-aquire-SAS/td-p/413360)

During testing of the Enterprise Message Bridge implemented using Apache NiFi, it was identified that the Content-Type header becomes null when messages are routed from Message Broker A to Message Broker B.The bridge is designed to enable message flow between the two brokers using Apache NiFi as the intermediary integration layer. However, while validating message headers at the destination broker, the Content-Type attribute is not being preserved and is received as null.This behavior was observed during message inspection at the consumer side connected to Broker B.May I know how to preserve message Content-Type attribute/properties ? Using : ConsumeJMS and PublishJMSProtocol: AMQP 1.0

Have enabled retry option in ListSMB and FetchSMB, since for some reason they are not able to list/fetch files in first attempt.  However, on retry the processor lists and fetches the file successfully.But, the scheduler is skips fews runs.Example:Scheduler is set to run every 30 min and let's assume the job first runs at 00:00 AM.On first run it fails to either list/fetch the file and the retry kicks in and the files is listed and fetched successfully.But, the next scheduler run which is suppose to be at 00:30 AM and 01:00 AM are skipped.Would like to know why retry is impacting scheduler. Thanks.    

SFTP CLI connects to the server and works fine.But, from PutSFTP it fails with below error, this is the only logs we are getting from PutSFTPnet.schmizz.sshj.transport.TransportException: Server closed connection during identification exchangeAlso, both CLI and NiFi are running on same server 

Hello, I am working on a CDP 7.1.9 environment and I am currently testing the Impala ODBC connection using the Cloudera Impala ODBC Driver 2.8.4 on a Unix system with unixODBC. The configuration is as follows: ODBC driver installed and registered in /etc/odbcinst.ini:   [Cloudera ODBC Driver for Impala 64-bit]Description=Cloudera Impala ODBC Driver (64-bit)Driver=/opt/cloudera/impalaodbc/lib/64/libclouderaimpalaodbc64.soUsageCount=1   DSN configured in /etc/odbc.ini SSL certificates correctly configured and integrated into the OS trust store Trying to connect using LDAP authentication (username/password)         [Impala_CDP]Driver=/opt/cloudera/impalaodbc/lib/64/libclouderaimpalaodbc64.soHost=****************Port=21050Schema=*************AuthMech=3SSL=1TrustedCerts=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pemLogLevel=6LogPath=/tmp/impala2.log However, the connection fails with the following error: isql -v Impala_

How to change the company in the profile? i select wrong company name 

Hello Support Team,We are setting up a CDP Trial environment and noticed that accessing " https://archive.cloudera.com/cdp-public/ "consistently returns an HTTP 404 error, even without authentication. This prevents us from verifying access to public CDP artifacts during the trial installation. Could you please confirm whether the cdp-public path is available for trial users or if it has been deprecated or moved?Best regards,Dae‑Yeol Lee

Hello Everyone,We are currently conducting tests  to read data from CDC Debezium-enabled postgreSQL database and write to Azure EventHub.Actions performed:We uploaded three NARs needed to use the putWe added the following NARs as extensions:nifi-azure-nar-2.4.0.narnifi-azure-services-api-nar-2.3.0.narnifi-cdf-debezium-postgresql-nar-1.28.1.2.1.7.2006-1.narAfter a manifest refresh in EFM, we were able to see and use the PutAzureEventHub and CaptureChangeDebeziumPostgreSQLprocessor processors.We then configured the flow, published a new version, and confirmed that the flow was successfully pushed to MiNiFi.IssueThe Flow is pushed to MiNiFi but it is not being validated, thus the publish request fails, we have errors as if the newly added processors are not recognized, I believe?2026-04-02 12:22:09,379 INFO [MiNiFi logging handler] org.apache.nifi.minifi.StdOut 12:22:09.379 [Validate Components Thread-3] DEBUG org.apache.nifi.controller.AbstractComponentNode

Hello NiFi Community! I have a three-node NiFi cluster. I use this to ingest data from multiple source systems. From time to time, we experience this issue where the flowfiles are getting stucked in round robin queues and just sit idly there.   I've also tried adding funnels just to test if they can still proceed downstream. Upon monitoring the disk usage of the cluster, it never exceeds 30% utilization. Anyone knows what's causing this issue? Tried checking the nifi-app logs but no luck. Thank you!  

HI,I have built a custom processor that uses another java library. This 3rd party JAR requires one sub directory ./data containing a file named config.ini.I put the NAR into the ./extensions subfolder of NiFi. When NiFi starts up, it unpacks the NAR  into the folder nifi/work/nar/extensions/my-processor-nar-1.0.nar-unpacked/This folder contains a subfolder NAR-INF.And this NAR-INF is the location where my subfolder "data" needs to be copied. When I manually copy the folder with the containing config file into NAR-INF, the processor works fine.But after the next restart of NiFi the subfolder is missing again.I need the structure of the unpacked NAR to include this:nifi/work/nar/extensions/my-processor-nar-1.0.nar-unpacked/data/config.iniDoes anyone have an example how I can build the NAR so that this subfolder and file are included?Like an example pom.xml snippet?Cheers,Karsten

We have requirements in a project to Analyze whole workload on Cloudera HDFS data and metadata. We have created a tool which have integration with AWS Redshift, GCP BigQuery and Snowflake. This would be our first on-premise integration in to the tool. Any way I can replicate Cloudera Hadoop on Windows for initial prototype testing without VM which makes system very slow ? RegardsRahul

Hi, I'm having a problem with a NiFi flow. I use UpdateAttrubute processor when I create an attribute like this:  formatDateFrom ${dateFrom:format("yyyy-MM-dd")} After this I have a PutSql processor that changes a status in DB. When everything is fine, it is ok. However, when something is wrong with the change, this status and my flow go to retry this attribute formatDateFrom disappears and is no longer present in the flow. Why does this happen and how can I secure this attribute? Attributes created with EvalueateJsonPath didn't disappear, only those created with UpdateAttribute.

I am working with Apache NiFi and trying to replicate a pattern that we had done in existing system.In existing system, we have multiple (around 10) email accounts, and we iterate over them using a loop, connecting to each mailbox, reading emails, and processing them.In NiFi, I have configured a ConsumePOP3 processor and it works fine for a single email account. However, I need to handle multiple email accounts in a scalable and maintainable way. We are not able to connect any processor or something before ConsumePOP3.

HiIam using ListSMB which lists all the files correctly on the source system.However, not all files are fetched using FetchSMB, some of the files are skipped with below error.could this be due to // in the file path ?is it due to some issue on the SMB server settings ?FetchSmb[id=03ab6b24-019d-1000-0000-000032e5cf1d] Could not fetch file SDM-Prod-Reports/EOS_TFC_Metrics//EOS_TFC_Metrics.csv.: java.io.IOException: Could not create session for share smb://ifiler-smb03:445/SFTP- Caused by: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired- Caused by: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired- Caused by: java.util.concurrent.ExecutionExcep

Hello everyone,I’m currently facing a performance issue with Cloudera Impala when using it through the ODBC driver.Here is my environment:CDH: 6.2Impala: 3.2.0 (Impala 2.7.1 ODBC Driver)Integration: SAS 9.4 M7 (SAS Access to Impala)Authentication: Active Directory (user/mdp)Security: SSL enabledI have configured my /etc/odbc.ini as follows:[Impala_KRB14]Description=ODBC Cloudera Impala SSL Node 14Driver=/opt/cloudera/impalaodbc/lib/64/libclouderaimpalaodbc64.soHOST=""""""""" ##hiddenPORT=21050Database=""""""""""" ##hiddenAuthMech=3SSL=1TrustedCerts=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pemLogLevel=2LogPath=/tmp/impala_odbc.logSSP_BATCH_SIZE=60536RowsFetchedPerBlock=60536TSaslTransportBufSize=4000I also tried adding several ODBC options/tuning as seen in the odbc.ini parameters, but unfortunately I did not see any performance improvement.The issue is mainly that:Opening a table through SAS takes a long timeEven simple queries like SELECT ... WHERE ... are very slowI would like t

I have an old cluster that had some hard drives fail. On one server (node06), the OS drive failed and instead of building it from scratch, I created an image from another node (node02) in the cluster. I then started the new server (node06) and changed the name and the IP address to match node06. But when I put it back on the network, Cloudera kind of went crazy because it thought that node was node02 and duplicated it. Is there something I need to do on the imaged node (node06) so it doesn't think it is node02? I hope this makes sense.

Environment - NiFi 1.23.2, 3-node clusterProblemAll of sudden, all the cluster nodes becomes unavailable:Spoiler (Highlight to read)Error: Received disconnection request message from cluster coordinator with explanation: org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption.Disconnecting node due to org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption.Event Reported for atl-fm-nifi01.corp.pps.io:8443 -- Node disconnected due to org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption.Erro

Hi allI am using Knox for CDP SSO and would like to Enable the REST Catalog Service but use Keycloak for SSO. Is this possible? Private CDP, not Cloud. Thanks

Hi, Nifi keep disconnecting due to below error , appreciated any help .Node disconnected from cluster due to org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption.2023-09-21 12:34:09,572 INFO [main] o.a.n.c.l.e.CuratorLeaderElectionManager Cannot unregister Leader Election Role 'Primary Node' because that role is not registered

Dears,The Hive MERGE is failing with the error below. Since yearmonth is a partition column, implemented a Type-2 SCD script to track changes. However, even after including the partition column in the join condition, the same error persists.Error while compiling statement: FAILED: SemanticException [Error 10007]: Ambiguous column reference deal_yearnumber in tgt.ScriptMERGE INTO accrued_interest_metric tgtUSING temp_ACCRUED_INTEREST_METRIC srcON tgt.deal_number = src.deal_numberAND tgt.deal_metric_end_date = DATE '9999-12-31'WHEN MATCHEDAND tgt.deal_hash <> src.deal_hashTHEN UPDATE SETdeal_metric_end_date = CURRENT_DATE - 1,update_date = current_timestamp()WHEN NOT MATCHED THENINSERT (deal_number,deal_metric_start_date,deal_metric_end_date,deal_branch_cd,deal_type_cd,deal_reference_number,currency_cd,deal_maturity_date,deal_last_rollover_date,deal_next_rollover_date,deal_term_type_cd,deal_accrued_interest_rate,status_cd,status_description,deal_account_number,deal_interset_account

Edit: Tried using CSVWriter, XMLWriter and AvroWriter. All NG. Nifi Version2.5.0ProcessorQueryDatabaseTableRecord Drivercom.microsoft.sqlserver.jdbc.SQLServerDriver I have a requirement to pull varbinary data from a table however the  output is always text.  I expect a result starting with 0x. According to the jdbc driver within nifi, varbinary is supported however I see it gets converted to bytes. Is there a way to maintain the 0x format for the writer? NotesJDBC source for statement varbinary is supported at https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java (line 360)

Hello All,i have plan to renew the tls certificates for my prod cluster. I have already generated csr for all hosts and got the signed from my clients. Could you please share the steps for renewing. I gone through cloudera documentation but still having doubts. Can someone help. It would be much appreciated. 

Hello, Does NiFi’s PutS3Object processor perform an MD5 integrity check for the files uploaded to Amazon S3?Thank youCristian

Hello everyone, I have been trying to solve an issue on my on-prem NiFi 2.4.0 application for a month which being PutEmail timeout errors. Specifically, we were using NiFi 1.9.0 and local smtp server to send mails, after updating to 2.4.0 we started using office365 smtp server. swaks --server smtp.office365.com --port 587 --tls --helo test.local --auth LOGIN --auth-user notifymail@****.com --auth-password '****' --from notifymail@****.com --to okan.ergun@****.com --header "Subject: Test Email" --body "Test SMTP Relay Service"With the command above I never had timeouts, and every mail was successful from the application environment, although NiFi gets timeouts time to time and doesn't have any specific times to have timeouts. Processor Details:SMTP Hostnamesmtp.office365.comSMTP Port587Authorization ModeUse PasswordSMTP Usernamenotifymail@****.comSMTP PasswordSensitive value setSMTP AuthtrueSMTP STARTTLStrueSMTP Socket Factory${literal('')}SMTP X-Mailer HeaderNiFiAttribut