Solved

Have enabled retry option in ListSMB and FetchSMB, since for some reason they are not able to list/fetch files in first attempt.  However, on retry the processor lists and fetches the file successfully.But, the scheduler is skips fews runs.Example:Scheduler is set to run every 30 min and let's assume the job first runs at 00:00 AM.On first run it fails to either list/fetch the file and the retry kicks in and the files is listed and fetched successfully.But, the next scheduler run which is suppose to be at 00:30 AM and 01:00 AM are skipped.Would like to know why retry is impacting scheduler. Thanks.    

SFTP CLI connects to the server and works fine.But, from PutSFTP it fails with below error, this is the only logs we are getting from PutSFTPnet.schmizz.sshj.transport.TransportException: Server closed connection during identification exchangeAlso, both CLI and NiFi are running on same server 

How to change the company in the profile? i select wrong company name 

Hello Everyone,We are currently conducting tests  to read data from CDC Debezium-enabled postgreSQL database and write to Azure EventHub.Actions performed:We uploaded three NARs needed to use the putWe added the following NARs as extensions:nifi-azure-nar-2.4.0.narnifi-azure-services-api-nar-2.3.0.narnifi-cdf-debezium-postgresql-nar-1.28.1.2.1.7.2006-1.narAfter a manifest refresh in EFM, we were able to see and use the PutAzureEventHub and CaptureChangeDebeziumPostgreSQLprocessor processors.We then configured the flow, published a new version, and confirmed that the flow was successfully pushed to MiNiFi.IssueThe Flow is pushed to MiNiFi but it is not being validated, thus the publish request fails, we have errors as if the newly added processors are not recognized, I believe?2026-04-02 12:22:09,379 INFO [MiNiFi logging handler] org.apache.nifi.minifi.StdOut 12:22:09.379 [Validate Components Thread-3] DEBUG org.apache.nifi.controller.AbstractComponentNode

HI,I have built a custom processor that uses another java library. This 3rd party JAR requires one sub directory ./data containing a file named config.ini.I put the NAR into the ./extensions subfolder of NiFi. When NiFi starts up, it unpacks the NAR  into the folder nifi/work/nar/extensions/my-processor-nar-1.0.nar-unpacked/This folder contains a subfolder NAR-INF.And this NAR-INF is the location where my subfolder "data" needs to be copied. When I manually copy the folder with the containing config file into NAR-INF, the processor works fine.But after the next restart of NiFi the subfolder is missing again.I need the structure of the unpacked NAR to include this:nifi/work/nar/extensions/my-processor-nar-1.0.nar-unpacked/data/config.iniDoes anyone have an example how I can build the NAR so that this subfolder and file are included?Like an example pom.xml snippet?Cheers,Karsten

HiIam using ListSMB which lists all the files correctly on the source system.However, not all files are fetched using FetchSMB, some of the files are skipped with below error.could this be due to // in the file path ?is it due to some issue on the SMB server settings ?FetchSmb[id=03ab6b24-019d-1000-0000-000032e5cf1d] Could not fetch file SDM-Prod-Reports/EOS_TFC_Metrics//EOS_TFC_Metrics.csv.: java.io.IOException: Could not create session for share smb://ifiler-smb03:445/SFTP- Caused by: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired- Caused by: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: java.util.concurrent.TimeoutException: Timeout expired- Caused by: java.util.concurrent.ExecutionExcep

Hi allI am using Knox for CDP SSO and would like to Enable the REST Catalog Service but use Keycloak for SSO. Is this possible? Private CDP, not Cloud. Thanks

Edit: Tried using CSVWriter, XMLWriter and AvroWriter. All NG. Nifi Version2.5.0ProcessorQueryDatabaseTableRecord Drivercom.microsoft.sqlserver.jdbc.SQLServerDriver I have a requirement to pull varbinary data from a table however the  output is always text.  I expect a result starting with 0x. According to the jdbc driver within nifi, varbinary is supported however I see it gets converted to bytes. Is there a way to maintain the 0x format for the writer? NotesJDBC source for statement varbinary is supported at https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java (line 360)

Hello, Does NiFi’s PutS3Object processor perform an MD5 integrity check for the files uploaded to Amazon S3?Thank youCristian

I am a Korean user. Recently, I received instructions to address the Apache Shiro security issue pointed out by the Korean Financial Supervisory Service. I am currently using CDH 7.1.8-1 and was instructed to update Shiro from the current version 1.11 to 1.13 or higher. As far as I know, it is being used in multiple places such as Knox and Kafka. Could you please let me know how to update it as soon as possible?

production pyspark jobs failure, on prem cloudera version 7.1.8 not supported at present. ranger kms health is unhealthy and failing pyspark job write to lake any suggestion to resolve ranger kms server issues. 

Hi everyone,I'm currently trying to write records containing timestamps to parquet.The writing part seems to work fine, as the resulting parquet file contains the avro schema '{"name":"ts_string","type":{"type":"long","logicalType":"timestamp-millis"}}' and the parquet column type "timestamp[ms, tz=UTC]".However when reading the same parquet file, I get the error message:ERROR [Timer-Driven Process Thread-6] o.a.n.processors.standard.ConvertRecord ConvertRecord[id=c2746584-b622-3127-f4fd-c46a9243553e] Failed to process StandardFlowFileRecord[uuid=cef04323-a7c0-4965-84d6-c2b8d16c3fac,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1768470437932-1, container=default, section=1], offset=17423, length=6057],offset=0,name=73ca82fa-02af-44d1-9c23-27e0b2b6588f,size=6057]; will route to failurejava.lang.ClassCastException: class java.time.Instant cannot be cast to class java.lang.Long (java.time.Instant and java.lang.Long are in module java.base of loader 'bootstrap')at org.

I'm experiencing very slow HDFS start in CDP 7.1.7SP1 for a cluster with a huge number of blocks (over 300 million, with each server having up to 40 million)I've checked thishttps://community.cloudera.com/t5/Community-Articles/Scaling-the-HDFS-NameNode-part-5/ta-p/327450and I wonder if setting dfs.blockreport.split.threshold to 0 might somehow speed up the processI've seen that the setting should go in NameNode Advanced Configuration Snippet (Safety Valve) for hdfs-site.xmIs this setting service wide so that a full restart is necessary?

I'd like to add runtimecatalog on the CML service. I'm currently using 12.8cuda version and Nvidia H100Gpu, and I'd like to use 3.12 jupyterlab. Is this version supported? When I looked it up on github, there didn't seem to be anything. Or can I customize it and use it after image build? Please adviseThank you.

Hi Team,We are using NIfi 1.26 in the Dev/Prod and we would like to get rid of using the Nifi template usage instead we are intend to use Flow definitions. However the users are still using the import and export method. Is there a way to disable the operations of import and export templates for all the users. 

I am trying to connect between NiFi and NiFi Registry. I'm able to authenticate with Registry from NiFi but I'm unable to see the buckets, not even the public ones. I've configured a SSL Context on NiFi that references a truststore containing Registry's Truststore and Keystore. I've imported NiFi's node certificates into the truststore on the registry's side. I've given the 'proxy and manage user bucket' permissions to the Node's identity in the Registry UI and Read Write and Execute permission to the same user on the Buckets. But I'm still unable to see the buckets on NiFi's UI. The API Responses also indicate that NiFi only has read permissions to the buckets. This is as if there is some anomaly during login.. Note:1) I'm using a Clustered NiFi Setup. I have verified that my node identity is 'CN=node-0-nifikop'.2) Both NiFi Cluster (NiFiKop) and the NiFi Registry (Helm Release) are running inside a Kubernetes ClusterError Samples:The Error on NiFi UI:  Error.jpegRegistry Bu

Dear Community,   After a successful installion of Cloudera Runtime 7.0.3, we tried to do a Kerberization process. (We did the same before with 5.14) Everything went fine with the Kerberos wizard, but after in the config deployment phase, the YARN NodeManagers failed to start with the following error:   Error starting NodeManagerorg.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container executor at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:394) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:936) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1016)Caused by: java.io.IOException: Linux container executor not configured properly (error=-1) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:307) at org.apac

Dear Support Team,I’m reaching out to request an update to the email address linked to my Cloudera account.I have already reached to the Cloudera community but haven’t received a response yet, so I’m turning to you for assistance. Thank you in advance for your help .

According to https://hive.apache.org/general/downloads/"The Apache Hive Community has voted to declare the 3.x release line as End of Life (EOL). This means no further updates or releases will be made for this series.We urge all Hive 3.x users to upgrade to the latest versions promptly to benefit from new features and ongoing support."CDP 7.3.1 is still very much based on Hive 3.https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/hive-introduction/topics/hive_whats_new_in_this_release_hive.htmlhttps://docs.cloudera.com/cdp-private-cloud-base/7.3.1/hive-introduction/topics/hive-apache-hive-3-architectural-overview.htmlWhen will Cloudera on base support Hive 4?Thanks 

Hi everyone.Just want to check as of latest CML version, has there been any support for using GPU for Spark in CML ?I have seen:1. GPU used for Spark in CDE, or2. GPU used in general Python project ( using torch, tensorflow, rapids' cudf/cuml) in CML,but can't find any for Spark in CML. If there is no such support; is there any specific reason why it cannot be done while it can be done for general Spark on k8s.Thanks for any info...

Hi Team,We are planning to deploy the Iceberg Hive runtime JAR (iceberg-hive-runtime.jar) in our CDP Private Cloud Base 7.1.9 cluster (Cloudera Manager 7.11.3) to enable Hive and Spark to work with Iceberg tables.Iceberg Hive runtime JAR: iceberg-hive-runtime-1.3.1.jar,iceberg-spark-runtime-3.3_2.12-1.3.1.jarstep: 1Upload the JAR to HDFSCreate a directory in HDFS:hdfs dfs -mkdir -p /user/share/libUpload the JAR:hdfs dfs -put iceberg-hive-runtime.jar /user/share/lib/hdfs dfs -chmod 755 /user/share/lib/iceberg-hive-runtime.jarConfigure Hive via Cloudera ManagerNavigate to Hive → Configuration → Auxiliary JARs DirectoryAdd:hdfs:///user/share/libStep 2:Configure Spark via Cloudera ManagerNavigate to Spark → Configuration → Spark Client Advanced Configuration Snippet (Safety Valve)Add:spark.yarn.jars hdfs:///user/share/lib/iceberg-hive-runtime.jarStep 3:Restart ServicesRestart Hive, Spark, and YARN. We would like guidance and confirmation on the steps, any possible impacts, and best pr

Hello,<in a Public Cloud Environment in the Operational Database with Azure: Snapshots were failing.The HBase-Service Command Tab showes that it occurs while copying a snapshot to another directory because it could not acquire a SAS token. (*)Usually those errors are related to missing rights on Managed Identities. - But we double-checked all Azure Managed Identity IAMs and RBACs.Sometimes the SAS-Token Failure is related to Kerberos which is disabled./>What could have gone wrong? (*):Client.RangerRESTClient: ===>> RangerRESTClient.init() : Since mKeyStoreType is NULL, setting System default.[mKeyStoreType=jks] Exception in thread "main" Failed to acquire a SAS token for get-acl on / due to org.apache.hadoop.security.AccessControlException: Permission denied. at org.apache.hadoop.fs.azurebfs.services.AbfsClient.appendSASTokenToQuery(AbfsClient.java:1233)

Hello communityI was wondering if anyone has any experience with writing entries into redis with dynamic TTLs. From what I can tell, it is only possible to set the TTL via the DistributedMapCacheClientService and as such it is set 'globally' per flow. For me the ideal case would've been if it was possible to set the TTL based on flowfile attributes.Any advice would be greatly appreciated. Thanks 🙂 

I am new to Nifi and trying to load csv data in postgres but I am getting error:FlowFile[filename=test_data.csv]. Routing to failure.: java.lang.IllegalArgumentException: Record does not have a value for the Required column 'name'so I have checked everything and the csv file I am uploading contains the data.I have also checked solutions for issues like this but nothing worked out. I will attach liks to the screenshots it contains the configuration for PutDatabaseRecord and the query for the table I created in postgre and screenshot of the csv file. https://drive.google.com/file/d/1TCJTcNJHRslG4wEct1eZmuYz54Gp9e9o/view?usp=drive_linkhttps://drive.google.com/file/d/1Ye6pggRN4YTQVGxylNarjZh4dlX-ByQ1/view?usp=drive_linkhttps://drive.google.com/file/d/17zLRe88DSssU9DI3rrJLCHQmUgAj9FYX/view?usp=drive_linkhttps://drive.google.com/file/d/1XKt1jbZks8jE0kO4CrT3nmRR4LFcIn1e/view?usp=drive_link Note: please let me know how I can upload image here directly

Hello,We are on CDP 7.1.9 SP1 and have Hue configured to be accessible via Knox. We would like to limit the access to Hui UI to a particular user group. I did found below documentation, however, unsure if that will help incase of accessibility via Knox as it looks to be more LDAP relevant.https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/securing-hue/topics/hue-restrict-user-login.htmlPlease advise.ThanksSnm1523