Solved

I am getting an error when using the ExcelReader to convert an Excel sheet to JSON. Everything works except for some dates. If I remove the Date Format setting on the controller, no issue, but get the date in milliseconds from 1/1/1970, which I do not want. The ExcelReader is failing on the date 05/01/2017. This is a valid MM/dd/yyyy date. The error I get is: ERROR015de330-01a0-1000-0000-00001cb4e04eAll NodesConvertRecord[id=015de330-01a0-1000-0000-00001cb4e04e] Failed to process FlowFile[filename=Test_File.xlsx]; will route to failure: org.apache.nifi.processor.exception.ProcessException: Could not parse incoming data- Caused by: org.apache.nifi.serialization.MalformedRecordException: Read next Record from Excel XLSX failed on row 59 in sheet Data- Caused by: org.apache.nifi.serialization.record.field.FieldConversionException: Conversion failed for [05/01/2017] named [column_9] to [java.time.LocalDate] [java.time.format.DateTimeParseException] Text '05/01/2017' could not be parse

HelloHow to read the body of the mail in nifi? I don't see any relevant processor for this.I see there is an "ExtractEmailAttachments" processor but will it read body reliably? We tried to test and see that it is able to make two flowfiles after executing. One is with a text/plain and has the email body present and another one is text/html having some html content having what we sent in mail as well.But due to its name being "ExtractEmailAttachment" and not something like ExtractEmailBody I am not sure.How to effectively and reliably read the mail body? In NiFi 1.28 we had Python (which was able to read the body of mail) but in NiFi 2.8 we don't have Python and able to see GroovyScript which is not allowing import of javax.mail type libraries which is required for writing the logic to fetch the email body.

Hello.Before deleting rows in a specific table (463,462 rows in table), HDFS file size is :$ hadoop fs -du -s -h /apps/hive/warehouse/prd_thmil.db/th_mil_fb_code_value_brut54.2 M 162.6 M /apps/hive/warehouse/prd_thmil.db/th_mil_fb_code_value_brut54.2 Mb is the size of 1 file and each file is replicated 2 times so 162.6 Mb is the total size, it OK.But after deleting more than 450,000 rows in the table (12,890 rows remaining after the DELETE), the file size didn't change at all.Is it normal ? When new rows are added in the table, file size won't grow and HDFS will 'overwrite' older data with the new one ?Regards

I'm running several ETL flows in CDF/Nifi. The flows all involve writing some data to Iceberg tables. These flows are mostly running on a schedule that ranges from hourly to weekly (i.e., infrequent writes). Each data flow deployment has a few independent DAGs that all end with the PutIceberg processor, such that there are multiple separate ETL processes running at different intervals all in the same Nifi deployment.The problem I see occurs pretty rarely (~once a month) and only for some deployments. Occasionally, the PutIceberg processor will error with 'Failed to acquire a SAS token' (see full error log below).This will continue happening every time the processor runs unless I restart the flow and then it will run fine again. It seems to happen more often when the processor runs at a daily interval. My flows are using Nifi runtime 1.27.0.2.3.14.0-14 and I'm on CDP Public Cloud on Azure.```org.apache.hadoop.fs.azurebfs.contracts.exceptions.SASTokenProviderException: Failed to acq

PVC 서비스 재시작 후 'Pod Count' 서비스 재시작이전보다 감소하는 증상이 발생하여 정상유무를 확인하고자문의를 드립니다.+ 첨부 그림 파일처럼 서비스 재시작전에는 "Pod Cont" 갯수가 54개로 표시되지만, 서비스 재시작 후 18개로 줄어들어서 해당 증상이 비정상일 경우 확인해야 되는 항목과 참고할 수 있는 로그정보가 필요합니다.'Pod Count'에는 어떤 정보가 포함되어 있나요? 

Hello TeamWe have an increasing number of applications being added to our NiFi server. Now the nifi has got too slow that it won't load the home page or when we click a process group then it won't open despite browser url showing the process group id as if page should have gone there.Also the apis we have implemented using HandleHttpRequest return Socket Timeout error.Please note these were working fine earlier. What should we do?

I missed my CDP-0011 certification appointment. What should I do?The certification fee is very expensive for me, so I cannot repurchase it. Has this situation happened to any of you, and what is the correct action I should take?

 Hi all,I am working with Cloudera Data Visualization 7.1.6 and creating a dashboard that includes a KPI visual.The visual uses a dataset based on a Hive query. Is there any way to pass dashboard filter values or parameters directly to the dataset query?Based on the official documentation and the information I found online, it appears that Cloudera Data Visualization applies filters only at the visualization or UI level and does not support passing parameters to the underlying dataset query.Could someone confirm whether this is an architectural limitation, or whether there is an alternative solution or workaround?Thanks in advance for any suggestions or help.

 I have a question regarding the long-term support alignment for CDP Private Cloud Base 7.1.9.As you know, CDP 7.1.9 is an LTS release supported until October 2028. Initially, this version only supported Python 3.8, but due to the relatively early EOL of Python 3.8, support for Python 3.11 was introduced starting from SP1 CHF9.On the other hand, CDP 7.1.9 depends on OpenJDK 17, whose end of support is currently December 31, 2027. This creates a potential misalignment with the CDP LTS support period extending into late 2028.Given the precedent with Python (i.e., adding support for a newer version within the same LTS release via CHF updates), we would like to understand the roadmap for Java:Are there any plans to support a newer OpenJDK version (e.g., OpenJDK 21 or later) within CDP 7.1.9 through future SP/CHF updates?If so, is there an estimated timeline or target CHF/SP where this support might be introduced?If not, how does Cloudera plan to address the gap between the OpenJDK 17

During my preparation, I used multiple learning resources, including official Cloudera documentation, hands-on practice labs, and practice questions, which helped me assess my knowledge and identify areas that needed improvement. Some tips for candidates preparing for the exam: Focus on understanding concepts rather than memorization.Gain hands-on experience with Cloudera technologies.Review the official exam objectives carefully.Take practice tests to evaluate your readiness.Spend extra time on troubleshooting and real-world scenarios. I would like to thank the Cloudera community for the valuable discussions and learning resources that contributed to my success. Best wishes to everyone currently preparing for their Cloudera certification exams. Keep learning and stay focused—you can do it!

 **Environment**- Cloudera CFM 2.1.7 / Apache NiFi 1.26.0- Cloudera CDP 7.3.1- Flow source: Cisco ASA Firewall (IPFIX export) **Issue** The `ListenNetFlow` processor successfully decodes standard IPFIX/NetFlow v9 fields but produces a WARN for every packet that contains Cisco ASA enterprise fields: ```WARN com.cloudera.nifi.netflow.ListenNetFlow: Sender [/x.x.x.x] Field Type [33002] Length [2] Field Decoder not foundWARN com.cloudera.nifi.netflow.ListenNetFlow: Sender [/x.x.x.x] Field Type [33001] Length [12] Field Decoder not foundWARN com.cloudera.nifi.netflow.ListenNetFlow: Sender [/x.x.x.x] Field Type [33000] Length [12] Field Decoder not foundWARN com.cloudera.nifi.netflow.ListenNetFlow: Sender [/x.x.x.x] Field Type [40000] Length [12] Field Decoder not found ``` At production NetFlow volume, this generates thousands of WARN lines per second, making `nifi-app.log` effectively unusable for monitoring other issues. More critically, the data in thes

"Hello everyone, I am researching the licensing model for Cloudera Private Cloud. Could someone clarify the baseline metrics for an On-Premises deployment? Is the subscription billed per Node or per CPU Core? Also, is there a minimum number of nodes required for a starting enterprise subscription? We want to budget this for our upcoming infrastructure planning."

"Hi all, I am a Data Engineer looking to test and familiarize myself with the latest Cloudera Manager and Cloudera Runtime 7.x features. Since Cloudera Express is deprecated, does Cloudera offer any Developer License, Academic License, or a low-cost Non-Production subscription specifically for sandbox/R&D environments? Thank you!"

"Hi Community, our team is planning to evaluate CDP Private Cloud Base for our internal big data modernization project. We want to set up a small 3-node cluster for a Proof of Concept (PoC) on our local infrastructure. Since the repository is now password-protected, how can we officially request a temporary trial license key and repository credentials to test the installation with Cloudera Manager? Any guidance would be highly appreciated."

The CM version is 7.13.1. Airflow is not available as an option to "Add Service". Is it possible to use Cloudera's Airflow with 7.1.9sp1?If yes, where  can I download the CSD for Airflow?If there is no CSD, how do I install Cloudera's Airflow as a stand-alone service?Thank you.  

I'm copying several directories with several files eachYarn application full termination seems affected by the number of object transferredIn this case we have 9 minutes between 100% completion and job termination26/06/05 10:47:20 INFO mapreduce.Job: map 100% reduce 0%26/06/05 10:56:25 INFO mapreduce.Job: Job job_1780210407885_1954 completed successfullywithFiles Copied=68744DIR_COPY=23058while here few seconds26/06/05 10:43:11 INFO mapreduce.Job: map 100% reduce 0%26/06/05 10:43:32 INFO mapreduce.Job: Job job_1780210407885_1958 completed successfullywithFiles Copied=12300DIR_COPY=191 Is the any explanation for such a behaviour?

Hello it seems templates which existed in NiFi 1 is removed in NiFi 2.8.0How to import the flow definition to the new instance? Earlier we were able to move nif flow to new instances of NiFi using templates.Any screenshot will be helpful for NiFi 2. Currently I can only download flow definition by right clicking a process group. Some article suggested Registry. Do we have any other approach like nifi 1 where we used to browse and upload.

Hello we have single user based NiFi but in reality around 15 to 20 people use that instance. How to know who added a process group or stopped a processor and at which point of time was that processor stopped? Do we have any IP getting recorded or anything like that? Is it possible in NiFi having OIDC/Oauth2 based authentication that which user added/removed something?

I just finished installing a new cluster but failed to start the zookeeper service. Each instance started but had the "bad health" flag. The zookeeper log on node 3 showed this error repeatedly. ++++Cannot open channel to 1 at election address node1/61.62.63.1:4181java.net.ConnectException: Connection refused++++On node 3, the connection to node 1 zookeeper port showed no issue.$ nc -zv node1 4181Ncat: Version 7.92 ( https://nmap.org/ncat )Ncat: Connected to 61.62.63.1:4181.Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.On node1, the zookeeper port is open too$ ss -tulnp | grep 4181tcp LISTEN 0 50 61.62.63.1:4181 0.0.0.0:*The private cloud base cluster is 7.1.9 sp1.Thank you.Regards,

Hello Cloudera Community,During the same Test cluster upgrade from Cloudera Runtime 7.1.9 to 7.3.2.0 GA (before Prod rollout), we encountered a persistent issue with Ranger UserSync, which we were not able to resolve using supported configuration or migration steps.EnvironmentOriginal Runtime: 7.1.9Upgraded Runtime: 7.3.2.0 (GA)Parcel: CDH-7.3.2-1.cdh7.3.2.p0.77083870Cloudera Manager: 7.11.3OS: RHEL / Oracle Linux 8.7Authentication: LDAP / KerberosUpgrade type: Test cluster before ProdAfter upgrade to 7.3.2.0, Ranger UserSync and RMS does not start, while Ranger Admin and Ranger Tagsync starts and works normally.UserSync fails immediately with error:Error: Could not find or load main classorg.apache.ranger.authentication.UnixAuthenticationServiceCaused by: java.lang.ClassNotFoundExceptionWe tried all supported and commonly recommended remediation steps:Deleting the existing Ranger UserSync role (originally created on 7.1.9);Re‑creating a new Ranger UserSync role;Configuring LDAP‑b

Hello Cloudera Community, We upgraded a Test cluster from Cloudera Runtime 7.1.9 to 7.3.2.0 GA (before Prod rollout) and encountered an issue where Apache Atlas fails to start after the upgrade.EnvironmentOriginal Runtime: 7.1.9Upgraded Runtime: 7.3.2.0 (GA)Parcel: CDH-7.3.2-1.cdh7.3.2.p0.77083870Cloudera Manager: 7.11.3OS: RHEL / Oracle Linux 8.7Cluster Java: OpenJDK 17 (required by 7.3.2)HDFS metadata upgrade: completedAfter upgrade, Atlas fails with:Unrecognized VM option 'CMSClassUnloadingEnabled' Error: Could not create the Java Virtual MachineThe generated JVM command line contains CMS GC flags: -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled.  These flags are not supported on Java ≥ 9.Atlas in 7.3.2.0 GA is compiled with Java 17, so running Atlas on Java 8 or 11 is also not possible: UnsupportedClassVersionError (class file version 61.0).Looks like CMS GC flags are hard‑coded in the parcel . We attempted multiple supported CM‑level overrides,

I'm exporting several tables and I observe that some files are copied to the target path using DistCp (slow) while other files with some other (fast) meanThere is no evidence of the rational behind the choice but the other odd thing is that even if a table is made up of multiple files, hive starts 1 DistCp for each file instead of passing the entire directoryIs there any option to drive the behaviour?

Hello I need to invoke a url which is protected by oauth2. I know to invoke simple apis but not with oauth2. How can we make a flow?

Hi everyone,I’m facing an issue while initializing a compute cluster on an existing CDP Public Cloud environment (AWS) using restricted IAM policies.The operation fails with the following error:IAM Restricted Resource Policy validation cannot be completed on AWS:secret encryption is enabled but the secret encryption KMS key is not provided.Liftie does not have the permission to create the KMS key.Please provide a valid Customer Managed Key for secret encryption In the other experience (CDE,CDF) I bypassed this error by adding skip validation. Unfortunately, skip validation is not possible when activating the compute cluster. I've modified also the cross account role by adding the action as mentioned on paragraph "For Let CDP generate CMK" https://docs.cloudera.com/dataflow/cloud/aws-requirements/cdf-aws-requirements.pdf but nothing to do. Do you have any suggestion? Thanks

Hello, everytime I make a new version of or copy 4-5 big process groups, the Nifi UI start throwing 503 errors and sometimes It dies for 15-30 minutes. I tried changing the Garbage Collector algorythm but it did not work out, and I know that adding up memory heap will be useless. What else can I do? Thank you