I know this is old topic, but do you have solution applied on this so it can be shared with the community @phr1 ?  ... View more

Hello @hus,  Directly the ExecuteSQL processor does not support binding.  The input should be directly SQL query.  https://nifi.apache.org/components/org.apache.nifi.processors.standard.ExecuteSQL/ But, according to the API docs, the processor can be started via a FlowFile:  https://javadoc.io/static/org.apache.nifi/nifi-standard-processors/2.4.0/org/apache/nifi/processors/standard/ExecuteSQL.html  So technically you should be able to start this query by using ReplaceText to inject values dynamically, for example.  ... View more

Hello @jI-mi,  Thanks for being part of our community.  Can you please explain your question a little bit more?  Also, can you please tell us your CML version?  I was quickly checking on CML 1.5.5 and the CUDA 12.8 is not listed as supported, so this may also give you some hints:  https://docs.cloudera.com/machine-learning/cloud/runtimes/topics/ml-runtimes-nvidia-gpu.html  ... View more

Hello @SalimAlhajri,  The Transparent Authentication is built-in Cloudera AI.  When the application is started, it will inject the REMOTE-USER and REMOTE-USER-PERM HTTP headers automatically, this is why it is transparent, no manual intervention is needed.  https://docs.cloudera.com/machine-learning/1.5.5/applications/topics/ml-securing-applications.html  ... View more

Hello @zzzz77,  Maybe this blog can help you: https://community.cloudera.com/t5/Community-Articles/Understanding-how-NiFi-s-Content-Repository-Archiving-works/ta-p/249418  There explains how to handle the repository archive and it could work for what you need.  Also, there are other options like the Reporting Tasks documented here: https://nifi.apache.org/docs/nifi-docs/  SiteToSiteProvenanceReportingTask is an option for your need.  ... View more

Hello @NadirHamburg  Thanks for being part of our Community.  I'm not an expert on Clickhouse, but was reading that it could be something on the DB causing the batches to repeat and causing that amount of duplicated records.  From NiFi side, you can try to set the batch size at the same amount of records, this should work for you. But I know that for big databases it could be a problem.  From Clickhouse, I found this documentation:  https://clickhouse.com/docs/engines/table-engines/mergetree-family  There talks about ReplicatedMergeTree, which should be a good option to avoid duplicates.  Do you have your table with those settings? Do you see any errors on PutDatabaseRecord log? If so, can you share them?  ... View more

Hello @garb,  Thanks for being part of our community.  I was reviewing the information and even though Calcite is the engine used for the SQL, looks like not all the queries are supported officially.  Looking in several places, I do not see LPAD used anywhere.  But something that may work for what you need is CONCAT, which should give you the correct format properly and is broadly used in the community:  SELECT MsgSeqNbr, PostTime, SSN, EmployeeID, LName, FName, MName, CAST(TRIM(LCN) AS BIGINT) AS LCN, RIGHT(CONCAT('00000', PIN), 5) AS PIN, EmployeeType, ValidityCode, AgencyOwner, AgencyLocated, BadgeCreatedBy, BadgeCreatedTime, BadgeModifiedBy, BadgeModifiedTime, Clearance, Error, Status FROM FLOWFILE Based on the error "No match found for function signature LPAD" it looks like the engine configured for NiFi does not support LPAD even when Calcite do support it.  I was trying to find on the code the supported functions, but did not find LPAD.  This looks to be the most accurate reference we have where we do see CONCAT: https://github.com/apache/nifi/blob/main/nifi-docs/src/main/asciidoc/record-path-guide.adoc  ... View more

Hello @backtohome,  So far I know, we do support GPUs for Spark workloads on CML.  The documentation talks about that:  Autoscaling: Cloudera AI also supports native cloud autoscaling via Kubernetes. When clusters do not have the required capacity to run workloads, they can automatically scale up additional nodes. Administrators can configure auto-scaling upper limits, which determine how large a compute cluster can grow. Since compute costs increase as cluster size increases, having a way to configure upper limits gives administrators a method to stay within a budget. Autoscaling policies can also account for heterogeneous node types such as GPU nodes. https://docs.cloudera.com/machine-learning/1.5.5/spark/topics/ml-apache-spark-overview.html  You have to configure them by following this doc:  https://docs.cloudera.com/machine-learning/1.5.5/gpu/topics/ml-gpu.html  If you do not have the GPUs configured on CML, the UI will not show you the options, such like this:    ... View more

Hello @zzzz77,  Glad to have you on the community.  What you are asking should be done with this kind of flow:  GetFile → SplitContent → Transfer → MergeContent → PutFile The SplitContent will split the file and the attributes will be get duplicated, because they are saved on the FlowFile, not on the content.  More attributes will be added for the fragmentation part.  The MergeContent will rebuild the content and the original attributes properly.  So the metadata will not be lost.  ... View more