@pnac03 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks. ... View more

Yes, you're right, that is public cloud.  I do not know why that environment cannot be chosen.  What I think is that some prerequisites or compatibility issues.  Maybe you can check here:  https://docs.cloudera.com/data-engineering/cloud/release-notes/topics/cde-dl-compatibility.html  ... View more

Hello @PeterKa,  Thanks for reaching our Community.  For this, JOLT always need to have the values, it's not very much flexible if an expected value is missing.  But, something that JOLT offers is the default operation, that may help you to add "{"data": if it is already included.  You can see it here: https://lucabiscotti.github.io/jolt-guide.github.io/#default  That option may help you to have the needed format always before the shift.  ... View more

Hello @enguias,  Sometimes this error could be for an old algorithm such as SHA-1. And most possible if using Java 11 that is more strict with the policies.  One option is to use connection without TLS, if possible and acceptable in your environment:  jdbc:sqlserver://<host>:<port>;databaseName=<DB>;encrypt=disable Or, update the certificate to SHA-256 with TLS 1.2.  This is the best option if you prefer to maintain the encryption.  Maybe this documentation from SQL can help too:  https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/configure-sql-server-encryption?view=sql-server-ver17  ... View more

Hello @yoonli,  Thanks for contacting our Cloudera Community and sharing your question.  Something I have to mention is that Cloudera does not support standalone Spark cluster, we only work with YARN clusters.  Anyway, taking a quick look on this issue, I see that you're not mentioning any principal or keytab, when you use Kerberos, it will always try to run kinit, but if there are no principal and keytab, it will fail.  Something you can try is using the Simple Auth method, you can add these two settings on your spark-defaults.conf:  spark.hadoop.fs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider spark.hadoop.security.authentication=simple hadoop.security.authentication=simple https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html  ... View more

Hello @donaldo71,  This looks like SQL is getting the deadlock because of the many records in once.  You can try couple of this.  First, enable the retry option on the PutSql or PutdatabaseRacord processors. If the retries helped previously, this could also help in your case. Also, decrease the concurrency and batch sizes to try to decrease the SQL load.  Additionally, on the SQL side, if you can, use row versioning isolation to reduce locking:  ALTER DATABASE DBNAME SET READ_COMMITTED_SNAPSHOT ON; ... View more

Understood.  Hopefully the missing NAR's pointed on the previous update help you figure the issue.  ... View more

Hello @AlokKumar,  Thanks for using Cloudera Community.  As I understand, what you need is to add one more step in your flow:  HandleHttpRequest-> MergeContent -> ExecuteScript (Groovy)-> HandleHttpResponse Since you have JSON fields and files, you're getting multiple FlowFiles. So this extra MergeContent phase will combine the JSON and the file into a single FlowFile  On the MergeContent, set Merge Strategy as “Defragment” and set Correlation Attribute Name as http.request.id. that is unique from each HandleHttpRequest ... View more

Hello @Brenda99,  The question is very wide, there are many things that can help to improve the performance.  Some basic recomendations are documented here:  https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/tuning-spark/topics/spark-admin_spark_tuning.html  Take a look on the documentation, that could help you.  Also, it will worth to talk with the team in charge of your account to found deeper performance tuning analysis.  ... View more

I guess, that my problem has not solution from NiFi side and we just need to correct HDFS settings to accept other encryption types in addition to arcfour-hmac-md5. ... View more