@rupeshh Docker container are never fun for this and other reasons.   I still think you have missing permissions on the file.  I cannot see the ownership of the file listing, and i cannot see the path in the error.  At any rate, the error definitely suggests the processor does not see the file.      One suggestion would be to use the nifi user and cli on docker to ls the directory and files.  If that user cannot see the files,  that would indicate the same issue the error states (directory path or file does not exist, or not seen due to permissions).   ... View more

@wert_1311 Your error indicates that two of your roles are missing or incomplete.   1. Data Access Role (arn:aws:iam::8859X2XX911XX:role/Cloudera-datalake-admin-role) is not set up correctly. Please follow the official documentation on required policies for Data Access Role. Missing policies (chunked): arn:aws:iam::8859X2XX911XX:role/Cloudera-datalake-admin-role:s3:AbortMultipartUpload:arn:aws:s3:::cdp-my-bucket/hive_replica_functions_dir/* 2. Data Access Role (arn:aws:iam::8859X2XX911XX:role/Cloudera-ranger-audit-role) is not set up correctly. Please follow the official documentation on required policies for Data Access Role. Missing policies (chunked): arn:aws:iam::8859X2XX911XX:role/Cloudera-ranger-audit-role:s3:PutObject:arn:aws:s3:::cdp-my-bucket/ranger/audit/* Go back to the quickstart and docs, and make sure completd all the setups.  Here is a link with more about the credentials   https://docs.cloudera.com/cdp-public-cloud/cloud/requirements-aws/topics/mc-aws-req-credential.html   There are steps in the page above describing the 2 roles which have conflicts in your error.  The error indicates, in the end of each message, which policies are missing.     ... View more

@Vasu_   Can you provide screen shots of config for the processor(s) and the SSL Context Service you created? I can suggest more specific commands to build the keystore and truststore, but I will need to see what the hostname is for elastic.   Additionally, if you did attempt to create a keystore/truststore from the self signed cert,  be sure to share the commands you used.   Commands/Code would go in a Preformatted box (top right in the full wysiwig pannel) like this ... View more

@Vasu_  I am most definitely an expert in NIFI SSL Context Services and the various different ways to build the controller services with cacerts, public certs, and self signed certs.   Here is an article i just wrote about Modern NiFi and SSL: https://community.cloudera.com/t5/Community-Articles/NIFI-SSL-in-Modern-Versions-of-NiFi/ta-p/371937   It is important to understand how to make a working SSL Context Service before trying to make custom ones.  So the examples in here to use nifi's own cert's keystore and truststore, and local java cacerts as keystore/trustore, will build confidence in how to configure the controller service before trying to build custom keystores and truststores yourself.   The following link has a technical example, and both solutions you can use here.   Use java cacerts if your elastic endpoint is a public signed ssl cert to build a SSL Context Service Build you own custom keystore and truststore using actual cert from the endpoint and use that to build a SSL Context Service https://community.cloudera.com/t5/Support-Questions/Configure-StandardSSLContextService-for-Elasticsearch/m-p/302719   ... View more

@SandyClouds You can make nifi run as a service, then add that to startup services.  This will ensure that nifi is always running after system reboots.      https://nifi.apache.org/docs/nifi-docs/html/getting-started.html#installing-as-a-service Installing as a Service Currently, installing NiFi as a service is supported only for Linux and macOS users. To install the application as a service, navigate to the installation directory in a Terminal window and execute the command bin/nifi.sh install to install the service with the default name nifi. To specify a custom name for the service, execute the command with an optional second argument that is the name of the service. For example, to install NiFi as a service with the name dataflow, use the command bin/nifi.sh install dataflow. Once installed, the service can be started and stopped using the appropriate commands, such as sudo service nifi start and sudo service nifi stop. Additionally, the running status can be checked via sudo service nifi status.   Stopping or starting flows or other actions against flows would need to be done with command line or nifi rest api.     ... View more

@drewski7   You are looking for something like this,  for "yesterday":   ${now():minus(86400000):format('MM-dd-yyyy hh:mm:ss') } Then change 24 hours to 10 hours and add not reduce:   ${now():plus(36000000):format('MM-dd-yyyy hh:mm:ss') }   Let me know if this adapts to fit your use case! ... View more