I solved my problem. In my case one of the talbes' name was starting with a character underscore "_" because of which there was an issue where 2 single quotes were added automatically in the path of the hdfs directory where the copy of the file was stored. I changed the name of the column by removing the underscore character and now i can import the table into Hive database. I think special characters like that are not easily parsed in Hive or HDFS. ... View more

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect. Check out this way to do SNS up to 2gb per payload:   https://github.com/awslabs/amazon-sns-java-extended-client-lib     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

I suspect you have not completed a step, or missing something.  The cacerts works for me in all cases if the cert is publicly trusted (standard public cert from public CA) which it should be.    You should share info on the configurations you tried and what if any errors you got from that.  The bare minimum settings you need for that are keystore (file location), password, key type (jks), and TLS version.   Assuming you copied your java cacert file to all nodes as /nifi/ssl/cacerts the controller service properties should look like:       If cacerts doesnt work, then you must create keystores and/or trust stores with the public cert.   Use the openssl command to get the cert.  That command looks like:   openssl s_client -connect https://secure.domain.com   You can also get it from the browser when you visit the elk interface; for example cluster health, or indexes.   Double click cert lock icon in the browser then use the browser's interface to see/view/download public certificate.  You need the .cer or .crt file.   Then you use the cert to create the keystore with keytool commands.  An example is:   keytool -import -trustcacerts -alias ambari -file cert.cer -keystore keystore.jks   Once you have created a keystore/truststore file you need to copy it to all nifi nodes, ensure the correct ownership, and make sure all the details are correct in the SSL Context Service.  Lastly you may need to modify the TLS type until testing works.   Here is working example of getting the cert and using it with keytool from a recent use case:   echo -n|openssl s_client -connect https://secure.domain.com | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > publiccert.crt keytool -import -file publiccert.crt -alias astra -keystore keyStore.jks -storepass password -noprompt  keytool -import -file publiccert.crt -alias astra -keystore trustStore.jks -storepass password -noprompt  mkdir -p /etc/nifi/ssl/ cp *.jks /etc/nifi/ssl chown -R nifi:nifi /etc/nifi/ssl/     ... View more

I think the most straight forward would be to drop the infer schema into your version of NiFi.   The procedure is not that hard, you just have to be surgically careful.  The process is explained a bit here in reference to adding parquet jars from new version, into older version.   Be sure to read all the comments:   https://community.cloudera.com/t5/Support-Questions/Can-I-put-the-NiFi-1-10-Parquet-Record-Reader-in-NiFi-1-9/td-p/286465   ... View more