Hi, let me advertise an option for git clone using git+ssh. I have put together a Docker image which supports git cloning using git+ssh out-of-the box. Give it a try and let me know how it works. Usage is as follows: docker run --name nifi-registry \ -p 18080:18080 \ -v ~/.ssh:/home/nifi/.ssh \ -e 'FLOW_PROVIDER=git' \ -e 'GIT_REMOTE_URL=git@github.com:michalklempa/docker-nifi-registry-example-flow.git' \ -e 'GIT_CHECKOUT_BRANCH=example' \ -e 'FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git' \ -e 'FLOW_PROVIDER_GIT_REMOTE_TO_PUSH=origin' \ -e 'GIT_CONFIG_USER_NAME=Michal Klempa' \ -e 'GIT_CONFIG_USER_EMAIL=michal.klempa@gmail.com' \ -d \ michalklempa/nifi-registry:latest or using ~/.ssh as a bind mount point: docker run --name nifi-registry \ -p 18080:18080 \ -e 'FLOW_PROVIDER=git' \ -e 'GIT_REMOTE_URL=git@github.com:michalklempa/docker-nifi-registry-example-flow.git' \ -e 'GIT_CHECKOUT_BRANCH=example' \ -e 'FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git' \ -e 'FLOW_PROVIDER_GIT_REMOTE_TO_PUSH=origin' \ -e 'GIT_CONFIG_USER_NAME=Michal Klempa' \ -e 'GIT_CONFIG_USER_EMAIL=michal.klempa@gmail.com' \ -e 'SSH_PRIVATE_KEY='$(base64 -w 0 < ~/.ssh/id_rsa) \ -e 'SSH_KNOWN_HOSTS='$(base64 -w 0 < ~/.ssh/known_hosts) \ -e 'SSH_PRIVATE_KEY_PASSPHRASE=' \ -d \ michalklempa/nifi-registry:latest Using HTTPS is also supported: docker run --name nifi-registry \ -p 18080:18080 \ -e 'FLOW_PROVIDER=git' \ -e 'GIT_REMOTE_URL=https://github.com/michalklempa/docker-nifi-registry-example-flow.git' \ -e 'GIT_CHECKOUT_BRANCH=example' \ -e 'FLOW_PROVIDER_GIT_FLOW_STORAGE_DIRECTORY=/opt/nifi-registry/flow-storage-git' \ -e 'FLOW_PROVIDER_GIT_REMOTE_TO_PUSH=origin' \ -e 'FLOW_PROVIDER_GIT_REMOTE_ACCESS_USER=michalklempa' \ -e 'FLOW_PROVIDER_GIT_REMOTE_ACCESS_PASSWORD=thisisnotmypassword:)' \ -e 'GIT_CONFIG_USER_NAME=Michal Klempa' \ -e 'GIT_CONFIG_USER_EMAIL=michalklempa@gmail.com' \ -d \ michalklempa/nifi-registry:latest Full documentation available at github: https://github.com/michalklempa/docker-nifi-registry/#git-cloning-the-repository-at-startup and docker image on dockerhub: https://hub.docker.com/r/michalklempa/nifi-registry ... View more

To me, this seems to be a bug. I tried to report it against Ambari: https://issues.apache.org/jira/browse/AMBARI-22839 Maybe it is related to this fix? https://issues.apache.org/jira/browse/AMBARI-22086 ... View more

I have been using InferAvroSchema in dataflows for a while and: 1. It infers the schema for each file on input 2. saves the schema into ${inferred.avro.schema} attribute for that flowfile 3. it is not good for production use As schema inferrence is only a guess, I would recommend you to infer your schema once (double check manually for correctness) and then use it as a static schema in ConvertAvroTo... processors (prepend RouteOnAttribute if you need different schemas). In production, this is what you want. Sometimes, the data can be misleading for inferrence. For example, I have input CSV with empty column, which in fact is nullable long column. Schema inferrence cannot guess it is nullable long. So for one input file, where the values are filled in as numbers, it guesses long type, and for another, where the column is empty, it guesses nullable string... ... View more

Just a side comment: cacerts ("the default") truststore shipped with JRE does not always contain all certificates needed. I have run into issue, when using OS default CA certificates handling, the webpage was using valid certificate, but Java was considering the certification path incomplete. I am using Ubuntu and to mitigate this issue, one can import all certificates from ca-certificates package of Ubuntu into Java truststore to be used with NiFi. To import all ca-certificates from Ubuntu to your truststore, you can use openssl pkcs12 export tool: openssl pkcs12 -export -nokeys -in /etc/ssl/certs/ca-certificates.crt -out /etc/nifi/truststore.p12 where /etc/nini/truststore.p12 is the truststore to be set in SSLContextService. Remember to change also the type of keystore to pkcs12 (not JKS). If you are unlucky, like I was, you may run into issue where JRE is unable to parse PKCS12 generated by openssl (openjdk has this problem with IBM generated file https://bugzilla.redhat.com/show_bug.cgi?id=961069, it seems like Java implementation of PKCS12 is 'we had to do it, but we don't mind, use JKS). Then, one can import all /etc/ssl/certs/*.pem files into JKS truststore by using keytool from JDK distribution (this is bash code): for file in `ls /etc/ssl/certs/*.pem`; do keytool -noprompt -importcert -keystore /etc/nifi/truststore.jks -storepass changeit -file $file -alias $file; done Now we have JKS type keystore which can be read by Java (it was written by Java so we at least hope so Java can read it). Just set this truststore in SSLContextService and you have all certs which Ubuntu has provided to you as trusted. As a verification that import worked, one can compare count of *.pem files to count of certificates in truststore: ls -1 /etc/ssl/certs/*.pem | wc keytool -storepass changeit -list -keystore /etc/nifi/truststore.jks | grep finge | wc Number of lines should be equal. ... View more