Getting below error while Start making an entry for Version Control in NiFi Registry: Version Details: HDF - 3.3.0 NiFi - 1.8.0 NiFi Registry - 0.3.0 NiFi Console Log: Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Contact the system administrator. NiFi Registry Log: 2019-06-19 06:04:03,248 INFO [NiFi Registry Web Server-18] o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry. Returning Conflict response. 2019-06-19 06:04:03,620 INFO [NiFi Registry Web Server-17] o.a.n.r.w.s.NiFiRegistrySecurityConfig Client could not be authenticated due to: org.springframework.security.authentication.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext Returning 401 response. 2019-06-19 06:22:49,013 INFO [NiFi Registry Web Server-63] o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry. Returning Conflict response. 2019-06-19 06:22:49,263 INFO [NiFi Registry Web Server-17] o.a.n.r.w.s.NiFiRegistrySecurityConfig Client could not be authenticated due to: org.springframework.security.authentication.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext Returning 401 response. 2019-06-19 06:27:01,720 INFO [NiFi Registry Web Server-68] o.a.n.r.w.s.NiFiRegistrySecurityConfig Identity in proxy chain not trusted to act as a proxy: org.apache.nifi.registry.web.security.authentication.exception.UntrustedProxyException: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Returning 403 response. 2019-06-19 06:28:10,402 INFO [NiFi Registry Web Server-68] o.a.n.r.w.s.NiFiRegistrySecurityConfig Identity in proxy chain not trusted to act as a proxy: org.apache.nifi.registry.web.security.authentication.exception.UntrustedProxyException: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Returning 403 response. Configuration: SSL - For both NiFi and NiFi Registry, we are using a host-specific self-signed certificate. And both services refer to the same Java Keystores. Authentication Used - LDAP Kerberos - No Ranger Plugin to NiFi - Yes NiFi Service: Authorization - Ranger Initial Admin Identity - admin nifi.security.user.login.identity.provider - ldap-provider Node Identities <property name="Node Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> <property name="Node Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> NiFi Registry Service: Initial Admin Identity CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com nifi.registry.security.identity.provider=ldap-identity-provider NiFi Identities <property name="NiFi Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> <property name="NiFi Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> authorizers.xml <userGroupProvider>     <identifier>file-user-group-provider</identifier>     <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>     <property name="Users File">{{nifi_registry_internal_config_dir}}/users.xml</property>     <property name="Initial User Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>     <property name="Initial User Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>     <property name="Initial User Identity 3">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>     {{nifi_registry_ssl_config_content | replace("NiFi","Initial User")}} </userGroupProvider> <accessPolicyProvider>     <identifier>file-access-policy-provider</identifier>     <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>     <property name="User Group Provider">file-user-group-provider</property>     <property name="Authorizations File">{{nifi_registry_internal_config_dir}}/authorizations.xml</property>     <property name="Initial Admin Identity">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>     {{nifi_registry_ssl_config_content}} </accessPolicyProvider> Note: I am able to login to both the services using LDAP user credentials. Also, created Ranger Policy: /proxy - Read & Write - CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US Read & Write - CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US Kindly help me to resolve this issue. ... View more

We need to take backup all the topics in Kafka to the file named in respective topic names and need to restore the topic as per user requirement. Note: This script needs to be run in the Kerberized environment. kafkabackup.sh Making required directories monyear=`date | awk '{print $2$6}'` dat=`date| awk '{print $2$3$6}'` export BACKUPDIR=/root/backup/$monyear mkdir -p $BACKUPDIR mkdir -p $BACKUPDIR/$dat cd $BACKUPSDIR BKDIR=$BACKUPDIR/$dat Log into Kafka Get topics from Kafka Broker kinit -kt /etc/security/keytabs/kafka.service.keytab kafka/node1.localdomaino@domain.co cd /usr/hdp/current/kafka-broker/bin/ export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_client_jaas.conf" ./kafka-topics.sh --zookeeper adminnode.localdomain:2181 --list > $BKDIR/listtopics.txt Remove if any mark of deletion topics exists sed -i.bak '/deletion/d' $BKDIR/listtopics.txt Starting kill script in parallel bash checkandkill.sh& Reading the file contents for topics for line in $(cat $BKDIR/listtopics.txt) do echo $line ./test.sh --bootstrap-server node1.localdomain:6668 --topic $line --consumer.config /home/kafka/conf.properties --from-beginning --security-protocol SASL_SSL > $BKDIR/$line done Delete empty files /usr/bin/find . -size 0 -delete Killing checkandkill daemon and exit ps -ef |grep -i checkandkill.sh| grep -v grep | awk '{print $2}' | xargs kill exit When consumer runs, it constantly waits for messages to receive. We need to kill the process. checkandkill.sh sleep 0.5m for line in $(cat /root/backup/listtopics.txt) do echo $line sleep 1m ps -ef |grep -i $line| grep -v grep | awk '{print $2}' | xargs kill done Need your help to complete restoration script. ... View more