Member since
05-17-2016
190
Posts
46
Kudos Received
11
Solutions
01-09-2020
07:45 AM
Hi Matt, The case was bit different than in the screenshot. This was a multi node cluster and instead of "localhost" @VijaySankar had one of the hostnames configured in the hostname field. The processor was however configured to run on all nodes. This was causing the Error messages. Cleared off the hostname field so that the processor is able to spin up a HTTP service on each host:port and the error doesn't occur anymore.
... View more
11-09-2018
04:22 AM
Tested against HDF Version 3.1.0
... View more
11-09-2018
04:11 AM
2 Kudos
Hi, In this article, let us take a look at how to delete a schema from the Hortonworks Schema Registry. Let me start with a word of caution that the approach is not recommended for Production systems and use these steps at your own risk. Also, would like to thank Brian Goerlitz for his ideas towards this post. Currently it is not possible to delete a schema from the UI. So the steps below shows how to delete the schema from its backend datastore. I am using MySQL as my backend datastore for the schema registry and the queries will be related to MySQL. You should change them according to your database type. Step 1 Verify that the two tables schema_version_info and schema_field_info have CASCADE ON UPDATE and CASCADE ON DELETE enabled. This can be done by the below queries on information_schema database select UPDATE_RULE,DELETE_RULE,REFERENCED_TABLE_NAME from REFERENTIAL_CONSTRAINTS where table_name='schema_version_info'; and select UPDATE_RULE,DELETE_RULE,REFERENCED_TABLE_NAME from REFERENTIAL_CONSTRAINTS where table_name='schema_field_info'; Step 2 Stop Schema Registry Service from Ambari Step 3 Backup the database Below is the content of my schema registry before the delete operation and I am interested in deleting the person.demographic.details schema Step 4 Identify the id of the schema to be deleted. For this, you need to switch to the database provisioned to store the schema registry information. In my case it is 'registry' and issue the select query. select id from schema_metadata_info where name ='person.demographic.details'; Step 5 Delete the schema from schema_serdes_mapping based on the id we queried in step 4 above delete from schema_serdes_mapping where schemaMetadataId=1; Step 6 Delete the schema from schema_metadata_info based on the id we queried in step 4 above delete from schema_metadata_info where id =1; We observe that the schema has been deleted from the tables. Step 7 Start the schema registry service via Ambari, and verify that the schema is deleted. Optionally we can recreate the schema with the same name on the UI and explore the front-end and back-end to ensure the schema can be re-created with no issues. We observe that the new schema was created with the same name and a different id. Thanks -Arun A K-
... View more
Labels:
02-16-2018
05:48 PM
3 Kudos
In this article, we will have a walk through of integrating LDAP with NiFi Registry. The precondition for LDAP to work with NiFi-Registry is that SSL need to be enabled. This article also covers the "How To" enable SSL for NiFI Registry. For the sake of simplicity, I am using self signed certificates (JKS, using keytool). Steps for creating self signed certificates are as below :- Generate KeyStore keytool -genkey -keyalg RSA -validity 3650 -alias <alias_name> -keypass <pwd> -storepass <pwd> -dname "cn=hostname, ou=home, o=ak, c=us" -keystore nifi_reg_keystore.jks Export a certificate with the public key keytool -export -alias <alias_name> -file nifi_reg.cer -storepass <pwd> -keystore nifi_reg_keystore.jks Generate TrustStore keytool -import -noprompt -alias nr-c0 -file nifi_reg.cer -storepass changeitchangeit -keystore nifi_reg_truststore.jks
Below, is a representation of the NiFi Registry UI with default http and anonymous user login. Now we will login to Ambari and use the above generated certificate details to complete setting up SSL. On the configuration tab, search for the SSL settings. The section to be edited is as below:- Populate the form with the details of the truststore and keystore that we generated above. At this stage, SSL setup for NiFi Registry is complete, however we have't assigned any users to login to the UI. Either we generate a certificate for an Initial Admin, or create an initial admin from the LDAP user base. We will use an LDAP user as the Initial Admin for the NiFi Registry. The configuration information for the same is as below, There are 4 sections that need to be edited, Configure Initial Admin Configure Security Identity Provider (nifi.registry.security.identity.provider) Configure login-identity-providers.xml
Remove the 2 lines that say
To enable the ldap-identity-provider remove 2 lines. This is 1 of 2. To enable the ldap-identity-provider remove 2 lines. This is 2 of 2. Fill details specific to your LDAP server. I am using a SIMPLE auth strategy with a non ssl LDAP server. Below are relevant sections from my configuration window
Configure authorizers.xml
Remove the 2 lines that say
To enable the ldap-user-group-provider remove 2 lines. This is 1 of 2. To enable the ldap-user-group-provider remove 2 lines. This is 2 of 2. Configure ldap-user-group-provider Configure accessPolicyProvider The below screenshot shows the relevant section for configuring the ldap-user-group-provider Screenshot below shows the configuration changes needed to the access policy provider. Set User Group Provider to ldap-user-group-provider. At this stage, we are in a position to save all the configuration changes and restart the NiFI Registry services. Follow the Ambari prompts and you should see as below Now we should be good to access the NiFi Registry UI and login as the Initial Admin that is configured. guest1 in my example. Access NiFi Registry UI from Quick Links Login Using the Initial Admin Credentials Verify login is successful Verify users are available/sync-ed You should be able to proceed using the NiFi registry from here on.
... View more
Labels:
11-28-2017
10:31 PM
@Matt Clarke : question on the /resources policy - The server running Ranger should be granted “read” privileges to this resource. How do we accomplish this? Is SSL for Ranger mandatory in this case?
... View more