@pavanshettyg5  The TLS implementation for NiFi requires proper configuration of both keystore and truststore your organization's signed certificate with SAN entries to enable secure HTTPS access. Based on your description, you've received a signed certificate (nifi.crt) but are experiencing issues with the complete TLS setup.  Required Certificates & Files Certificate Authority (CA) Certificate: The root certificate from your signing authority Truststore: Contains the Certificate Authority (CA) root/intermediate certificates that signed your NiFi certificate (for mutual TLS or cluster communication). Signed Certificate (nifi.crt): Your domain certificate with the SAN entries Private Key: The private key used to generate the CSR 1. Prepare Certificate Files Ensure you have: The signed certificate (nifi.crt) Your private key The CA certificate (request from your CA if not available) Problem 1: Missing Private Key or Certificate Chain If you only have nifi.crt, you must also have: The private key (e.g. nifi.key) generated during the CSR process. The CA root/intermediate certificates (if your organization uses a private CA). Problem 2: Improper Keystore/Truststore Format NiFi uses Java KeyStores (JKS or PKCS12). Ensure your keystore/truststore is in the correct format. If your organization uses OpenSSL-based tools, convert the PEM files (nifi.crt + nifi.key + CA chain) into a PKCS12/JKS keystore. Problem 3: SAN Entries Not Recognized Verify the SAN entries in your certificate match the NiFi node hostnames (e.g. nifinode1.x.x.net). Use openssl x509 -in nifi.crt -text -noout to check SANs. 3. Step-by-Step Solution A. Prepare the Keystore Combine Certificate and Private Key: If you have nifi.crt and nifi.key, create a PKCS12 keystore: openssl pkcs12 -export \ -in nifi.crt \ -inkey nifi.key \ -chain -CAfile ca_chain.crt \ # Include CA chain if needed -name "nifi" \ -out nifi-keystore.p12 -password pass:keystorepassword Use a password (e.g., keystorePassword). 2. Convert to JKS (if required) Java Keystore from PKCS12 keytool -importkeystore \ -srckeystore nifi-keystore.p12 \ -srcstoretype PKCS12 \ -destkeystore nifi-keystore.jks \ -deststoretype JKS B. Prepare the Truststore Import CA Certificates: If your organization uses a private CA, add its root/intermediate certificates to the truststore keytool -import -trustcacerts \ -alias ca-root \ -file ca_root.crt \ -keystore nifi-truststore.jks C. Configure NiFi Update nifi.properties # HTTPS Settings nifi.web.https.host=0.0.0.0 nifi.web.https.port=9443 nifi.web.https.network.interface.default= # Security Properties # nifi.security.keystore=/path/to/keystore.jks nifi.security.keystoreType=JKS nifi.security.keystorePasswd=keystorepassword nifi.security.keyPasswd=keystorepassword nifi.security.truststore=/path/to/truststore.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd=truststorepassword # Truststore (required for cluster nodes/ZooKeeper) nifi.security.truststore=./nifi-truststore.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd=truststorePassword # Enable TLS for cluster nodes nifi.cluster.protocol.is.secure=true nifi.web.http.port= nifi.web.https.port=9443 nifi.zookeeper.connect.string=zookeepernode1.x.x.net:2181,zookeepernode2.x.x.net:2181,zookeepernode3.x.x.net:2181 nifi.zookeeper.client.secure=true 2. Updateauthorizers.xml (for mutual TLS): Configure for Cluster Communication Configure authorizers.xml and nifi-registry.properties for secure cluster communication using the same certificates. <property name="Initial Admin Identity">CN=admin, OU=YourOrg</property> <property name="Node Identity 1">CN=nifinode1.x.x.net, OU=YourOrg</property> D. Validate the Setup After configuration Test Keystore/Truststore: # Verify keystore contents keytool -list -v -keystore keystore.jks -storepass keystorepassword # Verify truststore contents keytool -list -v -keystore truststore.jks -storepass truststorepassword # Test SSL configuration openssl s_client -connect nifi-dev.x.x.net:9443 -showcerts Troubleshooting Common Issues Certificate Chain Issues: Ensure your keystore includes the full certificate chain # Concatenate certificates if needed cat nifi.crt intermediate.crt root.crt > fullchain.crt SAN Validation: Verify certificate has correct SAN entries   openssl x509 -in nifi.crt -text -noout | grep -A1 "Subject Alternative Name" Java Compatibility: Ensure Java version compatibility with TLS   # Add to bootstrap.conf if using older Java versions java.arg.16=-Dhttps.protocols=TLSv1.2 Cluster Communication: Set proper node identities for cluster   # In nifi.properties nifi.cluster.node.address=nifinode1.x.x.net nifi.cluster.node.protocol.port=11443 nifi.remote.input.secure=true ZooKeeper Security: Only if using secure ZooKeeper connections   # In zookeeper.properties secureClientPort=2281 serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory ssl.keyStore.location=/path/to/keystore.jks ssl.keyStore.password=keystorepassword ssl.trustStore.location=/path/to/truststore.jks ssl.trustStore.password=truststorepassword Verify HTTPS Access: Access https://nifi-dev.x.x.net:9443/nifi in a browser. Use curl -vk https://nifi-dev.x.x.net:9443/nifi to debug TLS handshake errors.        2. Troubleshooting "SSL Peer Unauthenticated": Ensure the truststore contains the CA certificate that signed the NiFi certificate. "Certificate Doesn't Match Hostname": Verify SAN entries in nifi.crt include all NiFi node hostnames. Keystore Password Mismatch: Ensure nifi.security.keystorePasswd and nifi.security.keyPasswd match in nifi.properties. Additional Recommendations Use strong, unique passwords for keystores and truststores Implement proper certificate rotation procedures Consider automating certificate management with tools like cert-manager Implement client certificate authentication for additional security Ensure proper DNS resolution for all SAN entries By following the above steps, you’ll enable HTTPS for NiFi with proper SAN support and resolve keystore/truststore issues Happy hadooping ... View more

@pavanshettyg5  But looking at the error messages when running zkServer.sh status on the ZooKeeper nodes: on zookeepernode1 and 2, there's a message: "Client port not found in static config file. Looking in dynamic config file." Then a grep error: "grep: : No such file or directory". This suggests that the static zoo.cfg is missing the clientPort entry, and the dynamic configuration file (which is probably specified via dynamicConfigFile in zoo.cfg) is either not present or misconfigured. To resolve the ZooKeeper and NiFi connectivity issues, follow these steps and hopefully that will resolve your nifi connectivity issue Step 1: Configure ZooKeeper to Bind to All Interfaces Problem: ZooKeeper nodes are binding to localhost, preventing remote connections from NiFi. Fix: Update zoo.cfg on each ZooKeeper node to bind to 0..0.0.0 (all interfaces). 1. Edit zoo.cfg on each ZooKeeper node: vi /opt/zookeeper/conf/zoo.cfg 2. Add/Modify these lines clientPort=2181 clientPortAddress=0.0.0.0 3. Restart ZooKeeper on each node: /opt/zookeeper/bin/zkServer.sh restart Step 2: Verify ZooKeeper Configuration After restarting, check the status /opt/zookeeper/bin/zkServer.sh status Expected Output: Client address: 0.0.0.0(not localhost). One node should beleader, others follower. Step 3: Check ZooKeeper Network Connectivity From NiFi nodes, test connectivity to ZooKeeper: telnet zookeepernode1 2181 telnet zookeepernode2 2181 telnet zookeepernode3 2181 If connections fail, check firewalls/security groups to allow traffic on port 2181. Step 4: Validate ZooKeeper Dynamic Configuration (If Applicable) If using dynamic reconfiguration: 1. Ensure the dynamic config file (e.g., zoo_dynamic.cfg) has entries like server.1=zookeepernode1:2888:3888:participant;zookeepernode1:2181 server.2=zookeepernode2:2888:3888:participant;zookeepernode2:2181 server.3=zookeepernode3:2888:3888:participant;zookeepernode3:2181 2. Confirm the static zoo.cfg references the dynamic file: dynamicConfigFile=/opt/zookeeper/conf/zoo_dynamic.cfg Step 5: Verify NiFi Configuration Ensure nifi.properties points to the correct ZooKeeper ensemble: nifi.zookeeper.connect.string=zookeepernode1:2181,zookeepernode2:2181,zookeepernode3:2181 Step 6: Restart NiFi Services Restart NiFi on all nodes: /opt/nifi/bin/nifi.sh restart Check logs for successful connections: tail -f /opt/nifi/logs/nifi-app.log Troubleshooting Summary ZooKeeper Binding: Ensure ZooKeeper listens on 0.0.0.0:2181, not localhost. Firewall Rules: Allow traffic between NiFi and ZooKeeper nodes on ports 2181, 2888, and 3888. Hostname Resolution: Confirm zookeepernode1, zookeepernode2, and zookeepernode3 resolve to correct IPs on NiFi nodes. By addressing ZooKeeper's binding configuration and network accessibility, NiFi should successfully connect to the ZooKeeper cluster. Happy hadooping ... View more

@drewski7  The error message says there's no EntityManager with an actual transaction available. That suggests that the code trying to persist the user isn't running within a transactional context. In Spring applications, methods that modify the database usually need to be annotated with `@Transactional` to ensure they run within a transaction. Looking at the stack trace, the error occurs in `XUserMgr$ExternalUserCreator.createExternalUser`, which calls `UserMgr.createUser`, which in turn uses `BaseDao.create`. The `create` method in `BaseDao` is trying to persist an entity but there's no active transaction. So maybe the `createUser` method or the code calling it isn't properly transactional. In version 2.4.0, this worked, so something must have changed in 2.5.0. Perhaps the upgrade introduced changes in how transactions are managed. Maybe a method that was previously transactional no longer is, or the transaction boundaries have shifted. Step 1: Verify Database Schema Compatibility Ranger 2.5.0 may require schema updates. Ensure the database schema is compatible with the new version: Check Upgrade Documentation: Review the Ranger 2.5.0 Release Notes for required schema changes. Example: If migrating from 2.4.0 to 2.5.0, you may need to run SQL scripts like x_portal_user_DDL.sql or apache-ranger-2.5.0-schema-upgrade.sql. Run Schema Upgrade Scripts: Locate the schema upgrade scripts in the Ranger installation directory (ranger-admin/db/mysql/patches) and apply them: mysql -u root -p ranger < apache-ranger-2.5.0-schema-upgrade.sql Validate the Schema: Confirm that the x_portal_user table exists and has the expected columns (e.g., login_id, user_role). Step 2: Check Transaction Management Configuration The error suggests a missing @Transactional annotation or misconfigured transaction manager in Ranger 2.5.0: Review Code/Configuration Changes: Compare the transaction management configurations between Ranger 2.4.0 and 2.5.0. Key files: ranger-admin/ews/webapp/WEB-INF/classes/conf/application.properties ranger-admin/ews/webapp/WEB-INF/classes/spring-beans.xml Apache Ranger JIRA: Search for issues like RANGER-XXXX related to transaction management in Ranger 2.5.0. Ensure Transactional Annotations: In Ranger 2.5.0, the method createUser in UserMgr.java or its caller must be annotated with @Transactional to ensure database operations run in a transaction. @Transactional public void createUser(...) { ... } 3. Debug Transaction Boundaries: Enable transaction logging in log4j.properties to trace transaction activity log4j.logger.org.springframework.transaction=DEBUG log4j.logger.org.springframework.orm.jpa=DEBUG Step 3: Manually Create the User (Temporary Workaround) If the user drew.nicolette is missing from x_portal_user, manually insert it into the database: INSERT INTO x_portal_user (login_id, password, user_role, status) VALUES ('drew.nicolette', 'LDAP_USER_PASSWORD_HASH_IF_APPLICABLE', 'ROLE_USER', 1); Note: This bypasses the transaction error but is not a permanent fix. Step 4: Verify LDAP Configuration Ensure LDAP settings in ranger-admin/ews/webapp/WEB-INF/classes/conf/ranger-admin-site.xml are correct for Ranger 2.5.0: <property> <name>ranger.authentication.method</name> <value>LDAP</value> </property> <property> <name>ranger.ldap.url</name> <value>ldap://your-ldap-server:389</value> </property> Step 5: Check for Known Issues Apache Ranger JIRA: Search for issues like RANGER-XXXX related to transaction management in Ranger 2.5.0. 2. Apply Patches: If a patch exists (e.g., for missing @Transactional annotations), apply it to the Ranger 2.5.0 codebase. Step 6: Test with a New User Attempt to log in with a different LDAP user to see if the issue is specific to drew.nicolette or a systemic problem. If the error persists for all users, focus on transaction configuration or schema issues. If only drew.nicolette fails, check for conflicts in the x_portal_user table (e.g., duplicate entries). Final Checks Logs: Monitor ranger-admin.log and catalina.out for transaction-related errors after applying fixes. Permissions: Ensure the database user has write access to the x_portal_user table. Dependencies: Confirm that Spring and JPA library versions match Ranger 2.5.0 requirements. Happy hadooping   ... View more