Member since
10-03-2022
13
Posts
5
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
328 | 05-27-2024 05:11 AM | |
655 | 11-20-2023 01:12 AM |
06-30-2024
05:50 AM
1 Kudo
Hi all, I am trying to integrate sending emails from NiFi using the put email processor. Below is my configuration: When I start the processor I get this error: Failed to properly initialize Processor. If still scheduled to run, NiFi will attempt to initialize and run the Processor again after the 'Administrative Yield Duration' has elapsed. Failure is due to java.lang.NullPointerException: java.lang.NullPointerException Do you have any advice regarding the configuration? Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
05-27-2024
05:11 AM
2 Kudos
@MattWho To authenticate to the web ui in NiFi i use the ldap credentials (myuser). For Kerberos authentication via shell I use myuser@REALM. After setting the following parameters in nifi: nifi.security.identity.mapping.pattern.kerb=^(.*?)(?:@.*?)$
nifi.security.identity.mapping.value.kerb=$1
nifi.security.identity.mapping.transform.kerb=NONE Now the token via kerberos works and I no longer get permission errors. Thanks! Lorenzo
... View more
05-08-2024
05:52 AM
1 Kudo
I temporarily solved it by eliminating the dynamic child creation
... View more
05-08-2024
05:48 AM
Hi everyone, I'm trying to use rest api in a cloudera cluster with ssl and kerberos. I am testing the use of the same by authenticating with a bearer token to gain access to the resource. Below is what is used and working: curl 'https://nifi-node:8443/nifi-api/access/token' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' --data 'username=myuserad&password=mypasswordad' --compressed --cacert /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem curl -H 'Authorization: Bearer token generated 'Content-Type: application/json' -XPUT -d '{"id":"****","state":"RUNNING"}' https://nifi-node/nifi-api/flow/process-groups/****--cacert /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem To avoid entering the password in clear text as in curl N.1 I am testing the token generation via Kerberos: curl -X POST --negotiate -u : https://nifi-node:8443/nifi-api/access/kerberos --cacert /var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts .pem Using this mode the token is correctly generated but when I try to execute API N.2 I receive the following error: o.a.n.w.a.c.AccessDeniedExceptionMapper identity[myaduser], groups[] does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response. Do you have any advice?
... View more
02-07-2024
02:09 AM
Hello, the ranger version is 2.1.0 and there are no error logs. the ranger-ugsync-site.xml file contains: <?xml version="1.0" encoding="UTF-8"?> <!--Autogenerated by Cloudera Manager--> <configuration> <property> <name>ranger.usersync.cookie.enabled</name> <value>true</value> </property> <property> <name>ranger.usersync.enabled</name> <value>true</value> </property> <property> <name>ranger.usersync.filesource.text.delimiter</name> <value>,</value> </property> <property> <name>ranger.usersync.group.memberattributename</name> <value>member</value> </property> <property> <name>ranger.usersync.group.nameattribute</name> <value>cn</value> </property> <property> <name>ranger.usersync.group.objectclass</name> <value>group</value> </property> <property> <name>ranger.usersync.group.searchbase</name> <value>OU=CLOUDERA,OU=APPLICATION GROUPS,OU=GRUPPI,DC=test,DC=test</value> </property> <property> <name>ranger.usersync.group.searchscope</name> <value>sub</value> </property> <property> <name>ranger.usersync.keystore.password</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/altscript.sh sec-0-ranger.usersync.keystore.password</value> </property> <property> <name>ranger.usersync.ldap.binddn</name> <value>CN=clouderabind,OU=CLOUDERA,OU=USER DI SERVIZIO,OU=UTENTI,DC=test,DC=test</value> </property> <property> <name>ranger.usersync.ldap.dtestasync</name> <value>false</value> </property> <property> <name>ranger.usersync.ldap.grouphierarchylevels</name> <value>0</value> </property> <property> <name>ranger.usersync.ldap.groupname.caseconversion</name> <value>lower</value> </property> <property> <name>ranger.usersync.ldap.ldapbindpassword</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/altscript.sh sec-0-ranger.usersync.ldap.ldapbindpassword</value> </property> <property> <name>ranger.usersync.ldap.referral</name> <value>ignore</value> </property> <property> <name>ranger.usersync.ldap.starttls</name> <value>false</value> </property> <property> <name>ranger.usersync.ldap.url</name> <value>ldap://test-dc08.test.test:389</value> </property> <property> <name>ranger.usersync.ldap.user.nameattribute</name> <value>sAMAccountName</value> </property> <property> <name>ranger.usersync.ldap.user.objectclass</name> <value>user</value> </property> <property> <name>ranger.usersync.ldap.user.searchbase</name> <value>OU=UTENTI,DC=test,DC=test</value> </property> <property> <name>ranger.usersync.ldap.user.searchscope</name> <value>sub</value> </property> <property> <name>ranger.usersync.ldap.username.caseconversion</name> <value>lower</value> </property> <property> <name>ranger.usersync.logdir</name> <value>/var/log/ranger/usersync</value> </property> <property> <name>ranger.usersync.metrics.enabled</name> <value>true</value> </property> <property> <name>ranger.usersync.metrics.filename</name> <value>metrics.json</value> </property> <property> <name>ranger.usersync.metrics.filepath</name> <value>/var/log/ranger/metrics-usersync</value> </property> <property> <name>ranger.usersync.metrics.frequencytimeinmillis</name> <value>60000</value> </property> <property> <name>ranger.usersync.pagedresultsenabled</name> <value>true</value> </property> <property> <name>ranger.usersync.pagedresultssize</name> <value>500</value> </property> <property> <name>ranger.usersync.policymanager.maxrecordsperapicall</name> <value>1000</value> </property> <property> <name>ranger.usersync.policymgr.username</name> <value>rangerusersync</value> </property> <property> <name>ranger.usersync.port</name> <value>5151</value> </property> <property> <name>ranger.usersync.role.assignment.list.delimiter</name> <value>&</value> </property> <property> <name>ranger.usersync.sleeptimeinmillisbetweensynccycle</name> <value>60000</value> </property> <property> <name>ranger.usersync.source.impl.class</name> <value>org.apache.ranger.ldapusersync.process.LdapUserGroupBuilder</value> </property> <property> <name>ranger.usersync.truststore.file</name> <value>/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks</value> </property> <property> <name>ranger.usersync.truststore.password</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/altscript.sh sec-0-ranger.usersync.truststore.password</value> </property> <property> <name>ranger.usersync.unix.backend</name> <value>passwd</value> </property> <property> <name>ranger.usersync.unix.minUserId</name> <value>500</value> </property> <property> <name>ranger.usersync.user.searchenabled</name> <value>true</value> </property> <property> <name>ranger.usersync.username.groupname.assignment.list.delimiter</name> <value>,</value> </property> <property> <name>ranger.usersync.users.groups.assignment.list.delimiter</name> <value>:</value> </property> <property> <name>ranger.usersync.kerberos.keytab</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/ranger.keytab</value> </property> <property> <name>ranger.usersync.policymanager.baseURL</name> <value>https://test-clmaster03.test.test:6182</value> </property> <property> <name>ranger.usersync.credstore.filename</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/conf/rangerusersync.jceks</value> </property> <property> <name>ranger.usersync.policymgr.keystore</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/conf/rangerusersync.jceks</value> </property> <property> <name>ranger.usersync.keystore.file</name> <value>/var/run/cloudera-scm-agent/process/1546329977-ranger-RANGER_USERSYNC/conf/unixauthservice.jks</value> </property> <property> <name>ranger.usersync.policymanager.mockrun</name> <value>false</value> </property> <property> <name>ranger.usersync.passwordvalidator.path</name> <value>/opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p1000.24102687/lib/ranger-usersync/native/pamCredValidator.uexe</value> </property> <property> <name>ranger.usersync.sink.impl.class</name> <value>org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder</value> </property> <property> <name>ranger.usersync.ssl</name> <value>true</value> </property> <property> <name>ranger.usersync.unix.group.file</name> <value>/etc/group</value> </property> <property> <name>ranger.usersync.unix.password.file</name> <value>/etc/passwd</value> </property> <property> <name>ranger.usersync.ldap.bindalias</name> <value>ranger.usersync.ldap.bindalias</value> </property> <property> <name>ranger.usersync.policymgr.alias</name> <value>ranger.usersync.policymgr.password</value> </property> <property> <name>ranger.keystore.file.type</name> <value>jks</value> </property> <property> <name>ranger.truststore.file.type</name> <value>jks</value> </property> <property> <name>xasecure.policymgr.clientssl.keystore.type</name> <value>jks</value> </property> <property> <name>xasecure.policymgr.clientssl.truststore.type</name> <value>jks</value> </property> <property> <name>ranger.usersync.kerberos.principal</name> <value>rangerusersync/_HOST@test.test</value> </property> </configuration> ranger.usersync.ldap.user.searchbase OU=utenti,DC=test,DC=test ranger.usersync.group.searchbase OU=Cloudera, OU=Application Groups,OU=Gruppi, DC=test,DC=test Thanks in advance.
... View more
02-05-2024
08:21 AM
Good evening everyone, I have a problem on ranger, the users have access to the databases and everything works, but on the ranger web ui if I try to search for users belonging to the groups the following screen is shown. Even if checking in Unix the group contains several users: I saw that there is a KB for the same problem after upgrade but I already upgraded to 7.1.7sp1. Can anyone help me? Thank you
... View more
Labels:
11-20-2023
01:12 AM
I temporarily solved it by eliminating the dynamic child creation
... View more
10-30-2023
01:14 PM
Hi everyone. I created 2 queue yarns in cdp 7.1.8. Precisely I created 2 queues: users queue --> configured capacity 50% and maximum capacity 100% hive queue -->configured capacity 50% and maximum capacity 100% I enabled child queues mode for all 2 queues with the following configuration parameters: Dynamic Queue Minimum User Limit 100% Dynamic Queue User Limit Factor 1 Dynamic Queue Maximum Applications 1000 Dynamic Queue Maximum AM Resource Limit 20% Dynamic Queue Ordering Policy Fair The problem we encounter is that if even just 2 queries are executed in the same queue the second goes pending and is not executed in parallel, I show an example below Another example would be an oozie job that only calls a spark job. The oozie job runs infinitely waiting for the pyspark to run. However, in the default queue this problem is not present.The development cluster is made up of 3 worker nodes (144 vcores 180 gb yarn memory). Can you provide us support or optimizations to do at the queue configuration level? Thanks in advance Lorenzo
... View more
09-22-2023
06:02 AM
1 Kudo
Thank you, for now i implemented scala on hue and it works
... View more