Member since
11-13-2017
31
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3610 | 10-30-2018 11:08 AM |
09-23-2019
02:44 PM
Hello Haris, we are in the same situation, in my opinion the number of processors is a hard limit in a NiFi cluster. Given the timer-driven architecture, the number of processors running at the same time is heavily influenced by the amount of threads available. In this regard, any suggestion from NiFi developers is welcome. For the other questions: the type of processors used and the overall flow design will also have an impact on performance, I would not consider the input volume as a problem since you can scale horizontally. R.
... View more
08-08-2019
08:14 AM
Hello Satish, this error is not related to the procedure you followed, you need to kinit as kafka's user.
... View more
11-06-2018
08:25 AM
Which version of Kafka are you using? (not ConsumeKafka) Can you also post the configuration?
... View more
11-02-2018
09:12 AM
@siddharth pande The default behavior is the one you described at the beginning, with each node consuming from a different partition. You should share the Processor's configuration and a describe of the topic. Check also that the ConsumeKafka processor is compatible with the version of Kafka you are using.
... View more
10-30-2018
11:08 AM
Properly setting up the nifi.security.identity.mapping.pattern.kerb and nifi.security.identity.mapping.pattern.dn fixed the problem. Also, while debugging these kind of problems, it's best to delete ranger plugin cache (under /etc/ranger/SERVICE_NAME/policycache/) to ensure that there are no communication problem between NiFi and Ranger.
... View more
10-29-2018
10:37 AM
This seems to have worked for me. 1) kinit 2) source the env 3) execute zkCli.sh. Thanks
... View more
08-30-2018
03:35 PM
There aren't many information in the manual is it possible I have to manually configure all the options under "Advanced ranger-nifi-plugin-properties" (in the ambari console)? Currently only a few of those properties are configured.
... View more
08-30-2018
02:44 PM
Thanks, I will use this configuration while testing in the future.
... View more
08-30-2018
02:43 PM
Hello @Steven Matison, thanks for replying. I believe that everything is setup as you proposed, I also added the NiFi proxy users to their own policy but nothing changed. Tailing the usersync.log doesn't provide any additional evidence.
... View more
08-29-2018
04:32 PM
Hello, I am setting up an HDF 3.2 cluster that's fully Kerberized and I am trying to handle authorization through Ranger. We have a single Active Directory which also acts as KDC. The technical users (for example the service principals automatically created by HDF) are mapped in the following organization unit: OU=HDF,DC=example,DC=com while "normal" users (devs/admins) are mapped in OU=USERS,DC=example,DC=com The problem is that after enabling NiFi plugin and Kafka plugin I have not been able to use any of the two (I added my username to the "admin" policies of both services in ranger). Since the two problems are probably linked, I will start from NiFi and if necessary expand on Kafka in another post. NiFi authentication works (the user is recognized) but I receive the following error: "Unable to view the user interface. Contact the system administrator." When I check the audit log I notice that the User is indicated with the full qualified domain name USER@EXAMPLE.COM (instead of just the username) and the access is denied. nifi.security.user.login.identity.provider=kerberos-provider
I tried the following NiFi properties without success: nifi.security.identity.mapping.pattern.kerb=^(.?)@(.?)$
nifi.security.identity.mapping.value.kerb=$1
Could you help me solve this problem? Thanks
... View more
Labels: