Member since
10-31-2021
11
Posts
0
Kudos Received
0
Solutions
01-12-2024
12:30 AM
Hello @LKB , Just saw your message. Since I have changed my job, I am not able to access those files anymore. Sorry for that.
... View more
05-17-2023
12:03 AM
It was a really annoying problem. Somehow, I managed to solve at that time but I dont remember how right now and I cannot check config files because I have changed my job. I don't remember exactly, but it was fixed by making some changes in the config files. So you have to make sure you set your configs properly for both nifi and registry. Sorry, couldnt help 😞
... View more
02-23-2022
11:54 PM
Hi @araujo, Thank you for your answer. Our Kafka servers are always designed to consist of 100 partitions in my company and each partition belongs to a data of an id. I explicitly generate the partition value attribute according to an id before going into Publish Kafka processor. So, there is no chance that some values could not be valid partitions. Also, I dont know why but every flow file with same partition number does not result in same partition. This one confused me. Maybe, hash function returns a value more than 100 and that value may be processed randomly in the kafka servers since we have only 100 partitions. As a last ditch solution and if it makes sense for the community, I may open a pr to add another config in publish kafka processor whether partition value should be hashed or not.
... View more
02-23-2022
05:22 AM
Hello everyone, I want to use the Publish Kafka processors and I need to specify the partition for each flow file. But when I choose Expression Language Partitioner class and add my partition info on Partition field, it does not publish my records to appropriate partitions. I realize that, instead, it gets my partition string and hashes. Then it publishes the records to partition with hashed value. I dont understand why it works like this. I want to choose a spesific partition for each flow file. How can I achieve this? Here is the code from nifi github repo. As we can see it hashes the partition value: private Integer getPartition(final ProcessContext context, final FlowFile flowFile) {
final String partitionClass = context.getProperty(PARTITION_CLASS).getValue();
if (EXPRESSION_LANGUAGE_PARTITIONING.getValue().equals(partitionClass)) {
final String partition = context.getProperty(PARTITION).evaluateAttributeExpressions(flowFile).getValue();
final int hash = Objects.hashCode(partition);
return hash;
}
return null;
} https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-2-0-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/PublishKafka_2_0.java
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi
11-19-2021
06:25 AM
@MattWho I have added Nifi keystore Owner (CN=<host>, OU=NIFI) to Registry as an user and gave these privileges. No effect. It looks like the connection between Nifi and Registry is done correctly because when I made buckets publicly available, the buckets fetched successfuly from Registry. I can see my buckets from Chrome developer tools (Network tab) as above. So, there is a issue with authorization. Do I need to save my Nifi Keystore Owner string to LDAP? Do I need to import Nifi truststore or keystore to Registry stores?
... View more
11-16-2021
06:40 AM
Thank you for helping Matt but I have tried to authenticate my Nifi, still not worked. My nifi is not clustered. I checked the Nifi keystore, console printed this output. Alias name: 1 Creation date: Mar 6, 2020 Entry type: PrivateKeyEntry Certificate chain length: 3 Certificate[1]: Owner: <owner DN> Issuer: CN=GeoTrust RSA CA 2018, OU=www.digicert.com, O=DigiCert Inc, C=US Serial number: <serialnumber> Valid from: Tue Feb 25 03:00:00 EET 2020 until: Mon Apr 25 15:00:00 EET 2022 I have added the owner DN to Nifi Registry as an user (checked whitespaces carefully). And gave necessary permissions like proxying requests but still no effect.
... View more
11-12-2021
06:44 AM
Hi there, I have the same issue. Configured LDAP for both Nifi and Registry, both uses admin user which is me. When I select registry in NiFi, it says no available buckets. I gave full permission to myself from Registry. I checked Network tab when starting version control screen in Nifi. Buckets come correctly but seems no permission to write because canWrite property is false. I tried to make bucket publicly visible but not worked. Any help would be great pls. @MattWho
... View more
11-03-2021
05:55 AM
Hi everyone, I'm newbie to Nifi. I have already configured and build my test server. I have different DB connections in my NiFi. However, our DB connection passwords are dynamic in my company. Because we are using CyberArk security system, our DB user passwords are not constant. Our applications send a kind of query to cyberark system to fetch the password on the fly and use that password to connect to DBs. My question is how to do this on a NiFi system. Can we configure the sensitive data on the controllers dynamically?
... View more
Labels:
- Labels:
-
Apache NiFi
11-02-2021
07:19 AM
Thanks a lot Matt. It worked finally after trying for 2 weeks. I was tired of trying to get this to work. I really appreciate it.
... View more
11-01-2021
12:43 AM
Hi, Im trying to connect LDAP from Nifi. I've done most of the instructions in the official documentation and read most posts on the Cloudera Community. However, I couldn't connect to my company's LDAP system. I configured login-entity-providers and authorizers for LDAP. When I start Nifi, it starts successfully and gives no error. When I try to log in with my initial admin it gives me "Unable to validate the supplied credentials. Please contact the system administrator." error. My questions are: 1- When we are setting up initial admin, we are not setting password of that user. How do we log in without password? 2- I'm using USE_DN for identity strategy for ldap-provider. My initial admin name and ldap user are same. So I'm trying to login with my user like: Username: CN=K015576,CN=Users,DC=tcmb,DC=gov,DC=tr Password: "my_ldap_password" What path should I follow? Is there a mistake in my configure files? I have added my conf files below: nifi.properties, login-entity-provider and authorizers. Removed sensitive data. nifi-properties login-identity-providers authorizers-1 authorizers-2
... View more
Labels:
- Labels:
-
Apache NiFi