1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
775 | 04-03-2024 06:39 AM | |
1425 | 01-12-2024 08:19 AM | |
771 | 12-07-2023 01:49 PM | |
1327 | 08-02-2023 07:30 AM | |
1922 | 03-29-2023 01:22 PM |
12-06-2024
04:40 AM
1 Kudo
I was having similar issue, where I had around 7000+ flow files and I needed to merge them into one. Provided that, I didn't know what will be the exact number of flow files, I couldn't specify that number in Minimum Number of Entries attribute of mergeContent processor to make it only 1 flow file after merge. After trying a lot of things, including trying to count the flow files or induce a delay before sending them off to merge, I finally found the solution on the mergeContent documentation below : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.MergeContent/index.html#:~:text=The%20%27Defragment%27%20algorithm%20combines%20fragments%20that%20are%20associated%20by%20attributes%20back%20into%20a%20single%20cohesive%20FlowFile.%20The%20%27Bin%2DPacking%20Algorithm%27%20generates%20a%20FlowFile%20populated%20by%20arbitrarily%20chosen%20FlowFiles And there you go ! setting 'Merge Strategy' property to 'Defragment' and 'Attribute Strategy' property to 'Keep Only Common Attributes' fixes the problem for me. In all my flow files, they have common attributes and are now merged into a single flow file. The original question is from 2017, but I hope it helps out somebody else looking for an answer. You can check the original answer here https://stackoverflow.com/questions/56356164/nifi-merging-all-of-necessary-flowfile-in-one-shot-with-mergecontent-processor/79257994#79257994
... View more
10-17-2024
03:22 AM
1 Kudo
Hello. I encountered the same problem, but I need to pass avro as a key. Is it possible to do this in publishKafka? Now I’m generating an avro schema and using ExtractText I save it into an attribute, then I pass this attribute to the Kafka key, but it doesn’t work. The avro circuit breaks down. And it is not parsed by the agent. Help please.
... View more
08-08-2024
01:29 AM
@SteliosCh, are there any updates on the issue?
... View more
04-24-2024
10:18 AM
What S3 security are you using? Can you put the same file there manually? Is this regular AWS S3 or minio or another S3 compatible store.
... View more
04-23-2024
07:56 AM
Thank you, @MattWho for providing timely responses and quick solutions to the queries. You are really helping the community grow. Hats off to you. Appreciate it
... View more
04-17-2024
07:16 AM
1 Kudo
https://community.cloudera.com/t5/Support-Questions/ExecuteScript-error-ECMAScript-is-missing/m-p/346336 Nashorn is removed from JDK 15, NiFi 2 uses JDK 21 Keep an old JDK 8 to run
... View more
04-16-2024
06:30 PM
@tmarkfeld Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
04-03-2024
06:39 AM
https://community.cloudera.com/t5/Support-Questions/how-to-Setup-IBM-MQ-Configuration-for-Nifi/td-p/118155 https://nifi.apache.org/documentation/nifi-2.0.0-M2/components/org.apache.nifi/nifi-jms-processors-nar/2.0.0-M2/org.apache.nifi.jms.cf.JMSConnectionFactoryProvider/additionalDetails.html
... View more
03-25-2024
12:31 PM
2 Kudos
That is a scam not affiliated with Cloudera. Do not work with them. Do not use that website. Do not email them. Do not visit that website. This is not from Cloudera.
... View more
03-04-2024
07:30 AM
@MvZ The "file-login-provider" login identity-provider has never existed in any out-of-the-box release of Apache NiFi. If you have created or downloaded some custom implementation of this provider. You would need to consult with that author in getting it to work. Where did you obtain this provider from and what process did you follow to add it to your NiFi installation? The exception you have shared simply tells you that during startup NiFi is loading the nifi.properties file and the property "nifi.security.user.login.identity.provider" is configured with "file-login-provider"; however, when NiFi parsed the login-identity-providers.xml configuration file, no provider with: <identifier>file-login-provider</identifier> was found in that configuration file. I can't provide any guidance on this provider as I was unable to find anything online about what I am expecting is a custom add-on provider. The out-of-the-box available authentication providers are found in the NiFi documentation here: Apache NiFi 1.2x versions: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Apache NiFi 2.x versions: https://nifi.apache.org/documentation/nifi-2.0.0-M1/html/administration-guide.html#user_authentication NiFi Authentication and Authorization are two different configurations and independent configurations. Once you have chosen how you want to handle user authentication, you then move on to setting up user authorization: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization. For file based authorization, NiFi offers two providers: 1. Older deprecated FileAuthorizer 2. The current StandardManagedAuthorizer These providers are configured in the NiFi authorizers.xml file. No direct useer policies get defined in the authorizers.xml file. The FileAuthorizer or the FileAccessPolicyProvider referenced by the StandardManagedAuthorizer will generate the initial authorizations.xml file with the initial admin user configured in the provider chosen. You would not typically manually generate or manipulate this file. Instead you would acces your NiFi's UI using that initial admin and define additional user authorizations directly via the NiFi UI. Here is an example of what you would have in your authorizers.xml if using the StandardManagedAuthorizer: <authorizers>
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">ronald</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">ronald</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers> If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more