Member since
07-19-2018
613
Posts
100
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3141 | 01-11-2021 05:54 AM | |
2242 | 01-11-2021 05:52 AM | |
5993 | 01-08-2021 05:23 AM | |
5563 | 01-04-2021 04:08 AM | |
25734 | 12-18-2020 05:42 AM |
11-09-2020
04:40 AM
@vatodorov19 If the service you are connecting too is https, there is no way to use invokeHttp without SSL Verification. If the remote host is using a publicly signed cert, try using cacerts in your java location before trying to make keystores and truststores. If you do make keystores and truststores from the service's public or self signed certs you will need to make sure those certs are generated following best practices. For example: using a real hostname for the service and make sure you connect to the same hostname, not an ip or "resilient.localdomain". If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
11-02-2020
05:15 AM
@P_Rat98 Per our PM discussion, in your flow use DetectDuplicate before sending an email. This should rate limit the # of messages you send based on your configuration of detectDuplicate. Additionally when this is linked in your flow, and duplicates are auto-terminated it will drain the flow and stop it from filling up the queue. Also as suggested you can chose to retain the duplicates, but move them into a much bigger Queue which isnt going to back up the main flow. Then once you see the email, you can go look at flow, see what flowfiles were causing issues, and take some corrective action. If you really need to monitor flow for a queue being full, you would need to use the nifi API to check the status of the queue. This maybe more work than it is worth, when you can solve as above much easier. However, i would recommend you check the api out, there are a lot of api capabilities and I am beginning to use nifi api calls within my flow to monitor, stop, start, and take actions automatically that would normally require a human doing them in the UI. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-29-2020
08:15 AM
I recommend whatever works. One of things i like the most about nifi is there are many ways to do similar things. So if one way works for you, use it until you have some reason to try another way.
... View more
10-29-2020
05:28 AM
1 Kudo
@robnew666 yes you can do that, you just need to experiment with SQL statements that satisfy your requirements. You should work on some tests and inspect what you get in the flowfile after a select *. This will depend on your reader, etc. You can also have multiples QueryRecord Processors but dont forget it is possible to have multiple queries in a single processor. The key you use to hold the query will become a route out of the processor downstream. For example you could select all results that match a certain condition, and separate those from the rest.
... View more
10-29-2020
04:46 AM
@amey84 Yes. Although yum install still provides the bundled postgres, you can choose to install it or another database separately. During ambari-server setup you choose Y here: Enter advanced database configuration [y/n] (n)? y The following links will be helpful here for more info about ambari + postgres: https://docs.cloudera.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-administration/content/using_ambari_with_postgresql.html If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-29-2020
04:39 AM
@Kaur it appears like your nifi node does not have enough system ram to allow you to use 2g and 4g settings. I suggest increasing the node specification to at least 8gb or 16 gb of system ram and test boostrap config with 2g 4g or 4g 8g respectively. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-28-2020
05:54 AM
@robnew666 You should provide more information in order for community members to respond with specific details. Without seeing schema, sample data, what you have, etc, we have little to go off. That said below is an example of query record with a query. The important theory here is to use + to add new queries and use SELECT * FROM FLOWFILE. You can add WHERE [your query logic here] to operate agains schema columns just like SQL. You can have more than 1 query and route them separately down stream. So for testing start with your first query1 (result) like below. Next begin to add WHERE and operate against your requirements. As you further advance your query, make new query2, query3, working methodically towards what works. When done, delete the test queries you do not need and then route the final query downstream. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-28-2020
05:45 AM
@vatodorov19 i always find it interesting when someone posts on a topic I was just working on. In this case I was working on Map/Cache Use Case and struggled to get it working until I found a template: https://github.com/ds-steven-matison/NiFi-Templates/blob/main/DistributedCache_Demo.xml Install the template and get the controller services working, you will need to add the DistributedMapCacheServer before you can get the client working. With server and client controller services enabled, run the flow. Pay very close attention to the content of the flowfile sent to PutDistributedMapCache. Next look at the Cache Entry Identifier in the properties of PutDistributedMapCache. Once you have the demo working and are familiar with the basic implementaiton, modify this to fit your use case and operate within your flow. If you have more specific questions about your flow, or getting the demo working please let me know. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-19-2020
01:21 PM
The solution you are looking for is: ReplaceText: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.0/org.apache.nifi.processors.standard.ReplaceText/ You can find loads of examples here in the forum with this search: https://community.cloudera.com/t5/forums/searchpage/tab/message?advanced=false&allow_punctuation=false&q=replaceText If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
10-13-2020
10:05 AM
@HansH I am not sure if this helps but just yesterday i had to use the argument in the reader jdbc connection string like this: ?serverTimezone=UTC
... View more