Member since
07-19-2018
613
Posts
101
Kudos Received
117
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4901 | 01-11-2021 05:54 AM | |
3337 | 01-11-2021 05:52 AM | |
8642 | 01-08-2021 05:23 AM | |
8157 | 01-04-2021 04:08 AM | |
36035 | 12-18-2020 05:42 AM |
09-30-2020
05:08 AM
@praneet Adding the value to the processor (+) is a suitable method. You just need to make sure you get the right string in that field. It's blocked out, but appears not just the actual token but prepended with "Bearer". Try just the token string. One thing I like to do for any API, before I start to work on invokeHttp configuration, is to use Postman to help me identify all of the required settings to test connection to an API. Once this is complete, I can definitively ensure that nifi invokeHttp is sending the same request. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-30-2020
05:03 AM
@Manoj90 In addition o @PVVK point, you need to be careful with routing relationships back on the originating processor. During development I like to use a output port that I call End of Line or EOL1, EOL2, EOL3 as I need more in larger flows. This is to evaluate if something goes to fail, retry, etc. Later once I am certain the flow is working as I need, I either auto terminate these routes, or I route them out of my processor group to an Event Notification system. It looks like this: Using an output port to hold un needed routes during testing If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-30-2020
04:52 AM
@mansu You need to make sure the file and locations have the correct permissions for the nifi user. For example in linux: chown -R nifi:nifi /path/to/files If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-28-2020
09:58 PM
At the end of the day I used Steve Matison and 'PVVK's solutions. First I used PVVK's to modify my flowfile into valid html, then I successfully was able to use SplitJson processor to break up the flowfiles cleanly. Then since there were multiple 'logEvent' entries in some of the records I saved off the header items as attributes and then used SteveMatison's method to break down each logEvent and added the header attributes back to it for processing. Thanks all!
... View more
09-28-2020
03:54 PM
@vikrant_kumar24 Based on what you describe you are either exceeding the capabilities of the outbound network connection from nifi, or the receiving end API. I suspect the latter but its hard to tell without seeing the number of nodes and processor scheduling settings. You can try to slow down the execution of invokeHttp to confirm. If it is set to 0 sec it can create a lot of connections to the endpoint. Try 5,10,15 seconds and re-evaluate. Also If you have more than 1 one nifi node, it can create a great deal of concurrent connections. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-28-2020
03:46 PM
@aniket5003 Check out the following post for a few different ways to locate or update the password: https://community.cloudera.com/t5/Support-Questions/Default-password-for-Metron-UI-Version-0-4-0/td-p/199277 If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-28-2020
03:41 PM
@surajnag This is a great question. Here are some of my ideas which I have used in development and production nifi flows. During development route all flow exceptions to an output port. I call these EOL, or End of Line and increment them with #s, like EOL1, EOL2, EOL3. I use these to hold failure, retry, original, no-retry, etc type outbound connections during testing. As I move to production, some of these may be auto terminated, and some may remain. In production, I route ALL major points of failure which are not auto terminated to an output port called ERROR. In my flows I sometimes have multiple ERROR output ports for difference purposes. Depending on the use case, the error is sent to some type of event notification. For Example: send an email, or post to a slack channel. In other cases these are routed to a process group that is in a stopped/disabled state. Based on the error, I may make some change in the flow, and then enable/start the flow to create a method to REPLAY that flowfile. In reference to successful end of flow, I may just auto terminate, as I want to assume the flow is always on, and everything has succeeded if it is not routed as above in #1 or #2. In other cases the bottom or end of the flow is also routed to some type of Event Notification. However, be careful here, as you could create tons of notifications depending on your flow. I hope some of these ideas are helpful. If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
09-25-2020
12:17 PM
Hi Steven, I used @bingo 's solution to get Nifi to find my JAVA_HOME. But you mention that Nifi does not need this to run. Do you know what is the impact for running nifi without it knowing where Java is installed?
... View more
09-24-2020
08:34 AM
1 Kudo
I solved my problem. In my case one of the talbes' name was starting with a character underscore "_" because of which there was an issue where 2 single quotes were added automatically in the path of the hdfs directory where the copy of the file was stored. I changed the name of the column by removing the underscore character and now i can import the table into Hive database. I think special characters like that are not easily parsed in Hive or HDFS.
... View more
09-24-2020
06:34 AM
1 Kudo
@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord). However if its just one giant blob of JSON you may not find that very useful. Perhaps you can share some sample json to that effect. Check out this way to do SNS up to 2gb per payload: https://github.com/awslabs/amazon-sns-java-extended-client-lib If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more