This is awesome!   ... View more

@enam   I cant see the entire configuration of the second InvokeHTTP so I am not sure if you are passing those new attributes into the HTTP URL.   If you are not doing that, I think the solution you are looking for is to have the AttributesToJson set Destination to FlowFile Content.  Then when you post that to invokeHttp,  those values are seen as the post content.     ... View more

@edim2525   I have never had a concern to replicate kafka with nifi keeping exact order.    Perhaps others can respond with more details in a nifi context. That said,  my customers are using Streams Replication Manager (SRM) for heavy duty kafka replications.     https://docs.cloudera.com/cdf-datahub/7.2.16/howto-streams-replication-manager.html https://docs.cloudera.com/runtime/7.2.16/srm-overview/topics/srm-replication-overview.html There are several different form factors of SRM across private and public cloud but the doc links above should give you a good idea of what it is.   ... View more

@SAMSAL Thank you.     For future reference.  If you ever need to figure out what the request is,  simply use NIFI UI to do the action you want while monitoring your Developer Tools.   This will expose the full request/response on any given UI action. I have filed a JIRA for this. https://issues.apache.org/jira/browse/NIFI-12503 ... View more

You need to send a flowfile to InvokeHttp that has the content you want.   Make this flow to test:  GenerateFlowfile (body of flowfile is the JSON payload), route to InvokeHttp.  Your post headers are added to InvokeHttp in similar manner as to how you did "Body Content". ... View more

@Fayza  Without seeing any error it is impossible for us to solve a specific problem.   That said, I have some suggestions. 1.  Always make sure you have a working api call. For example: Use Postman.  You need to 100% know the call is working, auth, headers, proper format, proper payload, etc.  This also makes sure the receiving end is open (networking, firewall, etc). 2.  Test invokteHTTP without routing fail,retry,no-retry back to the processor.  I use an output port for everything other than response.  In testing, you need to know what happened.  Inspect flowfile attributes inside of fail,retry, no-retry, to get more details. When i know my flow works, i may auto terminate, or re-route failure back to processor again, but NOT during testing/creation of the flow. 3.  Last but not least, make sure that nifi host(s) are able to reach the endpoint.  I suspect this is the issue you are bumping into.     If you get invokeHttp talking to the endpoint, you should be seeing more details as to what if any errors are in the mutation payload.    ... View more

@I_Mesh01 This error is stating that your topic schema and flink schema do not match.   I cant see much above related to the shape of the data or the schema, but this is what you should be looking at.   Conflict in data/column types, missing data, etc, will result in the deserialization errors. ... View more

@hegdemahendra   This instability is usually a result of the cluster's tuning and/or workload greater than cluster tunning and spec can sustain for long period of time.   When you restart the cluster, you are effectively resetting the memory allocation.   If you are restarting the entire nifi server(s), then you are giving a fresh operating system just like rebooting windows if its running slow.  Things will work fine again until cross the fine line between working and not working. Before restart, and making tuning steps, you should document the state of the system when it is not responding as expected.  What is the load on each node?  What is the used/vs available memory? etc.   Next you should investigate Garbage Collection. ... View more