@bha  This can be done via a single long complex NiFi Expression Language (EL) statement or via a couple smaller EL statements within the "Advanced" Ui of the UpdateAttribute processor.   Before you can use NiFI EL to solve this query, you will need to get your date in to a FlowFile attribute.  Since i have no idea where the date 10/12/2019 originates from, I can't help much here.  If it is on the content, perhaps extractText processor can be used.   Lets assume we have a FlowFile with the following FlowFile attribute on it:     FlowFile attribute name:  FlowFile Attribute value: mydate  10/12/2019   We can pass this FlowFile through an UpdateAttribute processor where we create a new attribute with the date of the subsequent Sunday. (My EL assumes that if input date is already Sunday, it will calculate the next Sunday instead of just reporting that today is Sunday)   Solution1 (one long NiFi EL statement): The full EL is below (all one line):   ${mydate:toDate('MM/dd/yyyy'):format('u'):lt('7'):ifElse("${literal('7'):minus(${mydate:toDate('MM/dd/yyyy'):format('u')}):multiply('86400000'):plus(${mydate:toDate('MM/dd/yyyy'):toNumber()}):format('EEE,MM/dd/yyyy')}","${literal('604800000'):plus(${mydate:toDate('MM/dd/yyyy'):toNumber()}):format('EEE,MM/dd/yyyy')}")}   NOTE: Be careful with copy paste as it may screw up the single and double quote marks in the above EL making the EL statement invalid.   What above does is calculate day of week (1 - 7)... IF - Then if that number is less then 7:       --- it subtracts it from 7 to determine how many day until next Sunday remain.       --- Then multiples that number by number of millseconds in a day       --- Then adds that number to the millisecond value of your input date.         --- Then converts that new millisecond value back in to a date again (format here is Sun,10/13/2019) ELSE - When number is not less then 7       --- It adds 7 days in milliseconds to number to the millisecond value of your input date.       --- Then converts that new millisecond value back in to a date again (format here is Sun,10/13/2019)   So essential you have tow EL statements handled by an ifElse function. One covers days 1 -6 and the other covers day 7.   Solution 2 (Using "Advanced" UI of UpdateAttribute to break apart above long EL): The Advanced Ui allows you to construct rules.  Only the rules where the "conditions" resolve to "true" will apply the actions. So we simply create two rules.  One for days 1-6 and the other for when day of week is 7. Day less than 7: Day equal 7: Using the "Advanced"UI makes handling complex EL statements easier.   Hope this helps, Matt   ... View more

@littlesea374    There is no capability to externalize these rules used in the UpdateAttribute processor. There are numerous "lookup" processors within NiFi; however, none of them integrate with git. I think you may need a custom processor to do what you are looking for.  Contributions to the Apache NiFi community are always welcome. NiFi also offers numerous scripting processors which allow you to write your own scripts.    Matt     ... View more

Hello @shrujana    The ListenHTTP processor setups an internal Jetty server that listens for inbound http connections on the configured port.  It is not used to establish connections to external services. It may e helpful if you provide more detail about your use case(s) along with your current dataflow design specifics. How do you interface with your micro-services when not using NIFi? ... View more

Hello @littlesea374    The PutS3Object processor supports NiFi Expression Language on the majority of its properties. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.5.0/org.apache.nifi.processors.aws.s3.PutS3Object/index.html So rather then routing and creating a separate putS3Object processor for each unique setup of FlowFiles, you may consider using the Advanced UI ( ) in the UpdateAttribute to set FlowFile attributes to unique values based on your routing rules. You essentially create a new "Rule" for each of your unique match criteria. For each Rule you create an expression.  If the expression for a rule matches (resolves to "true"), all the actions will be applied to the matching FlowFile. Those actions would simply be setting FlowFile attributes for the putS3Object processor properties. Designing your dataflow in this manor allows you to scale up by simply adding new rules to the UpdateAttribute processor.  The only time you would need a different putS3Object processor is if some match requires a different property configured that does not support NiFi Expression Language (for example: different AWS credentials or region)   Hope this helps, Matt ... View more

Hello @girish6   Your first EL statement would work as well once you correct the single quote marks around the Z.  Many outside text editors end up messing up the single quotes, so the NiFi EL ends up being incorrect when you copy/paste. NiFi's EL editor built in to the processor property text blocks can help identify when the EL format is wrong. In above you can see your NiFi EL statement as copied and pasted on line 3. You'll notice the color coding indicates an issue with the function's input parameters. Line 1 has the properly formatted version of that same NiFi EL statement. Be careful when using copy paste from some external editors. Thank you, Matt ... View more

Hello @girish6,   NiFi Expression language does have a function that will increment a number, but it does not reset back to zero (unless you restart NiFi) and is incremented globally across call calls to that function.     So while this function will give each FlowFile a unique filename per NiFi node, there is no guarantee that they will be sequential or will start at 0 for new source zip file you extract.  If this works for your use case the NiFi Expression language statement would look like this in the UpdateAttribute processor: If you are just looking for filename uniqueness, a better solution may be to instead append the FlowFile's uniquely assigned UUID to the filename instead as follows:   Tips:  1. There is an expression language guide in the embedded help of your NiFi installation found by clicking on the global menu icon in upper right corner. 2. Within a NiFi processor property that support NiFi EL you can display a list of functions with details by typing "${" (signifies start of an EL statement) and then hold "Control" while clicking space bar.  This also works if you are mid statement and type ":"then "control" + click spacebar. Examples: file-${   (then cntrl+spacebar) ${uuid:  (then cntrl+spacebar) Hope this helps, Matt ... View more

Hello David,   The ReplaceTextWithMapping processor has been around for some time now and its design is rather simplistic in its current implementation. I t is designed to match a simple regex that will map one group of non-whitespace characters to another group of characters (can have white space and back references).   in your example you are matching on a a set of characters that contains whitespaces.   If you were to add a replaceText processor to your flow to replace all the whitespace with a non-whitespace character like "-", then it will work. sample ReplaceText processor configuration: Your mapping file would of course then be based off this change: account-sing-up asu account-sign-in asi account-sign-out aso   the resulting mapping would then work:   Thank you, Matt   ... View more

@sho aa Based on your first screenshot provided, it appears the putHbaseJson processor is processing FlowFiles. The stats indicate that ~783,000 FlowFiles had been pulled from the inbound connection queue in the last 5 minutes. The PutHbaseJson processor also shows that it has an active thread (1) at the time of the screenshot. It appears you are auto-terminating both the success and failure outbound relationships on your PutHbaseJson processor. If FlowFiles were auto-terminating on the Failire relationship, you should be seeing a Red square (bulletin) which your screenshot does not show. The bulletin reports an error log was produced. So I can only assume if you are not seeing bulletins being produced that FlowFiles are being auto-terminated via the success relationship. Have you verified that nothing is written to hbase? Have you verified all your hbase configurations are correct? The flow above indicates the issue is more of a throughput problem. The SplitJSON processor is producing large batches of output FlowFiles at one time (that is how that processor works). Back pressure is then applied on the connection between SplitJSON and PutHbaseJson processors (connection highlights red to indicate back pressure is being applied). SplitJSON will not be allowed to execute again until that back pressure drops below the configured threshold. The PutHBaseJson processor continues execute against FlowFiles in the connection queue. Have you tried the following to see if you see an improvement in throughput of the PutHbaseJson processor: 1. Increase configured batch size. The default is 25. Keep in mind that the content of every FlowFile in a batch is held in heap memory. Setting this value too high can lead to Out Of Memory (OOM) condition in NiFi's JVM. So increment slowly. 2. Increase configured concurrent tasks. Default is 1. One concurrent task allows this processor to execute on 1 batch at a time. Until that batch put is complete the next batch cannot be processed. Adding additional concurrent tasks will allow this processor to execute on multiple batches of FlowFiles at the same time. Keep in mind that you system does have a limited number fo cores. Setting this value too high can lead to your CPU becoming saturated. The max threads all processors on canvas must share is set via "Controller Settings" found in the global menu in the upper right corner of the NiFi UI. General rule of thumb is to set the "Maximum Timer Driven Thread Count" to a value equal to 2 to 4 times the number of cores on your NiFi server. Thank you, Matt If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more