About ahadjidj

ahadjidj · ‎02-01-2018

Hi @David Doran, I am not sure there out of the box solution to your problem but there several assets you can use to implement this. NiFi 1.5/HDF 3.1 introduced Variable Registry to handle this use case : https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry https://community.hortonworks.com/articles/154951/introduce-a-ui-driven-variable-registry-starting-f.html You have also a new flow registry that can be use as a repository to deploy these flows. Here an article that shows how to combine these two features : https://medium.com/@abdelkrim.hadjidj/fdlc-towards-flow-development-life-cycle-with-nifi-registries-82e1ee866fab Finally look to NiPyAPI for scripting these actions : https://github.com/Chaffelson/nipyapi https://community.hortonworks.com/articles/167364/nifi-sdlc-automation-in-python-with-nipyapi-part-1.html Thanks

ahadjidj · ‎01-11-2018

Hi @Ranith Ranawaka In your screenshot you are using 'equals' as a condition rather than 'lt'. Make sure to use the right relation. Also, you need to add the attribute maxTimestamp to the flowfile.

ahadjidj · ‎01-09-2018

Hi @Surendra Shringi Please create a new question for this

ahadjidj · ‎01-09-2018

Hi @Ranith Ranawaka You can use the state and conditions features of the UpdateAttribute processor to implement this https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.4.0/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html

ahadjidj · ‎01-08-2018

Hi @dhieru singh The action should be seq = 0 and not $(getStateValue("seq")) = 0.

ahadjidj · ‎12-09-2017

Hi @mayki wogno I didn't test it but you should be able to do it. At least RecordReader support it : https://community.hortonworks.com/questions/113959/use-nifi-to-change-the-format-of-numeric-date-and.html

ahadjidj · ‎12-09-2017

@Arullaldivakar Jayapalan For 1, I think you can use Rules feature of UpdateAttribute : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-update-attribute-nar/1.4.0/org.apache.nifi.processors.attributes.UpdateAttribute/additionalDetails.html You can for instance add a rule to compare between current date (now(YYYYMMDD) and the one stored in the state. If it's different, set seq to zero. You can have three attributes for instance Date Seq Filename which is a concat of the two For storing (date,max), I think the best is to use an external data store (MySQL for instance) and use NiFi to store the correct values. NiFi is not meant to keep data ... it's a data mover like DHL. Please let me know if you succeed in implementing this. Thanks

ahadjidj · ‎12-08-2017

Hi @Arullaldivakar Jayapalan You can do this with UpdateAttribute processor and the state feature. Configure it as below As you can see the results has a filename attribute as you want Does this answer your question?

ahadjidj · ‎12-05-2017

By convert I meant "create an article". Sorry for the confusion

ahadjidj · ‎12-05-2017

@Bjorn Olsen you can convert your question to an article to have more visbility

Online	Offline
Last Visited	‎08-19-2019 05:07 AM

Member Since	‎01-11-2016 06:11 PM
Last Visited	‎08-19-2019 05:07 AM
Posts	355
Kudos received	232

Cloudera Community

Re: How to access NIFI Process Group variable in E...

Re: GETSFTP with NiFi cluster

Re: how is Kafka different from Mosquitto(MQTT) ?

Re: Whitelisting using LookupAttribute

Re: Is there any ways if we can schedule or trigge...

Re: Parameterized NiFi template handling

Re: Store and compare timestamp in Nifi

Re: QueryDatabaseTable internal workings

Re: Store and compare timestamp in Nifi

Re: Resetting an attribute "seq" in updateattribut...

Re: Convert data from JSON/CSV/Avro to Parquet wit...

Re: Generate Sequence number in Apache NIFI

Re: Generate Sequence number in Apache NIFI

Re: Use NiFi to change the format of numeric, date...

Re: Use NiFi to change the format of numeric, date...