If you are trying to add an attribute to a flow file, you can use UpdateAttribute for that. If you are trying to add a property to a processor, then it depends on the processor whether it supports dynamic (or "User-Defined") properties. If a processor does not support dynamic properties, then when you try to add one, the processor will be deemed invalid. ... View more

QueryDatabaseTable is usually used for "incremental" fetching, meaning it will only grab "new" rows. This is based on the "Maximum Value Columns" property, which is usually set to an ID or timestamp field in the database. That is what allows the processor to only grab "new" rows, as it will keep track of the maximum value it has seen so far for the column, and the next time it runs, it will fetch only those rows whose value is greater than the last max it saw. If you are not setting a Maximum-Value Column, then QueryDatabaseTable acts much like ExecuteSQL, in the sense that it will keep repeating the same query and thus give duplicate rows. So for your use case I recommend setting a Maximum Value Column for that processor. If there is no such field in the table, then you're really looking at more of a "run-once" scenario, which is not currently supported. ... View more

What do you mean by receiving attributes from a URL? Do you mean query parameters? Or are you asking if there is a way to combine the fetching of the document via the URL and the extraction of JSON fields from the response? If the latter, there isn't a way to combine those currently; NiFi is a highly modular system so fetching is really a different operation than extractions, hence the two separate processors. What do you see as the issue in receiving the whole JSON response? The third-party libraries I'm familiar with still retrieve the whole HTTP response whether they expose it via an input stream or a big string or whatever. After the initial fetch/response, if you don't touch the content after that, there won't be an operational impact with the respect to the size of the JSON response. If you no longer need the content, you can replace it with something else (or nothing) using ReplaceText. One concept that is (hopefully) becoming more popular with web services is support for GraphQL. Using that paradigm and query language, you could ask for just the fields/structures you want, and you get the response you expect. ... View more

I don't think there is any other way, besides ExecuteScript or ExecuteStreamCommand (both of which require the HBase client libraries / programs, the former of which is already in the HBase NAR). I have added NIFI-3538 to track the addition of DeleteHBase processor(s). ... View more

It seems feasible to remove this restriction from Date and Time data types, as was done for Timestamps in NIFI-3430. Please feel free to write up a Jira case for this improvement. In the meantime, if you are trying to do something like use the value '2017-02-21' in a field specified as Date/Time type, you could first change the value (possibly in an UpdateAttribute processor) using NiFi Expression Language's toDate() function. So if you have an attribute such as 'my.date.string' set to '2017-02-21' you could set an attribute 'my.date' equal to the following: ${my.date.string:format('yyyy-MM-dd')} ... View more