I have a situation where I query Elasticsearch using "QueryElasticsearchHttp" to match a query then return results of the query to populate flowfile-attributes. However, if the query returns empty results it doesn't go to "failure queue". I need the results but the results are inconsistent, they may not be ready to be queried. I want it to fail so that I can "Retry at a later time" using ControlRate. However, whenever QueryElasticsearchHttp fails to find the document, it just disappears, no failure, no success queue file. The flowfile disappears. I CANNOT use FetchElasticsearchHttp because it has no property: "Target: flowfile-attributes" ( I need the flowfile content that is being queried ) I am really stuck now, because my results in elasticsearch are populated by a different nifi flow process, so it may not be ready to be "Queried from Elastic". That's a timing issue. I want it to try querying again later. ... View more

Hi, so I figure with Nifi 1.3 the best way to send PDFs between two systems is to put it inside a binary field ("bytes" field) in Avro. This comes in as binary, and then using maybe UpdateRecord you can modify some fields, then I want to store it in HBase. Now here is where the problem comes in. The binary in Avro is working but HBase only allows JSON storage (Same with Solr or Elasticsearch). So when you convert the binary in Avro to JSON, JSON cannot store binary (it may convert it to a byte-array type of integers). What's the best standard way of storing it in a JSON NoSQL database? Would it be smarter to convert to something like Base64 or whatever and have that as the JSON field? Additionally, whenever I use Avros with binary, I seem to have some trouble converting it, I get a lot of "ArrayIndexOutOfBounds" whenever I try to use UpdateRecord for example. I guess the only way is to splitAvro or SplitJSON and store the binary inside flowfile-content??? This would slow down the process a lot. ... View more