I'm not sure if there is a function in JSONPath to retain the escaped quotes or not, but you could use UpdateAttribute (between EvaluateJsonPath and ReplaceText) along with the escapeJson function to "re-introduce" the quotes, by setting the "observation" attribute to the following value: ${observation:escapeJson()} ... View more

In later versions of NiFi that use newer versions of Avro, Date and Timestamps are supported, but as logical types (backed by long values not strings). Previous versions of NiFi converted them to strings as the version of Avro at the time (1.7.7) didn't support logical types for Date/Timestamp ... View more

From the PutElasticsearchHttp documentation: "The name of the FlowFile attribute containing the identifier for the document. If the Index Operation is "index", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the attribute must evaluate to a non-empty value." So for "update", Elasticsearch needs to know which document to update. As a general rule, if the document has an identifier, you should use EvaluateJsonPath or something to extract the identifier into an attribute (let's call it "es.id" for this example), then in PutElasticsearchHttp you can set the Identifier Attribute property to "es.id". ... View more

PutHiveStreaming will accept all valid Avro files (the fields don't have to be strings), the problem here is that a bigint(20) will not fit in a Long object (the max number of digits is 19), so QueryDatabaseTable converts it to a String, and thus it won't go into your Hive table as that expects an int. If your values are not 20 digits long, I recommend you alter your table to make the ID column smaller, then things should work better. If you can't alter the table, but the values will never be 20 digits long, you could use ConvertAvroSchema to try and convert the ID values from String to Long/Int. ... View more

The relevant part of the log is "Address already in use", looks like you've configured ListenHttp to listen on port 8081 but some other process is using that port already. ... View more

You may be interested in the NiPyAPI module, it makes working with the NiFi API from Python MUCH easier. ... View more

No, you'd have to use ExecuteStreamCommand or ExecuteProcess for things like Anaconda environments, non-pure (CPython) modules, etc. ... View more

JOLT can be fairly complicated. In this case the * on the left is a non-greedy match, the &(x,y) on the right is to grab the individual pieces. ... View more