It occurred to me that it might be better to just convert the entire flowfile contents to the stringify version rather than isolating the problematic blob first. I attempted this by using the following in the ExecuteScript: var StreamCallback = Java.type("org.apache.nifi.processor.io.StreamCallback"); var IOUtils = Java.type("org.apache.commons.io.IOUtils"); var StandardCharsets = Java.type("java.nio.charset.StandardCharsets"); var flowFile = session.get(); if(flowFile != null) { // Create a new StreamCallback, passing in a function to define the interface method flowFile = session.write(flowFile, new StreamCallback(function(inputStream, outputStream) { var text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) outputStream.write(JSON.stringify(text).getBytes(StandardCharsets.UTF_8)) })); session.transfer(flowFile, REL_SUCCESS); } However, the resulting content has been modified where the existing json structure is now masked by the use of escape characters before double quotes for instance. Is there an easy way to convert this object back into a json array? Thanks. ... View more

Thank you for the reply Bryan. I'm stripping the data not needing to be converted prior to the ExecuteScript via two SplitContext processors that utilize hardcoded hexadecimal strings to isolate the string (called attributed 'default_payload') needing to be converted via the ExecuteStript: var flowFile = session.get(); if (flowFile !== null) { var StreamCallback = Java.type("org.apache.nifi.processor.io.StreamCallback"); var IOUtils = Java.type("org.apache.commons.io.IOUtils"); var StandardCharsets = Java.type("java.nio.charset.StandardCharsets"); flowFile = session.write(flowFile, new StreamCallback(function(inputStream, outputStream) { var contentObj = flowFile.getAttribute("default_payload"); outputStream.write(JSON.stringify(contentObj ).getBytes(StandardCharsets.UTF_8)); })); session.transfer(flowFile, REL_SUCCESS); } If I understand what your saying then the ExecuteScript would need to handled both the splitting and conversion at the same processor as to not write over attributes which would keep it synchronized in the flow? I had tried to use a parent UpdateAttribute to introduce a unique value to the flowfile(s) prior to the split/convertion and other JSON processing, but I am thinking this is to loose of a linkage for this type of processing. ... View more

I suppose I should look outside of NiFi for solutions as there doesn't seem to be similar experiences or use cases as I have attempted. ... View more

If loading the entire table it the only option then would not providing a downstream processor allow the table to drain without clogging memory or other resources? Which processor would give me the most control on the number of rows/records returned with each iteration? With GenerateTableFetch it appears to be handled with the Partion Size parameter. With the QueryDatabaseTable processor I've experimented with using the Columns to Return and Max Rows per File parameters without much success. The result of executing the QueryDatabaseTable processor always results in bring the NiFi cluster to a halt requiring one or more restarts to recover, so I'm reluctant to just arbitrarily try things. Any suggestions would greatly be appreciated. ... View more

The issue was with the customized bundle utilizing a NiFi 0.5 api where the NiFi running the processor was at 1.0.0. ... View more

I realize my question is too vague to assist based on what I've provided. I'm not sure the "thread-dump.txt" is even capturing anything related to the problem. I know the processor is entering the exception part of the custom code, but I cannot get the inherent getLogger() to produce the error logs. (I've tried a standard System.out.println as well, but I'm not sure where the console output would be written to) I've seen examples where the logger is instantiated via "final ComponentLog logger = getLogger()" and other examples that suggest the getLogger() is inherented from the "AbstractProcessor" thus no instantiation is required. What is further confusing that if I try the above I see an no such method errors on the 'org.apache.nifi.logger.ProcessorLog' which I thought was deprecated. Our NiFi is running on HDF Version 2.0.1. Thanks of any assistance you can provide. ... View more

Thank you for the prompt reply. I'll your suggestions a try. ... View more