Member since
11-16-2015
911
Posts
668
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 714 | 09-30-2025 05:23 AM | |
| 1077 | 06-26-2025 01:21 PM | |
| 936 | 06-19-2025 02:48 PM | |
| 1106 | 05-30-2025 01:53 PM | |
| 12302 | 02-22-2024 12:38 PM |
02-28-2017
06:04 PM
What do you mean by receiving attributes from a URL? Do you mean query parameters? Or are you asking if there is a way to combine the fetching of the document via the URL and the extraction of JSON fields from the response? If the latter, there isn't a way to combine those currently; NiFi is a highly modular system so fetching is really a different operation than extractions, hence the two separate processors. What do you see as the issue in receiving the whole JSON response? The third-party libraries I'm familiar with still retrieve the whole HTTP response whether they expose it via an input stream or a big string or whatever. After the initial fetch/response, if you don't touch the content after that, there won't be an operational impact with the respect to the size of the JSON response. If you no longer need the content, you can replace it with something else (or nothing) using ReplaceText. One concept that is (hopefully) becoming more popular with web services is support for GraphQL. Using that paradigm and query language, you could ask for just the fields/structures you want, and you get the response you expect.
... View more
02-28-2017
05:46 PM
3 Kudos
I don't think there is any other way, besides ExecuteScript or ExecuteStreamCommand (both of which require the HBase client libraries / programs, the former of which is already in the HBase NAR). I have added NIFI-3538 to track the addition of DeleteHBase processor(s).
... View more
02-23-2017
07:41 PM
1 Kudo
As of NIFI-3418, NiFi will allow the user to set both of the aforementioned properties.
... View more
02-21-2017
06:00 PM
1 Kudo
It seems feasible to remove this restriction from Date and Time data types, as was done for Timestamps in NIFI-3430. Please feel free to write up a Jira case for this improvement. In the meantime, if you are trying to do something like use the value '2017-02-21' in a field specified as Date/Time type, you could first change the value (possibly in an UpdateAttribute processor) using NiFi Expression Language's toDate() function. So if you have an attribute such as 'my.date.string' set to '2017-02-21' you could set an attribute 'my.date' equal to the following: ${my.date.string:format('yyyy-MM-dd')}
... View more
02-21-2017
04:13 PM
2 Kudos
FetchFile employs the following outgoing relationships: success Any FlowFile that is successfully fetched from the file system will be transferred to this Relationship. not.found Any FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship. permission.denied Any FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship. failure Any FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship. You could handle the non-success cases by routing the other relationship(s) back to FetchFile. Each non-success flow file will be penalized (the default is 30 seconds but is configurable). Then when processing the file no longer causes an error, it should be successfully processed and transferred to the success relationship.
... View more
02-16-2017
04:28 PM
1 Kudo
ExecuteScript creates a new ScriptEngine for each one of the tasks specified in the Max Concurrent Tasks property, and reuses those engines for each flow file. ExecuteScript basically allows you to implement an onTrigger() method using a scripting language, so it doesn't provide for other lifecycle things like setup and shutdown. For that you can use InvokeScriptedProcessor, there's a little more boilerplate as you must implement a subclass of Processor, but in return you can override the initialize() method to connect to the DMC once, as well as provide any number of extra properties and relationships to the "parent" InvokeScriptedProcessor for configuration. I have some examples on my blog including this one. In general, I should mention that the Jython engine is relatively slow anyway, so you won't see great performance from it. You can get better performance by porting to Groovy or Javascript if possible.
... View more
02-13-2017
06:44 PM
1 Kudo
For this, I am assuming that you have a property called "nifi.prefix.cat" defined in your Variable Registry (custom.properties, e.g.): nifi.prefix.cat=my_filename.txt Then assuming a flow file comes into UpdateAttribute and has the "suffix" property set to "cat", you can add a dynamic property called "nifi.filename" set to:
${${literal('nifi.prefix.'):append(${suffix})}} This should give you an attribute called "nifi.filename" set to "my_filename.txt". Please let me know if I've understood what you are trying to do, and I'll edit this as needed.
... View more
02-10-2017
09:30 PM
Try explicitly setting the Return Type as 'json' rather than 'auto-detect' or 'scalar'.
... View more
02-03-2017
07:55 PM
1 Kudo
I was able to get such a script working with those json-lib classes. I had different versions of some of those libraries though, your issue might be from using commons-collections-3.2.2 instead of 3.2.1. I started with the dependencies (and versions) listed here, and only downloaded what I needed to get things compiling: I set the Module Directory property to the folder containing all the JARs (versus an entry for each JAR): /Users/mburgess/Downloads/json-lib Here is the sample script I used (it's not pretty but compiles and runs): import net.sf.json.*
import net.sf.json.xml.*
class POGO {
String a
List<String> b
Map<String, Integer> c
}
def js = new JSONSerializer()
def xs = new XMLSerializer()
def flowFile = session.get()
if(!flowFile) return
def p = new POGO(a: "Hello", b: ["I", "am", "a", "list"], c: ['k1':1, 'k2':2])
def j = js.toJSON(p)
def x = xs.write(j)
flowFile = session.putAttribute(flowFile, 'new.value', x)
session.transfer(flowFile, REL_SUCCESS) This ignores the incoming flow file content and creates an Object which is transformed to JSON then XML (I wanted to exercise the toJSON() and write() methods), then puts the XML in an attribute (to make the example easier) and sends the flow file on.
... View more