Member since
11-16-2015
892
Posts
649
Kudos Received
245
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4596 | 02-22-2024 12:38 PM | |
1206 | 02-02-2023 07:07 AM | |
2786 | 12-07-2021 09:19 AM | |
4012 | 03-20-2020 12:34 PM | |
13421 | 01-27-2020 07:57 AM |
04-05-2016
11:44 AM
2 Kudos
Even with the standalone JAR, ExecuteSQL will not work with Hive due to the Hive JDBC driver not implementing some of the JDBC API calls made by ExecuteSQL. There is a Jira case to add Hive support: https://issues.apache.org/jira/browse/NIFI-981. There are some workarounds described in the following nifi-users email (from the archive): https://mail-archives.apache.org/mod_mbox/nifi-users/201601.mbox/%3C43A01CF2-2CC6-4A01-8836-86B9B3BEA31B@gmail.com%3E
... View more
03-11-2016
06:33 PM
1 Kudo
Being the client OS superuser doesn't imply that user is a superuser on HDFS. You'd need to add a user to Hadoop with the same UID and GID as your client user, and make the Hadoop user a superuser on that system, or better yet, just give the user the permissions it needs for the desired folder(s): http://stackoverflow.com/questions/24184306/how-to-add-user-in-supergroup-of-hdfs-in-linux
... View more
03-11-2016
04:01 PM
3 Kudos
If NiFi is running as a user that can change permissons in HDFS, you should be able to use the following properties in PutHDFS to do the chown/chmod: Permissions umask Remote Owner Remote Group ...according to the documentation: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.PutHDFS/index.html
... View more
03-11-2016
01:51 PM
8 Kudos
If the expression refers to attributes on a flow file, you will need to pass a reference to the flow file into evaluateAttributeExpressions: FlowFile flowFile = session.get();
jsonObject.put("hostname", context.getProperty(ADD_ATTRIBUTE).evaluateAttributeExpressions(flowFile).getValue()); If the property contains an attribute name (rather than an Expression containing an attribute name) and you want the value from the flow file: jsonObject.put("hostname", flowFile.getAttribute(context.getProperty(ADD_ATTRIBUTE).getValue())); If the attribute value itself contains Expression Language and you want to evaluate it, take a look at the following class: org.apache.nifi.attribute.expression.language.Query
... View more
03-02-2016
10:00 PM
3 Kudos
I was able to generate and send JSON to InvokeHttp POST using a test endpoint (http://httpbin.org/post), it read the incoming flow file contents (the JSON) and correctly responded based on the input file. Does the InvokeHttp response make sense for what you intend to send via POST? If not, you may need to (if you haven't already) set the "mime.type" attribute to "application/json" before sending the flow file to InvokeHttp. This can be done with the UpdateAttribute processor (see my example template below). If GetFile still doesn't seem to work, perhaps try ListFile followed by FetchFile. Having said that, I wouldn't think this would make a difference, I suspect there's something else in the flowfile content and/or attributes (or a lack thereof) that causes InvokeHttp to not respond as intended. My template is available as a Gist (here).
... View more
02-29-2016
06:23 PM
3 Kudos
I don't think PostHTTP outputs the response as a flow file. If you are looking for the response to be in a flow file, try InvokeHttp (with HTTP Method property set to "POST") instead and send its "response" relationship to your PutHDFS processor.
... View more
02-23-2016
05:07 PM
3 Kudos
For replacing contents of an incoming flowfile, the write(FlowFile, StreamCallback) method of ProcessSession is probably your best bet. There is a corresponding section of the NiFi Developer's Guide. There is an example of this in the ReplaceText processor code under the AlwaysReplace strategy. As an alternative to creating a full-fledged custom processor, as of NiFi 0.5.0 there are the InvokeScriptedProcessor and ExecuteScript processors. You can use InvokeScriptedProcessor to use a scripting language like Groovy to provide a Processor implementation, which avoids the need for building and deploying your own NARs. If you don't need a full processor and are just looking to manipulate the flowfile content, you could use the ExecuteScript processor, which allows easy access to incoming flowfiles and such. I've written a blog post on using ExecuteScript (with Groovy) to replace flowfile contents: http://funnifi.blogspot.com/2016/02/executescript-processor-replacing-flow.html Regards, Matt
... View more
02-03-2016
06:43 PM
1 Kudo
That probably works but defeats the purpose of HA. Hadoop is supposed to dynamically resolve 'provaha' to any of the physical nodes in the HA cluster. I wonder if PutHDFS is not successfully getting the physical node resolved from the HA node name, or if it is an error in config (core-site.xml)
... View more
01-12-2016
02:08 PM
4 Kudos
There sure is! I am currently working on NIFI-1275 to implement FetchElasticsearch and PutElasticsearch, the former uses the Get API and the latter uses the Bulk Insert API. Are there any particular features you're interested in having supported? We are hoping to offer support for secure Elasticsearch instances, but may not be able to provide out-of-the-box support due to licensing/distribution issues. Querying (aka the Search API) is not currently planned under NIFI-1275 but the InvokeHttp processor can perform the desired query, then once the document IDs are retrieved, FetchElasticsearch can be used to get the documents' contents.
... View more
11-22-2015
03:31 AM
I see at least two use cases: I know <non-Java language here> but I want to write a processor for NiFi. There is no (current) processor or workflow that takes care of my needs, so I’d like to write code to fill in the blanks( basically the body of an onTrigger() call ) Hot-reload is common to both. Adding new languages on-the-fly is not (IMHO). For "Invocable" script engines (like Jython) that allow you to compile an implementor of Processor, would be nice to call into those scripts and treat them like first-class Processors. For other scripts / script engines, it would be cool to treat the script body as the onTrigger() API call -- to be executed when scheduled and when a FlowFile is available.
... View more
- « Previous
- Next »