Member since
11-16-2015
911
Posts
668
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 704 | 09-30-2025 05:23 AM | |
| 1076 | 06-26-2025 01:21 PM | |
| 931 | 06-19-2025 02:48 PM | |
| 1103 | 05-30-2025 01:53 PM | |
| 12286 | 02-22-2024 12:38 PM |
10-23-2017
03:53 PM
Edited due to Matt Clarke's comment below: Support for Expression Language in SelectHiveQL properties was added to NiFi 1.3.0 under NIFI-3867. It is also available in HDF 3.0.x. One (less-than-ideal) workaround is to set the property to something invalid but identifiable (like "@HIVE_URL@") and have a script to replace that value in the template before uploading to a particular environment.
... View more
10-20-2017
02:55 PM
1 Kudo
In Apache NiFi 1.5.0 (not yet released at the time of this writing), SelectHiveQL (via NIFI-4473) will have a property to Normalize Names for Avro, so you won't have to do the alias.
... View more
10-19-2017
06:43 PM
I have a blog post that describes how to use ExecuteScript with Groovy and Sshoogr to execute remote commands via SSH. Not sure if this is what you're looking for but I thought I'd share in case it was useful.
... View more
10-18-2017
01:02 PM
IIRC, Groovy 3 is supposed to support the Java lambda syntax, but NiFi uses Groovy 2 which does not support it. However Groovy has always had Closures, which are very similar and used for the same purpose, so despite a small difference in syntax, you should be able to use Groovy closures the same as you would use Java lambdas. If you are referring to the Java Streams API (things like foreach() that take a lambda), Groovy has iterative and aggregate functions for that too, such as each() and eachWithIndex(), spread-dot and collect(), etc.
... View more
10-17-2017
06:32 PM
In order to distribute the fetch across the cluster, you will need ListSFTP -> Remote Process Group, the RPG should send to an Input Port on the same cluster, and that Input Port can be connected to the FetchSFTP. This RPG->Input Port connection will distribute the flow files containing the file names across the cluster, and each node's Input Port will receive a subset of those flow files, which will then be fetched in parallel across the cluster.
... View more
10-12-2017
06:40 PM
1 Kudo
Are you expecting a result set from the SELECT statement? If so then you might be better off with building a SQL query with the CLOB value inline (probably quoted and/or padded with rpad I imagine) and using ExecuteSQL vs PutSQL. The former is for queries that return result sets, the latter is for statements that don't (like INSERT).
... View more
10-11-2017
05:38 PM
1 Kudo
I have written NIFI-4479 to cover the addition of a DeleteMongo processor.
... View more
10-11-2017
04:29 PM
It's hard to tell from the formatting of your code above, but if you are executing session.remove(flowFile1) then later trying to transfer it to REL_SUCCESS, you will get that error. You can either change the logic to put the remove/transfer calls into an if-else block, or keep a boolean variable indicating whether the flow file has already been removed, and transfer if it has not been removed. It looks like you already have an if-clause checking the firstChild for "false", perhaps you could put the transfer in an else-clause.
... View more
10-09-2017
05:58 PM
What do you mean by "no convert happens?" PutParquet should write Parquet file(s) to the HDFS directory you configured in the processor. I believe the incoming flow file is the one transferred to the "success" relationship once the converted file has been successfully written to HDFS, not the converted file. For that I imagine there would have to be a ParquetRecordSetWriter and you'd use ConvertRecord instead of PutParquet.
... View more
10-09-2017
02:10 PM
How are you uploading the file? Processors like GetFile will set the filename attribute to the same name that was on disk.
... View more