Member since
11-16-2015
892
Posts
649
Kudos Received
245
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5222 | 02-22-2024 12:38 PM | |
1338 | 02-02-2023 07:07 AM | |
3004 | 12-07-2021 09:19 AM | |
4156 | 03-20-2020 12:34 PM | |
13955 | 01-27-2020 07:57 AM |
03-07-2017
08:00 PM
You need to keep the NIFi copy of the data even after writing a copy of it out via the putSFTP? If you need to retain a local copy of the data, route success twice form your putSFTP processor. So you should be able to do this with..... The UpdateAttribute processor can be used to update the filename by adding the following new property to its configuration: The Local copy of your Files remain unchanged down the success relationship to the left. The copy sent down the path to the right will have its content cleared, filename changed, and then sent via another PutSFTP. Thanks, Matt
... View more
03-02-2017
08:23 PM
1 Kudo
@nedox nedox You will want to use one of the available HDFS processors to get data form your HDP HDFS file system.
1. GetHDFS <-- Use if standalone NiFi installation
2. ListHDFS --> RPG --> FetchHDFS <-- Use if NiFI cluster installation
All of the HDFS based NiFi processors have a property that allows you to specify a path to the HDFS site.xml files. Obtain a copy of your core-site.xml and hdfs-site.xml files from your HDP cluster and place them somewhere on the HDF hosts running NiFi. Point to these files using the "Hadoop Configuration Resources" processor property. example: Thanks, Matt
... View more
02-06-2017
02:05 PM
Using commons-collections-3.2.1 solved the issue.Thanks, @Matt Burgess.
... View more
01-25-2017
08:42 PM
Correct, SelectHiveQL is for statements that return ResultSets (like SELECT *), those results are converted to Avro records. PutHiveQL is for executing statements (except callable statements like stored procedures) that do not return results, such as your ALTER TABLE example.
... View more
02-17-2017
05:44 AM
Thanks, now I can load from local
... View more
07-25-2019
03:44 PM
Hi, Matt! very useful and informative articles. Thank you veru much! Could you tell me how do I read content of a flowFile, transform it the way I like, and write the output to a new flowFile attribute (not back to the content)? I was trying to return transformation result from the callback but caught the error: None required for void return which is fairly expected behaviour - callback returns to the session.read function, but the latter does not return anything, I assume. So we get the flowFile itself and its content residing in different namespaces and I can't figure out how can I use content of the flowFile to place it into the attribute. Could you kindly help me, Matt?
... View more
05-15-2018
04:37 PM
Since this isn't related to the original question, please ask this as its own standalone question and I'd be happy to answer it. (The short answer is you might be able to use UpdateAttribute to change the 4 to the right column number for Table B if you can figure out whether a flow file is for Table A or B)
... View more
12-29-2016
08:41 AM
@Matt Burgess If the query returns too many rows in QueryDatabaseTable, can you set Max Rows Per Flow File and Fetch Size to something like 10000? -> yes, I have do that, but it doesn't work... I have 30 000 000 + data in postgreSQL so an exception of OutOfMemory is normal. I'm looking forward to your answer for PutElasticSearchHTTP.
... View more
06-07-2019
02:01 AM
Very interesting article, well done ! Have you done something similar with jython? I'm currently trying to use an execute script, with jython, to put key/values onto the distributed map cache but can't find a way to do it... BR, Paulo Rente
... View more
12-14-2016
02:45 PM
The UI uses the REST API so you can do it programmatically the same way: /flowfile-queues/{id}/drop-requests
... View more