- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Create files from GetHDFS processor flowfiles
- Labels:
-
Apache NiFi
Created 09-27-2017 02:24 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
i would like to create a Nifi flow that parses a number of files from HDFS with GetHDFS processor, creates a separate file for each incoming flowfile from HDFS with the same name but different extension and finally write both the original HDFS file and the newly created file to a remote location via FTP. Which Nifi processor would be suitable for this task?
The flow i imagine is as following:
GetHDFS ----> *Nifi processor* --> PutFTP
Created 09-27-2017 03:59 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would love to make this flow complicated.
> What if one putftp is success and another one failed. 🙂 do u want one to be success and other failed . or do you want to handle it like a retry
Created 09-27-2017 08:35 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Interesting, definitely the latter i would like to handle it as a retry. The files should be transfered only in pairs, how would you approach it?
Created 09-27-2017 04:01 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
One more point use listHDFS and putHDFS than getHDFS . If you are using TDE then there is a bug in getHDFS which can happen in rare scenarios.
Created 10-12-2017 02:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- « Previous
-
- 1
- 2
- Next »