Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI - Processing HDFS

avatar
Explorer

Hi,

I am new to NIFI. Got a requirement to read lzo files from HDFS apply some filter and write into local file system.

 

Could you let me know the list of components I should place please. I am struck how to convert HDFS  flow into normal. 

Any help would be really appreciated. Thanks in advance .

 

Thanks,

Kumar

3 REPLIES 3

avatar
Contributor

Hi....you can use GetHDFS to retrieve the files from HDFS. Use RouteonAttribute if you wish to apply some filter on the flowfile attribute and finally PutFile to save the file on local machine. 
Hope this helps. If the comment helps you to find a solution or move forward, please accept it as a solution for other community members.

avatar
Explorer

Thanks Sagar. My HDFS files are "lzo" files. Could you suggest me how to uncompress (What processors to use) before writing it into local file system please. 

avatar
Contributor

Hi...you can use compress content to decompress. I am not 100% if it decompresses lzo files. If not, you can executestreamcommand to run a shell command to uncompress the files. Hope this helps.