Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

transfer On premises file to AWS S3 using nifi

avatar

Hi,

I am currently working on a POC trying to send files from my local on-premises server to AWS S3. It is giving me error unable to execute HTTP request: Network is unreacheable

5 REPLIES 5

avatar
Rising Star

Are you able to ping the S3 or access the S3 endpoint from you NiFi server ?

If your NiFi server is on private subnet with no EIP attached you need to create an VPC endpoint to your S3 service

If this you case follow this link :https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/

avatar

Actually my nifi is on premises , anyways issue got fixed. need to provide proxy details like http url and port number in nifi. Its great tool which makes you to connect your on premises to cloud which you can send files. Right now i am working with smaller ones ,need to see how it behaves with Larger volumes

avatar
Explorer

Hi guys, I have the same task, I need to transfer On premises file (in hdfs) to AWS3 using Nifi, what are the processors you're using to complete the task?

 

Thank you so much.

avatar

You could easily use a combination of GetHDFS - PutS3Object. You could either user ListHDFS + FetchHDFS and save into S3 using PutS3Object

avatar
New Contributor

I need to transfer On premises file (in hdfs) to AWS3 using Nifi, what are the processors you're using to complete the task?