transfer On premises file to AWS S3 using nifi

[#1]

Hi,

I am currently working on a POC trying to send files from my local on-premises server to AWS S3. It is giving me error unable to execute HTTP request: Network is unreacheable

[#2]

Are you able to ping the S3 or access the S3 endpoint from you NiFi server ?

If your NiFi server is on private subnet with no EIP attached you need to create an VPC endpoint to your S3 service

If this you case follow this link :https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/

[#3]

Actually my nifi is on premises , anyways issue got fixed. need to provide proxy details like http url and port number in nifi. Its great tool which makes you to connect your on premises to cloud which you can send files. Right now i am working with smaller ones ,need to see how it behaves with Larger volumes

[#4]

Hi guys, I have the same task, I need to transfer On premises file (in hdfs) to AWS3 using Nifi, what are the processors you're using to complete the task?

Thank you so much.

[#5]

You could easily use a combination of GetHDFS - PutS3Object. You could either user ListHDFS + FetchHDFS and save into S3 using PutS3Object

[#6]

I need to transfer On premises file (in hdfs) to AWS3 using Nifi, what are the processors you're using to complete the task?