Reading files from s3 bucket sub folders

[#1]

Hi all,

I am trying to read the files from s3 bucket (which contain many sub directories). As of now i am giving the phyisical path to read the files. How to read the files without hard coded values.

File path : S3 bucket name/Folder/1005/SoB/20180722_zpsx3Gcc7J2MlNnViVp61/JPR_DM2_ORG/ *.gz files

"S3 bucket name/Folder/" this path is fixed one and client id(1005) we have to pass as a parameter.

Under Sob folder, we are having monthly wise folders and I have to take only latest two months data.

Please help me how to read the data without hard-coded.

Many thanks for your help.

[#2]

Hi Aditya,

Thanks a lot for your help. Is it possible to do in scala? As i dont have knowledge on python.

[#3]

@Lakshmi Prathyusha,

I'm not sure of how to do this in Scala. I guess you may have similar date time functions in Scala as well. You can apply this logic in Scala.