Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Get latest file from directory

avatar
New Contributor

i have a dir /automation_test/oozie/output/Bigdata_Counts/.i have multiple files in this dir.I want to read only latest file from this directory

2 REPLIES 2

avatar
Master Mentor

@chinni 

Please elaborate your question else the simplest way to get the latest file in Linux

 

$ ls -lrt /automation_test/oozie/output/Bigdata_Counts/

 

But I guess you want to do that programmatically?

avatar
New Contributor

I want latest file from hdfs dir.I am using below programm,but it get me all files into dataframe.I want to filter latest file and read only latest file.I am using scala

val  fname=spark.read.csv("hdfs://ndwns001.ndw.leidos.com/automation_test/oozie/output/Bigdata_Counts/")