question Re: hadoop files list sorted by time in Archives of Support Questions (Read Only)

hadoop files list sorted by time

bigdata_superno — Mon, 22 Aug 2016 21:02:25 GMT

Hi,

I am trying to run a very simple command

hdfs dfs -ls -t /

However, it prompts me saying that -t is an illegal option. However, when I look for documentation it says -t is supported. FYI, I am using Hadoop 2.7.1 version. Any idea how to list the files / directories in HDFS sorted by time?

Re: hadoop files list sorted by time

shivkumar82015 — Mon, 22 Aug 2016 21:04:44 GMT

Please check option as below .

hdfs dfs -ls /tmp | sort -k6,7

hadoop 2.7.x ls command , the below following options available :

Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args>

Options:
-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.

Re: hadoop files list sorted by time

sandyy006 — Mon, 22 Aug 2016 21:05:35 GMT

@bigdata.neophyte

As per this jira : https://issues.apache.org/jira/browse/HADOOP-8934

-t option is supported from hadoop 2.8.

Possibly a doc bug in apache doc here.

Re: hadoop files list sorted by time

sshimpi — Mon, 19 Aug 2019 11:05:49 GMT

@bigdata.neophyte

From below screenshot we see that the "-t" is not more with current version of hadoop.

You can try - hdfs dfs -ls -R / | sort -k6,7

Check - http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201108.mbox/%3CCAFDdpB10sHYdaJC4JRcHHrcrdiOj4LBU2OZboabh2H3L3UjbEw@mail.gmail.com%3E

Re: hadoop files list sorted by time

rguruvannagari — Mon, 22 Aug 2016 21:18:18 GMT

@bigdata.neophyte

-t option seems to not available in 2.7.1 version which is included with HDP 2.4.2.

This option is included in hadoop 2.7.3 which is available with HDP 2.5 Tech preview .

[hdfs@hdp1 ~]$ hadoop version
Hadoop 2.7.3.2.5.0.0-1133
Subversion git@github.com:hortonworks/hadoop.git -r 93bf28063ef319be6833d3d6f117d44e0b6b8fa9
Compiled by jenkins on 2016-08-03T11:38Z
Compiled with protoc 2.5.0
From source with checksum 1aed9e48ca6f7cd4ada3a36b1cd5feb
This command was run using /usr/hdp/2.5.0.0-1133/hadoop/hadoop-common-2.7.3.2.5.0.0-1133.jar
[hdfs@hdp1 ~]$ hdfs dfs -ls -t -help
-ls: Illegal option -help
Usage: hadoop fs [generic options] -ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [<path> ...]

[hdfs@hdp1 ~]$ hdp-select versions
2.5.0.0-1133