Hadoop2.8.0以下版本,内置是不支持按照时间等属性排序的。 我们可以结合Shell命令来实现按照文件最近修改的时间对 ls 命令输出结果进行排序。比如,按照最近修改的时间降序排序(也就是越早修改的文件显示在最下面)
代码语言:javascript复制hdfs dfs -ls /spark2-history | sort -r -k6,7
为啥是-k6,7
,因为第6,7个字段是时间字段。
-rwxrwx--- 2 ps_pro hadoop 1809416 2020-12-27 10:22 /spark2-history/application_1608084068201_400785_1
-rwxrwx--- 2 ap_poc hadoop 107590 2020-12-27 10:22 /spark2-history/application_1608084068201_400797_1
-rwxrwx--- 2 fcv hadoop 113705 2020-12-27 10:22 /spark2-history/application_1608084068201_400783_1
-rwxrwx--- 2 datamart_pro hadoop 991453 2020-12-27 10:22 /spark2-history/application_1608084068201_400780_2
-rwxrwx--- 2 datamart_pro hadoop 986122 2020-12-27 10:22 /spark2-history/application_1608084068201_400795_1
-rwxrwx--- 2 datamart_pro hadoop 80683 2020-12-27 10:22 /spark2-history/application_1608084068201_400711_2
-rwxrwx--- 2 rdvs hadoop 163833 2020-12-27 10:21 /spark2-history/application_1608084068201_400779_2
Found 60153 items
效果等同于2.8.0以上的
代码语言:javascript复制hdfs dfs -ls -t /spark2-history
Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args> Options: -d: Directories are listed as plain files. -h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864). -R: Recursively list subdirectories encountered. -t: Sort output by modification time (most recent first). -S: Sort output by file size. -r: Reverse the sort order. -u: Use access time rather than modification time for display and sorting.