Member since
04-16-2018
3
Posts
0
Kudos Received
0
Solutions
04-16-2018
07:35 AM
Thanks for your reply Harsh. I am able to get the file record counts, but cant get the filenames to append to them. Any idea how we can tweak your code for simple text files to have the filenames as well? Thanks, Surendran
... View more
04-16-2018
06:53 AM
Hi Gurus, We have an S3 location with multiple directories and files. We would like to get the filename and their corresponding record count. We were able to get the filename and filesize using the below command: hdfs dfs -ls -R /bucket_name/* | awk '{system("hdfs dfs -count " $8) }' | awk '{print $4,$3;}' Output: /bucket_name/Directory/File_name.txt 44998 --filesize Thanks in advance! Regards, Surendran
... View more
Labels:
- Labels:
-
HDFS