Member since
01-14-2015
5
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1337 | 04-07-2016 09:02 AM |
04-07-2016
09:02 AM
As an alternative solution, I am comparing the two file structures for the same number of folders and files.
... View more
04-04-2016
02:30 PM
I copied a large folder structure from HDFS to RedHat using copyToLocal. While it looked successful, i want to validate it was copied correctly by checking the size of the data in HDFS and in RedHat. I'm using "du" but my numbers are still off. I run the following on RH: "du -s -b <PATH>" I run the following on HDFS: "hadoop fs -du -s <PATH>" I noticed that RH reports 101 bytes for empty folders while HDFS (CDH5.5.2) reports 0 bytes for empty folders. So my question is, how to I validate the entire directory of data was fully transferred? Thanks,
... View more
Labels:
- Labels:
-
HDFS