Member since
04-24-2016
2
Posts
0
Kudos Received
0
Solutions
04-24-2016
09:54 AM
excellent! very helpful thanks! just a follow up on the verification thing. It seems to me that, in addition to that, hdfs compares the checksums (of the local vs hdfs copy) to assert that the download is finished. is that correct?
... View more
04-24-2016
09:11 AM
Hello everybody! Sometimes my computer crashes, while the data is transferring to hadoop via the command hdfs dfs -put myfile myfolder My question is: does hdfs verifies that the transfer went OK? For instance, by automatically comparing the size of the data on the local drive to the amount of data received? I am asking these because I am transferring very large files (200 GB) and the transfer take hours. It is not easy for me to check whether the files in hdfs are the correct ones or some partial versions of my local files (due to some interrupted transfer). Many thanks!
... View more
Labels:
- Labels:
-
HDFS