Posts: 1,903
Kudos: 435
Solutions: 307
Registered: ‎07-31-2013
Re: Handling Distcp of large files between 2 clusters
Are you using hftp:// or webhdfs://? I'd recommend trying with the latter.

For this specific exception in REST based copies, its usually not a fault with the network but a buggy state in the older Jetty used on the source cluster. Typically a rolling restart of the DataNodes will help resolve such a bad state of Jetty where it hangs up on a client mid-way during a response, causing the sudden EOF to the copying client in DistCp when it was expecting the rest of data.

View solution in original post

Who Me Too'd this solution