Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Help with downloading a file to local laptop

Help with downloading a file to local laptop

New Contributor

I would like to download some output files from PIG.  I am unsure how to do this.

I can use an SFTP program.

I don't know the username and password

The machine adddress is 10.0.2.15

Thanks

3 REPLIES 3

Re: Help with downloading a file to local laptop

Master Collaborator
The output from our Pig program is probably stored in HDFS (or possibly
another filesystem - depends on your Hadoop setup).

The easiest way to get them out would be with HDFS commands. For instance,
to put them in your current directory: "hdfs dfs -copyToLocal
/path/to/pig/output ./"

Re: Help with downloading a file to local laptop

New Contributor

How to I ftp to that server to get the data onto my laptop locally?

Re: Help with downloading a file to local laptop

Master Collaborator

Well you wouldn't use FTP unless you've set it up yourself. The simplest way to get it to your laptop would be to use the command I have above. It needs to be run on a machine that has the Hadoop configuration set up, so you may need to SSH to one of the machines running hadoop, copy the file from HDFS into the local file system, and the copy it to your laptop with scp (or pscp if you're using PuTTY, for instance).

 

If you find yourself having to do this a lot, there are ways to make it easier, such as mounting HDFS as an NFS drive. But that is not something Hadoop is configured to do by default, so I just mention it as something to look into if you do this often. To just get the files you need right now, going to that machine and running 'hdfs dfs -copyToLocal /path/to/pig/output ./', and then using scp

Don't have an account?
Coming from Hortonworks? Activate your account here