Member since
05-20-2016
5
Posts
2
Kudos Received
0
Solutions
05-23-2016
12:30 AM
Thank you very much. I have three follow-up questions:
a. Do I have to use user's (admin) access key and secret key or AWS access key?
b. Will this command copy whole S3 bucket without specifying folders or file names?
P.S.: I want to copy whole bucket as it is. I have many sub files and sub folders and it would consume a lot of time to specify each one of them one by one.
c. I am little confused how I am supposed to specify the dest and path. Any examples please? Thanks. Apologies for naive questions.
... View more
05-23-2016
12:22 AM
Thank you all for the response.
... View more
05-23-2016
12:19 AM
1 Kudo
Hi Pradeep, Thank you for your prompt response and apologies for the delayed reply. I added this code in my hdfs-site.xml and hive-site.xml files. For access key and secret access key, I used the AWS access key and secret key respectively. However, I am a little confused if I was supposed to use the user's (admin) access key and secret key or the AWS access and secret key? Code changed in xml files is as follows: <property>
<name>fs.s3n.awsAccessKeyId</name>
<value>xxxxxxxxxxxx</value>
</property>
<property>
<name>fs.s3n.awsSecretAccessKey</name>
<value>yyyyyyyyy</value>
</property> After editing .xml files using vi, I simply restarted my sandbox i.e. my virtual box m/c. Image is attached below:
This virtual box is basically stuck since more than 2 days. I tried restarting the sandbox 3-4 times. But no success!
... View more
05-20-2016
05:08 AM
Hi All,
I am a beginner in Hadoop and cloud computing. How can I migrate/transfer/copy AWS Data from volumes or S3 to a Sandbox with HDP 2.4 in the most easiest and efficient way. I actually want to close my AWS account and need to transfer my cloud data to a local hadoop cluster or in the sandbox.
P.S.: My data is ~10 GB. I am using t2. small instance in AWS.
Any suggestions with steps would be highly appreciated.
Thank you in advance.
... View more
05-20-2016
04:56 AM
1 Kudo
Hi All, I was trying to migrate AWS data files to my sandbox. I followed this: https://community.hortonworks.com/articles/25578/how-to-access-data-files-stored-in-aws-s3-buckets.html After I configured hdfs-site.xml and hive-site.xml files and tried to restart the sandbox, my sandbox is stuck and not booting the resource manager. I am unsure what the actual reason is. Can some one please throw light on this?
Thank you in advance.
... View more