Hi to myself and all the relevant viewers,
I solved the problem by allowing the ports in AWS. There are many but basically I allowed the following. For the full list refer to the documentation.
I will list the steps I followed to make Cloudera Quickstart work on AWS.
- First, downloaded CDH 5.3 Quickstart VMWare version.
- Download and setup AWS CLI Tools properly. Of course must have an AWS account and S3 storage.
- Tune up VM resources appropriately. Definitely use 8+ GB Ram and 2+ cores for this version of QuickStart.
- Since the downloaded version of VMWare VMDK is not uploadable to AWS. You must convert it to streamable (OVF) file type. I did that via VM Workstation trial version. In the end, you will have one solid VMDK file, not partial ones.
- Via AWS CLI command-line tool, use ec2-import-instance command to upload the instance to AWS EC2. It first uploads the file to S3. The region of S3 becomes important since it will create the instance in the same EC2 region with S3.
- Depending on your Internet upload speed, the upload time will vary. In Turkey it sucks in general so it took around 24 hours.
- If everything is allright so far, you will be able to see your instance created. But you may not connect to it if you did not have any keys created in the S3 region you got. It's all because AWS requires the instances to be created with your keys. This is an import without keys in my situation so it did not recognize it. I worked it around via stopping the instance and creating an image of it. Then terminating the existing one and launching it again from the image. Then it asks for keys; and then you simply create them. If your S3 and EC2 regions are same and you already have keys created in that EC2 region, then it should work though. I just missed that part.
- Then just start the instance and allow the ports above. You can connect to the CM via instance's public IP and port.
Also there is an alternative way. Maybe more useful and professional.
Tricks, tricks, tricks. It's so hard when you don't know but still exciting to learn from mistakes.
BR
Serhan
Hi to myself and all the relevant viewers,
I solved the problem by allowing the ports in AWS. There are many but basically I allowed the following. For the full list refer to the documentation.
I will list the steps I followed to make Cloudera Quickstart work on AWS.
- First, downloaded CDH 5.3 Quickstart VMWare version.
- Download and setup AWS CLI Tools properly. Of course must have an AWS account and S3 storage.
- Tune up VM resources appropriately. Definitely use 8+ GB Ram and 2+ cores for this version of QuickStart.
- Since the downloaded version of VMWare VMDK is not uploadable to AWS. You must convert it to streamable (OVF) file type. I did that via VM Workstation trial version. In the end, you will have one solid VMDK file, not partial ones.
- Via AWS CLI command-line tool, use ec2-import-instance command to upload the instance to AWS EC2. It first uploads the file to S3. The region of S3 becomes important since it will create the instance in the same EC2 region with S3.
- Depending on your Internet upload speed, the upload time will vary. In Turkey it sucks in general so it took around 24 hours.
- If everything is allright so far, you will be able to see your instance created. But you may not connect to it if you did not have any keys created in the S3 region you got. It's all because AWS requires the instances to be created with your keys. This is an import without keys in my situation so it did not recognize it. I worked it around via stopping the instance and creating an image of it. Then terminating the existing one and launching it again from the image. Then it asks for keys; and then you simply create them. If your S3 and EC2 regions are same and you already have keys created in that EC2 region, then it should work though. I just missed that part.
- Then just start the instance and allow the ports above. You can connect to the CM via instance's public IP and port.
Also there is an alternative way. Maybe more useful and professional.
Tricks, tricks, tricks. It's so hard when you don't know but still exciting to learn from mistakes.
BR
Serhan
