question Re: Cloudbreak communication with public IPs in Archives of Support Questions (Read Only)

Cloudbreak communication with public IPs

sunile_manjee — Tue, 03 May 2016 02:46:12 GMT

When does cloudbreak communicate with public IPs? I assume at some point it fetches the public repos. For example a security team does not want the process to communicate with any public IPs. What are the work arounds to handle this scenario on cloudbreak?

Re: Cloudbreak communication with public IPs

rdoktorics — Tue, 03 May 2016 02:48:20 GMT

@Sunile Manjee What cloud are they using?

Re: Cloudbreak communication with public IPs

sunile_manjee — Tue, 03 May 2016 03:00:29 GMT

@rdoktorics AWS

Re: Cloudbreak communication with public IPs

rdoktorics — Tue, 03 May 2016 03:07:19 GMT

The next release will include a feature when you can deploy nodes into an existing subnet which not able to assign public ip-s to the machines. So the machines can reach the internet on a NAT gateway and you can login into the machines with VPN connection.

Re: Cloudbreak communication with public IPs

sunile_manjee — Tue, 03 May 2016 03:10:37 GMT

@rdoktorics thanks for that info. I need to know which public IPs does cloudbreak hits, what is it pulling, and if it can be prevented by using local repos instead.

Re: Cloudbreak communication with public IPs

rkovacs — Tue, 03 May 2016 20:12:08 GMT

@Sunile Manjee,

what do you mean under "which public IPs does cloudbreak hits"?

- Cloudbreak <-> internet?

- Cloudbreak <-> cluster?

- Installed cluster <-> internet?

So the question is, is it possible to create cluster with Cloudbreak without internet connection? Am i right?

In short it isn't possible. In long there would be too many limitations, and you have to prepare well your local repos and others.

Re: Cloudbreak communication with public IPs

sunile_manjee — Wed, 04 May 2016 02:18:37 GMT

@rkovacs basically can cloudbreak use local repository instead of fetching them. Do I understand correctly that is not possible?

Re: Cloudbreak communication with public IPs

sunile_manjee — Wed, 04 May 2016 02:21:20 GMT

@rkovacs @rdoktorics I need to know all the repos cloudbreak fetches and where does it fetch it from for security reasons.

Re: Cloudbreak communication with public IPs

rkovacs — Wed, 04 May 2016 14:57:28 GMT

@Sunile Manjee,

On the Cloudbreak side there are a two things which require internet connection:

- SSSD configuration

- Public recipes

So if you skip them Cloudbreak should works as well. Ambari does the others. In Cloudbreak you could configure HDP repository, so if you create a huge local repo which contains everything related to Ambari it should work. For more please ask Ambari team, because Ambari installs many things in runtime, for example updates and patches.

Re: Cloudbreak communication with public IPs

sunile_manjee — Thu, 05 May 2016 02:22:09 GMT

@rkovacs Where would the repos be loaded in advance? The only node which is static on cloudbreak is the deployer node. How would the instances launched by cloudbreak utilize repos which exist on the deployer node?

Re: Cloudbreak communication with public IPs

rkovacs — Thu, 05 May 2016 02:32:24 GMT

@Sunile Manjee Cloudbreak writes the configured repo next to Ambari if there is, and Ambari does the install on it's own way. We don't have repository on the deployer node (we don't have repos at all), you have to install yours or use public repo.

Re: Cloudbreak communication with public IPs

sunile_manjee — Thu, 05 May 2016 02:34:55 GMT

@rkovacs forgive me I am not following you. I don't see any option during a cluster deployment via cloudbreak to have ambari use local repos. Where may I find more (and clear) instructions on deploying a cluster via cloudbreak, configuring cluster to use local (not fetching repos) repos?

what does this mean: "Cloudbreak writes the configured repo next to Ambari if there is, and Ambari does the install on it's own way."

Re: Cloudbreak communication with public IPs

rkovacs — Mon, 19 Aug 2019 10:06:59 GMT

You should find under advanced options.

So if you create a local repository, Ambari could use it.

Re: Cloudbreak communication with public IPs

sunile_manjee — Thu, 05 May 2016 03:23:10 GMT

@rkovacs well there you go! Nice. so this solves 1 of 2. the second part is does cloudbreak reach out to any other public ips for any activity? basically if i create a VPC and harden the security group to have zero access to any public IP will the cluster launch and be usable? I guess I can test this as well.

Re: Cloudbreak communication with public IPs

rkovacs — Thu, 05 May 2016 23:12:36 GMT

@Sunile Manjee There is one more thing what you have to know. Cloudbreak deployer also requires internet connection for example

- in the first time when downloading dependencies and containers

- some commands like 'cbd doctor', cbd upgrade' and 'cbd version'

- during upgrade because it downloads dependencies and containers

So to cut totally from the internet you need an ami where everything is preconfigured with a fixed version of Cloudbreak and deployer.

Re: Cloudbreak communication with public IPs

sunile_manjee — Fri, 06 May 2016 09:30:02 GMT

@rkovacs that totally makes sense. however if I setup local repos like you showed, can cloudbreak (outside of the deployer stuff you explained above) launch a cluster without internet connection?

Re: Cloudbreak communication with public IPs

rkovacs — Fri, 06 May 2016 12:31:46 GMT

Yes, Cloudbreak can install a cluster without internet connection if you set the local repo, skip SSSD, and live with limitation of recipes. But somehow you have to install Cloudbreak itself, which is impossible withoout internet connection. Easiest way to install and manage Cloudbreak installation is to use Cloudbreak deployer. But some of the commands are dowloading content, like init, upgrade, version, doctor. So first you have to install deployer with internet connection, for example you create an AMI which contains an already installed deployer and cloudbreak, and than in the secured network you start an instance of the created AMI.