In this section, we're going to assume you have created a cluster or instances on your private network, but only one of those instances has a public floating IP address. This means that all of those instances can connect happily between each other on the private network (via their 192.168.x.x address), but you can only connect to one of them directly (via a 146.118.x.x address). Ideally you want a way to access and manage that cluster as a whole. In this case, we have a test cluster of 5 instances:
|Instance Name||Private Address||Public Address|
Of course there is nothing stopping you from assigning a public IP address to every instance in your cluster, however you may prefer to keep external exposure to a minimum, especially if the bulk of the work the cluster will be doing is on the private network.
All 5 instances should have your SSH key loaded onto them, however only one (test-instance-1) you will be able to SSH to directly from your desktop:
In theory, you should be able to SSH from test-instance-1 to any of the other instances. However, as the instances are by default passwordless, you will need to use your SSH key to connect to them. There are a couple of different ways this can be done.
Copy SSH Key
The most direct approach is to manually copy your SSH key (both the private and public components) from your desktop to test-instance-1. That way you can use your credentials directly when SSHing to any other instance in the cluster:
The main drawback with this approach is that you may not want to have your SSH key stored on more machines than necessary, from a security perspective.
SSH Agent Forwarding
If you have a Linux or Mac based desktop, you can instead forward your SSH credentials from your desktop through your SSH connection to test-instance-1. This is done using ssh-agent. The exact process may vary depending on your specific operating system; the steps outlined below are for setting this up under Ubuntu 16.04.
1) If it isn't already started, run ssh-agent on your desktop:
2) You can use ssh-add to check that ssh-agent is running correctly. If it reports not being able to connect to an authentication agent, most likely it is because the environmental variable SSH_AUTH_SOCK has not been exported correctly:
To correct this, manually copy and run the first line of output when you first ran ssh-agent (without the semi-colon at the end) to set up SSH_AUTH_SOCK:
It is also worth confirming that the temporary socket this points to actually exists:
3) With ssh-agent up and running correctly, you can now add your SSH key. By default, the list of cached SSH keys should be empty:
Add your SSH key (in this example id_rsa):
4) If ForwardAgent hasn't already been enabled for SSH on your desktop (normally this would be found in /etc/ssh/ssh_config if it is), you will need to enable it locally for your account in ~/.ssh/config (create the file if it doesn't already exist). While you can enable it for all systems you SSH to, ideally you should restrict it to test-instance-1:
Also ensure that this config file isn't world-writable:
5) Now when you SSH to test-instance-1, not only will it cache your passphrase for your SSH key (if it has one), but you will also be able to SSH from test-instance-1 to any other instance in your private cluster as if your SSH credentials were stored locally on that instance:
It is probably also worth adding all of the instances in your private cluster to /etc/hosts on test-instance-1. That way, connecting to them will be simpler. Just add the entries to the end of your hosts file:
You can test this easily:
When running commands across all instances in the cluster, there are a number of tools that allow you to run parallel commands. One of the simplest ones is pdsh, which you only need to install on test-instance-1:
Once installed, you should configure pdsh to use SSH by default for rcmd, done by creating the file /etc/pdsh/rcmd_default (this will need to be done as root) which only contains "ssh":
You should also create a genders file (again as root) that contains the names of all of the other instances in the cluster:
Now you can run "pdsh -a" to run a specific command across all instances simultaneously:
- No labels