ca8a9c600a
* Dynamically retrieve aws_bastion_ami latest reference by querying AWS rather than hard coded * Dynamically retrieve the list of availability_zones instead of needing to have them hard coded * Limit availability zones to first 2, using slice extrapolation function * Replace the need for hardcoded variable "aws_cluster_ami" by the data provided by Terraform * Move ami choosing to vars, so people don't need to edit create infrastructure if they want another vendor image (as suggested by @atoms) * Make name of the data block agnostic of distribution, given there are more than one distribution supported * Add documentation about other distros being supported and what to change in which location to make these changes |
||
---|---|---|
.. | ||
docs | ||
modules | ||
templates | ||
.gitignore | ||
create-infrastructure.tf | ||
credentials.tfvars.example | ||
output.tf | ||
README.md | ||
terraform.tfvars | ||
terraform.tfvars.example | ||
variables.tf |
Kubernetes on AWS with Terraform
Overview:
This project will create:
- VPC with Public and Private Subnets in # Availability Zones
- Bastion Hosts and NAT Gateways in the Public Subnet
- A dynamic number of masters, etcd, and worker nodes in the Private Subnet
- even distributed over the # of Availability Zones
- AWS ELB in the Public Subnet for accessing the Kubernetes API from the internet
Requirements
- Terraform 0.8.7 or newer
How to Use:
- Export the variables for your AWS credentials or edit
credentials.tfvars
:
export AWS_ACCESS_KEY_ID="www"
export AWS_SECRET_ACCESS_KEY ="xxx"
export AWS_SSH_KEY_NAME="yyy"
export AWS_DEFAULT_REGION="zzz"
-
Rename
contrib/terraform/aws/terraform.tfvars.example
toterraform.tfvars
-
Update
contrib/terraform/aws/terraform.tfvars
with your data. By default, the Terraform scripts use CoreOS as base image. If you want to change this behaviour, see note "Using other distrib than CoreOs" below. -
Allocate a new AWS Elastic IP. Use this for your
loadbalancer_apiserver_address
value (below) -
Create an AWS EC2 SSH Key
-
Run with
terraform apply --var-file="credentials.tfvars"
orterraform apply
depending if you exported your AWS credentials
Example:
terraform apply -var-file=credentials.tfvars -var 'loadbalancer_apiserver_address=34.212.228.77'
-
Terraform automatically creates an Ansible Inventory file called
hosts
with the created infrastructure in the directoryinventory
-
Ansible will automatically generate an ssh config file for your bastion hosts. To connect to hosts with ssh using bastion host use generated ssh-bastion.conf. Ansible automatically detects bastion and changes ssh_args
ssh -F ./ssh-bastion.conf user@$ip
- Once the infrastructure is created, you can run the kubespray playbooks and supply inventory/hosts with the
-i
flag.
Example (this one assumes you are using CoreOS)
ansible-playbook -i ./inventory/hosts ./cluster.yml -e ansible_ssh_user=core -e bootstrap_os=coreos -b --become-user=root --flush-cache
Using other distrib than CoreOs If you want to use another distribution than CoreOS, you can modify the search filters of the 'data "aws_ami" "distro"' in variables.tf.
For example, to use:
-
Debian Jessie, replace 'data "aws_ami" "distro"' in variables.tf with data "aws_ami" "distro" { most_recent = true
filter { name = "name" values = ["debian-jessie-amd64-hvm-*"] }
filter { name = "virtualization-type" values = ["hvm"] }
owners = ["379101102735"] }
-
Ubuntu 16.04, replace 'data "aws_ami" "distro"' in variables.tf with data "aws_ami" "distro" { most_recent = true
filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-*"] }
filter { name = "virtualization-type" values = ["hvm"] }
owners = ["099720109477"] }
-
Centos 7, replace 'data "aws_ami" "distro"' in variables.tf with data "aws_ami" "distro" { most_recent = true
filter { name = "name" values = ["dcos-centos7-*"] }
filter { name = "virtualization-type" values = ["hvm"] }
owners = ["688023202711"] }
Troubleshooting
Remaining AWS IAM Instance Profile:
If the cluster was destroyed without using Terraform it is possible that
the AWS IAM Instance Profiles still remain. To delete them you can use
the AWS CLI
with the following command:
aws iam delete-instance-profile --region <region_name> --instance-profile-name <profile_name>
Ansible Inventory doesnt get created:
It could happen that Terraform doesnt create an Ansible Inventory file automatically. If this is the case copy the output after inventory=
and create a file named hosts
in the directory inventory
and paste the inventory into the file.
Architecture
Pictured is an AWS Infrastructure created with this Terraform project distributed over two Availability Zones.