- Ansible Amazon Base Repository
Ansible Amazon Base Repository is an MVP monorepo to rapidly develop scalable, reliable, high-quality components for Amazon Linux instance configuration management.
Note: This document assumes that you are working on Mac
-
Create a new virtual environment with pyenv
$ pyenv virtualenv miniconda3-latest ansible -
Activate your new python virtual environment
$ pyenv activate ansible -
Install poetry
$ conda install poetry -
Install dependencies
$ poetry install -
Check ansible
$ ansible --version
Code editors are major software development productivity tools. VSCode is a game changer.
VSCode should be configured for typical Python development with the following extensions:
- Ansible
- YAML
- Code Spell Checker
- Trailing Spaces
- Blank line at the End of File
- Markdown All in One
- Markdownlint
- Pylance
- Python
- Test Explorer
- Python Test Explorer
- Docker
- Remote SSH
- Remote Containers
- VirtualBox
- Vagrant
- Ruby
- Ruby Solargraph
- Github
- Github Actions
- Gitlens
VSCode should be configured to associate most of the files in the repository with Ansible. Check that your ansible files have Ansible set in the "Select Language Mode". In VSCode the difference between YAML and Ansible Language Mode is night and day. Example .vscode/settings.json
{
"files.associations": {
"kitchen*": "yaml",
"*.yml": "ansible"
},
}Python should be setup with the following:
"editor.renderWhitespace": "all",
"editor.rulers": [
80,
100,
120
],
"[python]": {
"editor.tabSize": 4,
"editor.insertSpaces": true,
"editor.formatOnSave": true
},
"[yaml]": {
"editor.insertSpaces": true,
"editor.tabSize": 2,
"editor.autoIndent": "none",
"editor.quickSuggestions": {
"other": true,
"comments": false,
"strings": true
},
"editor.formatOnPaste": true
},
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.linting.banditEnabled": true,
"python.linting.banditArgs": [
"-x",
"./tests",
"-r",
],
"python.linting.pylintArgs": [
"--disable=C0301,C0111,E0402,W0702,W0108,W0703"
],
"python.linting.pycodestyleEnabled": true,
"python.linting.pycodestyleArgs": [
"--ignore E501"
],
"python.formatting.provider": "black",
"python.languageServer": "Pylance",
"python.envFile": "/Users/current.user/.vspyenv",
"python.testing.pytestArgs": [
"-s",
"-vvvv"
],
"pythonTestExplorer.testFramework": "pytest",
"markdownlint.config": {
"MD013": {
"line_length": 120,
"tables": false,
"code_blocks": false
},
"MD025": false,
"MD033": false,
"MD036": false,
"MD041": false
},
"testExplorer.hideEmptyLog": false,Access to local VMs running Amazon Linux helps rapidly and safely iterate on Ansible code.
VirtualBox can be installed with Homebrew. However, every once in a while the latest build of VirtualBox has a broken functionality. At the time of writing, VirtualBox 6.1.28 has a broken Host Network Manager. Good build of VirtualBox is 6.1.26. Install Virtual Box from the link.
Vagrant is a HashiCorp Ruby project to provide VirtualBox abstraction. Use Homebrew to install vagrant
$ brew install vagrant
Test Kitchen is a Ruby project to automate Infrastructure as Code development life-cyle.
Install RVM
$ gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
$ \curl -sSL https://get.rvm.io | bash -s stable --ruby
Add RVM to your profile
# Add RVM
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"Change directory to anything other than ansible-amazon-base and change back to have RVM pull a ruby version and create a gemset.
$ cd ..
$ cd ansible-amazon-base/
ruby-3.0.0 - #gemset created /home/current.user/.rvm/gems/ruby-3.0.0@ansible-kitchen
ruby-3.0.0 - #generating ansible-kitchen wrappers.............
After RVM configures Gemset, run bundler to install gems
$ bundle install
Amazon Linux v2 is a feature-rich Linux distribution maintained by Amazon. Amazon page about Amazon Linux Images lists various formats available, including Amazon Linux v2 virtualbox.
Older Amazon Linux v2 vagrant box can be downloaded from HashiCorp Vagrant Cloud.
Import vagrant box:
$ vagrant box add amazon2 <downloaded box>
$ vagrant box list
Latest Amazon Linux v2 box can be built from Chef Bento project. Disabling Amazon SSM is a consideration.
Default Amazon Linux 2 vagrant box does not come with Ansible installed. Any time we run kitchen with the default Amazon Linux 2 box, kitchen will spend time installing Ansible. In order to save development time, we will build a box that includes Ansible and Docker.
Run kitchen converge with kitchen.box.yml
$ KITCHEN_YAML=kitchen.box.yml kitchen converge box
List running VirtualBox VMs and make note of the full name of the kitchen-ansible-amazon-base-box-amazon VM
$ VBoxManage list vms
"kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e" {9cf5ed90-d3a0-4e95-b742-6c9249c0cf34}
Run vagrant to export kitchen-ansible-amazon-base-box-amazon
$ vagrant package --base kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e
==> kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e: Attempting graceful shutdown of VM...
==> kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e: Clearing any previously set forwarded ports...
==> kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e: Exporting VM...
==> kitchen-ansible-amazon-base-box-amazon-cb9dedd7-fd27-4344-b026-bd3a2b7a340e: Compressing package to: /Users/current.user/work/ansible-amazon-base/package.box
Import package.box as amazon2-ansible
$ vagrant box add amazon2-ansible package.box
$ vagrant box list
amazon2 (virtualbox, 0)
amazon2-ansible (virtualbox, 0)
Remove kitchen instance
KITCHEN_YAML=kitchen.box.yml kitchen destroy box
Remove package.box from the local folder
Kitchen will use amazon2-ansible box by default. If you are using a different box, you can set KITCHEN_ANSIBLE_BOX environment variable.
Change directory to playbooks and run
$ kitchen list
To create a VirtualBox VM and apply an Ansible playbook run
$ kitchen converge docker
To destroy a created VM run
$ kitchen destroy docker
To connect to a VM run
$ kitchen ssh docker
Integration tests are written in TestInfra. To run tests
$ kitchen verify docker
In order to run Ansible Playbooks on AWS instances, first configure ANSIBLE_ROLES_PATH to point to the roles directory of the repository, e.g.
$ export ANSIBLE_ROLES_PATH=/Users/current.user/work/ansible-amazon-base/roles
Also configure ~/.ansible.cfg to format Ansible output as a more readable YAML
[defaults]
stdout_callback = yamlChoose a playbook and run Ansible
$ ansible-playbook -v -u ec2-user --private-key ~/.ssh/<instance>.pem -i <instance_ip>, playbooks/gst/gst_jupyter.yml
Why bother with Docker Desktop when you can build your own Containerd Server?
Check "Host Network Manager" in the File menu of your VirtualBox. Add an interface and make note of the subnet.
Edit kitchen.yml containerd suit and set private_network to a static IP of your choice.
- name: containerd
provisioner:
name: ansible_playbook
playbook: ./playbooks/docker/docker.yml
driver:
vm_hostname: containerd.local
network:
- ['private_network', {ip: '192.168.56.121'}]Run kitchen converge
$ kitchen converge containerd
...
PLAY RECAP *********************************************************************
localhost : ok=5 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
Downloading files from <containerd-amazon>
Finished converging <containerd-amazon> (0m5.54s).
-----> Test Kitchen is finished. (1m15.18s)
SSH into your Containerd VM. Change vagrant's user password from 'vagrant' to something secure. Add your SSH key to ~/.ssh/authorized_keys. Change permissions on /var/run/docker.sock.
$ ssh vagrant@192.168.56.121
vagrant@192.168.56.121's password:
Last login: Mon Nov 15 05:47:31 2021 from 10.0.2.2
__| __|_ )
_| ( / Amazon Linux 2 AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-2/
This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
[vagrant@containerd ~]$ passwd
Changing password for user vagrant.
Changing password for vagrant.
(current) UNIX password:
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[vagrant@containerd ~]$ vi ~/.ssh/authorized_keys
[vagrant@containerd ~]$ sudo chmod a+rw /var/run/docker.sock
exit
logout
Connection to 192.168.56.121 closed.
Create a new Docker context:
$ docker context create containerd1 --docker "host=ssh://vagrant@192.168.56.121"
containerd
Successfully created context "containerd1"
$ docker context use containerd1
$ docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
containerd1 * ssh://vagrant@192.168.56.121
default Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
Start using your new Containerd server:
$ docker version
Client: Docker Engine - Community
Version: 20.10.10
API version: 1.41
Go version: go1.17.2
Git commit: b485636f4b
Built: Fri Oct 15 14:45:13 2021
OS/Arch: darwin/amd64
Context: containerd1
Experimental: true
Server:
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.15.14
Git commit: b0f5bc3
Built: Tue Sep 28 19:56:28 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0
GitCommit: 84113eef6fc27af1b01b3181f31bbaf708715301
docker-init:
Version: 0.19.0
GitCommit: de40ad0
- Ansible 101 - Episode 1 - Introduction to Ansible
- Ansible 101 - Episode 2 - Ad-hoc Tasks and Inventory
- Ansible 101 - Episode 3 - Introduction to Playbooks
- Ansible 101 - Episode 4 - Your First Real-World Playbook
- Ansible 101 - Episode 5 - Playbook Handlers, Environment Vars, and Variables
- Ansible 101 - Episode 6 - Ansible Vault and Roles
- Make Your Ansible Playbooks Flexible, Maintainable, and Scalable