Hello all!
I hope you have had a great week so far!
My past week has not been that good, got sick and turned out I had caught Covid-19, despite being triple vaccinated. I think that thanks to being triple vaccinated; it did only appear like a nasty cold; it could probably have been much worse without the vaccination. The rest of the family seems to have not caught it and they are all ok so far.
Please be careful, I really hope you all can stay healthy at these times.
In this bulletin and the Tidy Cloud AWS website, automation is a key factor in the topics and material. There are a few terms in that space which may be good to have defined, and have a shared understanding of, so I added some general information in this bulletin.
Enjoy!
Defining infrastructure automation terms
This bulletin is about Amazon Web Services cloud automation, but what does that mean?
There are a few terms and areas which I think are useful to describe this space, which here will focus on infrastructure-related automation. This may be entirely new to you, or just familiar material. Either way, send a reply and tell me if there are key terms that are missing or specific areas that you want to see covered. Or just say hi :-)
Infrastructure as code
This is the practice of using processes and methods from the software development space to manage IT infrastructure.
This includes having a machine-readable description of the infrastructure itself, which can be used for automated provisioning of this infrastructure.
It also can include the configuration and set-up of a component within an infrastructure resource itself.
For example, to provision web servers with any associated network infrastructure, storage solutions and the configuration of the software components running on these web servers.
Machine-readable descriptions are not necessarily what people think of when they talk about code, which is often programming language code.
It includes anything that can be reliably processed by some piece of software, typically, which includes configuration files and data files that are used as part of the provisioning work.
Good practices such as version control of code (machine-readable descriptions), (automated) testing, code reviews are also part of the infrastructure as code concept.
Immutable infrastructure
This is the concept of infrastructure that is not changed, in terms of configuration, once it is provisioned.
Any changes mean you throw away the old infrastructure and replace it with new ones.
Anything that is not working properly or compromised, you throw it away and replace it with new ones.
This approach eliminates several issues around managing configurations of infrastructure components, in particular (virtual) machines.
This also requires different approaches to set up IT solutions most times and to be more strict with automating infrastructure management.
Declarative vs imperative infrastructure management
Declarative infrastructure descriptions describe the desired state of the infrastructure, whereas imperative infrastructure descriptions describe how to create/update the infrastructure.
For any non-trivial infrastructure set-up, a declarative description is more likely simpler to reason about, rather than just the steps to get to a certain state.
Also, the imperative steps are very much dependent on the context in which they execute.
Thus, an imperative approach may be more suitable for reading/query only tasks that do not change any state or a one-off task.
Note: There are a few tools in recent times that kind of combine a declarative state model with an option to use imperative steps to generate that model.
More on that in the AWS Tools section below.
AWS Tools and Services for automation
There are many tools and services from AWS themselves, as well as from 3rd party vendors that help with automation tasks.
I have tried to group these into a few categories here:
Infrastructure Automation
In this space, I put tools and services that create/update infrastructure components e.g. virtual machines, databases, network infrastructure.
Specific configuration management inside infrastructure components (e.g. virtual machines) are typically not their strong side, so they fit particularly well with an immutable infrastructure approach.
This includes tools such as AWS CloudFormation, Terraform, AWS Cloud Development Kit (AWS CDK), Pulumi, Cloud Development Kit for Terraform (CDKTF).
These tools use a declarative model as the foundation to describe the state of the infrastructure.
CloudFormation and Terraform use an approach that uses a configuration file format (JSON, YAML, HCL) and directly describes this model.
AWS CDK, Pulumi, and CDKTF use programming language code to generate the underlying declarative model.
Configuration Management Automation
Configuration management tools such as Puppet and Chef are supported via the AWS OpsWorks service and AWS Systems Manager has some support for working with Ansible as well.
These tools also use declarative state approaches but are a bit more focused on the state inside an infrastructure component, such as a virtual machine, than on the previous tools.
Test, Delivery and Deployment Automation
AWS has a bunch of services related to automation of test, delivery, and deployment.
This includes the Code-family (CodeBuild, CodeCommit, CodeDeploy, CodeGuru, CodePipeline, CodeStar) as well as multiple 3rd party tools, such as GitHub Actions, Azure DevOps, Bitbucket Pipelines, CircleCI, Jenkins and many more.
Task and Plumbing Automation
AWS provides multiple tools to automate/script tasks, including the very general AWS CLI (Command-line Interface), AWS Tools for PowerShell.
There is also the AWS Software Development Kits (SDKs) for more programming-oriented automation.
Services such as AWS Lambda and AWS Step Functions can provide some glue and orchestration capabilities to facilitate the plumbing work.
Also, there are many more specialized command-line and scripting tools as well, from AWS and others.
This includes, for example, eksctl for AWS EKS (Kubernetes), and AWS Copilot for AWS ECS.
You can find the contents of this bulletin and older ones, and more at Tidy Cloud AWS. You will also find other useful articles around AWS automation and infrastructure-as-software.
Until next time,
/Erik