Waterbear Cloud Newsletter, November 2019

By Kevin Teague
Published on Nov 27, 2019

Welcome to the November 2019 edition of our Waterbear Cloud monthly newsletter, where we share our news for the month and tell you about what we’ve been up to.

AIM supports AWS Backup

AWS Backup was released by AWS at the start of this year. It provides an easy way to back-up and restore your EBS volumes, EFS file systems, RDS databases, DynamoDB tables, and Storage Gateway volumes to S3 storage. AWS Backup is a robust, easy-to-use service that is reasonably priced per GB/month. It is missing some otherwise advanced back-up features, such as cross-region back-up and its per GB pricing makes its cost less competitive as you get into multi-TB back-up sizes. However, it does use similar back-up technology as AWS EBS snapshots, where you always have a full complete back-up available, but you can arbitrarily delete any of your snapshots. With AWS back-up, you can back-up frequently, and AWS only stores the differences between existing back-ups.

AWS Tags generated automatically by AIM

With AWS Backup you can select the resources you want to backup directly or you can automatically include resources based on tags. The tag selection method works well with AIM, which tags all of your resources automatically and 100% consistently. With the AIM-generated tags, you can apply backup policies to your environment and type (e.g. all EBS volumes in Production) or further specify different schedules per-application or per-resource (e.g. your WordPress application is backed up four times a day, while your SaaS is backed up nightly). As you add new resources to your environment, tag selections allow you to automatically include new resources in your back-up schedules even if you forgot about adding them to your back-up resource lists.

AIM command-line interface usability upgrade

Our vision for the cloud is to be able to cleanly re-use complete Infrastructure as Code solutions. We’ve been preparing our first turnkey AWS solutions, and in doing so, we gave our CLI an overhaul this month.

An AIM project is essentially a tree that describes how everything should be in your cloud. At the top level is directories for accountsnetworks and environmentsmonitoringglobal resources and service extensions. Within these directories are YAML files that describe an account or a network environment. Finally within each YAML file, the hierarchy continues — environments contain regions, which contain applications, which contain groups of resources, which finally contain individual resources.

AIM only has three cloud commands for converting this tree into actual resources in the cloud: validateprovision and delete. The validate command doesn’t actually modify any cloud resources, but simply validates that the resources are verified as correct by AWS. The provision command does all of the work; it creates and updates resources according to your AIM project model. The delete command simply cleans up when you no longer want resources in the cloud.

Each of these three cloud commands operates as a node within this tree and applies to all resources represented by that node and all child nodes within it. The new AIM CLI allows you to select a node with a dotted notation.

# validate the dev environment in the sandbox network
aim validate netenv.sbox.dev

# provision the prod environment in saas network
# for just the us-west-2 region
aim provision netenv.saas.prod.us-west-2

# provision in the eu-central-1 region for only
# the wordpress application
aim provision netenv.saas.prod.eu-central-1.applications.wordpress

# provision IAM Users across all your accounts
aim provision resource.iam.users

With other Infrastructure as Code management tooling (e.g. Terraform or Sceptre), you need to plan up-front how you will divide up your cloud resources for provisioning (typically by creating deeply nested directory structures). Once your project is organized in its own unique way, you are locked into that rigid orchestration strategy. With AIM you can select any level in your tree — from all your environments down to just a single resource in a single environment for a single application. This makes updates faster, and for production environments, can make them safer.

Waterbear Cloud is developing the next wave of cloud management tooling with AIM, which dramatically improves the robustness of Infrastructure as Code projects. Talk to us about how we can bring your AWS environments to the next level of control and reproducibility.