Turnkey AWS with Paco: Create and Manage a WordPress server — Part 2

By Kevin Teague
Published on Jan 17, 2020

In part 1 we showed you how to create a WordPress server on AWS. Now we’re going to give you a tour of making customizations to a Paco project and show you how to:

  • Secure your server by restricting SSH access
  • Configure a Route 53 Hosted Zone
  • Turn on AWS Backup to prevent data loss
  • Enable monitoring and alerting
  • Increase the server size to handle higher traffic

Secure your server by restricting or removing SSH access

Security Groups is AWS’s method of controlling network traffic. Your WordPress server runs an SSH service, enabling you to connect to your server over SSH and run remote commands. Your server’s SSH service listens on port 22 and this is what your SSH client needs to be able to connect to.

The network configuration for your Paco project defines a Security Group that allows all incoming (‘ingress’) traffic on port 22. In your Paco project’s netenv/wnet.yaml file you will see a security_groups: section that defines your network’s Security Groups:

network:
security_groups:
site:
web:
enabled: true
egress:
- cidr_ip: 0.0.0.0/0
name: ANY
protocol: "-1"
ingress:
- from_port: 80
to_port: 80
name: HTTP
protocol: tcp
cidr_ip: 0.0.0.0/0
- from_port: 22
to_port: 22
name: SSH
protocol: tcp
cidr_ip: 0.0.0.0/0

The first ingress: rule ends with cidr_ip: 0.0.0.0/0 — this is what allows all traffic regardless of IP address. As this is open to anyone in the world, your server will be vulnerable to SSH probes. If you only access your server from one computer or network, you can restrict traffic to specific IP addresses:

- from_port: 22
to_port: 22
name: SSH
protocol: tcp
cidr_ip: 255.128.90.80/32 # Bob's home computer

You can add as many entries to this list as you have IP addresses.

After you’ve SSHed to your server to get the initial WordPress starting admin password, you may decide to block all SSH traffic completely. You can delete or simply comment out the ingress rule:

# disabled until needed
# - from_port: 22
# to_port: 22
# name: SSH
# protocol: tcp
# cidr_ip: 0.0.0.0/0

To apply changes to the Security Group, simply run paco provision on the NetworkEnvironment:

paco provision -y netenv.wnet.prod

You can see your Security Group updated in the AWS console:

The Security Group for your WordPress server is named Wnet-Prod-Site-Web. Paco consistently names all your resources; the first part of the name is your NetworkEnvironment, the second is your Environment, the third is for a ResourceGroup, and the final is the name of the Resource.

Turn on AWS Backup

Paco provisions a dedicated EBS Volume (hard drive) that is used to store your WordPress data. This is very helpful if the hardware your EC2 instance is running on has a fatal hardware crash. The AutoScalingGroup that your server runs in will automatically launch a fresh instance and re-mount the existing EBS Volume to the new server. This enables your WordPress server to automatically heal itself within a few minutes of a hardware crash.

What if something happened to your data though? Perhaps you accidentally deleted some posts, or your WordPress server was hacked and everything was deleted? Or you installed poorly tested plug-ins that made a mess of everything?

If you have a back-up of your dedicated WordPress EBS Volume, you can restore your web site to any point in time you created a backup. Paco supports the AWS Backup service for this. Look inside your netenv/wnet.yaml network environment file and you will see this:

backup_vaults:
site:
enabled: false
plans:
ebs:
title: EBS Backups
enabled: true
plan_rules:
- title: Daily EBS backups
schedule_expression: cron(0 7 ? * * *)
lifecycle_delete_after_days: 7
selections:
- title: EBS Daily Backups Selection
tags:
- condition_type: STRINGEQUALS
condition_key: Paco-Application-Resource-Name
condition_value: wp_volume
- condition_type: STRINGEQUALS
condition_key: paco.env.name
condition_value: prod

This configures an AWS BackupVault to create an EBS Volume backup for the resource named wp_volume at 7 am every day, and keep those back-ups for 7 days.

In the turnkey Paco project in the prod environment, this BackupVault is disabled. Look at this configuration further down in your netenv/wnet.yaml file:

environments:
prod:
us-west-2:
enabled: true
backup_vaults:
site:
enabled: false # backups are off

In your environments: configuration section, you can override the default configuration for that NetworkEnvironment. In this case, the default value is false and the environment overrides it with false but let’s change that:

environments:
prod:
us-west-2:
enabled: true
backup_vaults:
site:
enabled: true # backups are ON!

Now re-run paco provision netenv.wnet.prod and a BackupVault will be created to create EBS Volume snapshots.

A technical note on EBS Volume backups: This is a primitive back-up solution for WordPress. If your WordPress database is being modified while the back-up is made, it’s possible to create a back-up with a broken database. This should be fine if you’ve only got a few editors and don’t have comments enabled, and you do your backups when you aren’t normally working on the site, but for a more important WordPress site, you should run a cron-job that creates a mysqldump of your database.

If you need to restore to a backup, you will need to visit the AWS Backup service, find the site Vault, and create a new EBS Volume from a Recovery Point. Then you will have to SSH to your WordPress server, manually attach and mount the newly created recovery volume and copy the data onto your secondary volume under the/mnt/bitnami/directory directory.

Create a Route 53 hosted zone for your domain

A Paco project has a resourcedirectory for global resources such as Route 53. If you registered a domain name for your WordPress site with Route 53, then Paco can automatically handle adding CNAMEs to your Route 53 HostedZone. See the Paco Route 53 docs for more information on using Route 53 with Paco.

There are many other domain registries, so the configuration to modify Route53 HostedZone is commented out in this starter project. If you’ve registered a domain name with another service, you can simply update your DNS entry to resolve your domain name to the Elastic IP that Paco provisioned for your WordPress server. Consult your particular domain registry for instructions on how to do that.

Otherwise, if you are using Route 53, to start you will need to uncomment and edit the dns:lines in your netenv/mynet.yaml file for your eip: configuration:

applications:
site:
groups:
site:
resources:
eip:
title: static IP address
type: EIP
order: 1
enabled: true
# dns:
# - domain_name: example.com
# hosted_zone: paco.ref resource.route53.site
# ttl: 60
# - domain_name: www.example.com
# hosted_zone: paco.ref resource.route53.site
# ttl: 60

Change the domains example.com and www.example.comto your own domain. If you wanted additional sub-domains, you can simply add them to the list here.

Next, open the resource/route53.yamlfile in your Paco project and see a file like:

hosted_zones:
site:
enabled: true
domain_name: example.com
account: paco.ref accounts.master

Change the domain_name: to your domain. Then create a Route 53 HostedZone by running:

paco provision -y resource.route53

When you register a domain through Route 53, it will automatically create a HostedZone for you. You will now have two HostedZones. In the AWS Console go to Route 53 then Registered Domains and click on your domain. You will see the Name servers for your domain — copy these somewhere, then go to Route 53 → Hosted zones and click on the HostedZone created by Paco (it will be the one without any text in the Comment field). Click on the domain name on the row with “Type: NS” and copy the Name servers for your domain into here. Now you can delete the HostedZone created by Route 53.

Finally, when you provision your NetworkEnvironment, it will create an A Record type in your HostedZone that will resolve your domain and sub-domains to your Elastic IP.

paco provision -y netenv.wnet.prod

Enable monitoring and alerting

What if your WordPress server got a lot of traffic and couldn’t handle the load? What if it crashed completely? Monitoring is about collecting metrics from your infrastructure and applications and alerting you when those metrics cross set thresholds. We’ll enable two kinds of monitoring for your WordPress server: external HTTP health checks and internal metrics to monitor CPU utilization, memory, and free disk space.

SNS Topics to Notify

Before we get started, we’ll need to create an SNS Topic with a subscription so that when alarms are triggered they have something to send notifications to. In your Paco project open resource/snstopics.yaml and change the you@example.com email address to your own email.

account: paco.ref accounts.master
regions:
- 'us-west-2'
- 'us-east-1'
groups:
admin:
title: "Administrator Group"
enabled: true
subscriptions:
- endpoint: you@example.com
protocol: email

Now create the SNS Topic(s) with the command:

paco provision -y resource.snstopics

If you want, you can have a list of several email subscriptions, or you can manually manage subscriptions to the SNS Topic after you make it using the AWS Console.

Why do I need two SNS Topics?

When you provision the SNS Topics, you’ll notice that Paco will create one topic in the us-west-2 region and a second topic in the us-east-1 region. This is assuming you provisioned your WordPress server in a region other than us-east-1. This is because the SNS Topic needs to be in the same region as the Alarms, but the Route 53 Health Check service only runs in us-east-1.

This means you’ll have to confirm SNS subscriptions twice after you make the SNS Topics.

At Waterbear Cloud we wrote a Paco Product add-on which can route alarm notifications from a variety of regions through a single Lambda, so that you only need to manage one SNS Topic when dealing with multi-region set-ups. Paco allows you to write your own add-ons, or you can license the Waterbear Cloud Paco Products.

Enable and provision the Alarms

Open your netenv/wnet.yaml file and find the prod environment configuration:

environments:
prod:
us-west-2:
applications:
site:
enabled: true
monitoring:
enabled: false # metrics are OFF
health_checks:
external_ping:
enabled: false # metrics are OFF

Change the monitoring to look like:

          monitoring:
enabled: true # metrics are ON
health_checks:
external_ping:
enabled: true # metrics are ON

Now enable monitoring by running paco provision— be aware that this will install the Amazon CloudWatch Agent on your WordPress server and will cause your existing server to terminate and launch a new one — you will have downtime to your WordPress site for a few minutes while this is happening:

paco provision -y netenv.wnet.prod

Also note that enabling these metrics and alarms will cost you a couple of dollars a month in additional charges to your AWS account.

External health checks with Route 53 Health Checks

AWS offers an external health check service (which is somewhat hidden within the Route 53 service). This service sends HTTP(S) requests to your web application from servers in a variety of other AWS Regions. If these health checks fail to get a 200 OK status back (from several difference regions at once), the health check will fail and an alarm will be triggered.

You can view the health check in AWS console by going to the Route 53 service and then going to Health checks:

The Route 53 Health Check created by Paco

If your site becomes unavailable, you will get an alarm.

If you add a domain name or modify your home page, you may need to modify your Route 53 Health Check configuration. In your netenv/wnet.yaml change the external_ping: configuration:

applications:
site:
monitoring:
enabled: false
health_checks:
external_ping:
type: Route53HealthCheck
enabled: false
title: HTTP Monitor
domain_name: example.com # add this field
# if you add domain_name remove the ip_address field
health_check_type: 'HTTP'
port: 80
# You can run health check on pages other than home page
resource_path: /about.html
# Change to match a string on your home page
match_string: "About Us"
failure_threshold: 3

And as usual, re-run paco provision to apply your changes:

paco provision -y netenv.wnet.prod

Internal health checks with CloudWatch

The AWS CloudWatch service is for collecting metrics and alerting on those metrics. Paco will configure a CloudWatch agent to run on the WordPress server to send swap and disk metrics to CloudWatch and will create alarms to watch four core metrics:

  • CPU Utilization: If the CPU goes above 90% and stays there for more than 30 minutes, an alarm will trigger.
  • Status Check: If there are problems with the underlying hardware your server runs on, an alarm will trigger.
  • Low swap space: Instead of a memory alarm, there is a swap space alarm. When a server consumes all available memory, it can use disk space as extra memory. The more disk space a server is using as memory, the slower it will run. This alarm is set to trigger once 80% of allocated swap space is consumed.
  • Disk space: As you create blog posts and content on your WordPress site, disk space will be consumed. You’ll get your first alert when disk consumed is at 60% and another when you’ve consumed over 80% of available disk space.
The Swap alarm for your WordPress server in CloudWatch

You can modify the file at monitor/AlarmSets.yaml to change alarms. If you wanted to be alerted when swap was at only 50%, change the SwapPercentLow threshold field:

ASG:
cwagent:
SwapPercentLow:
classification: performance
severity: low
metric_name: "swap_used_percent"
namespace: "CWAgent"
period: 60
evaluation_periods: 5
statistic: Maximum
threshold: 50 # changed from 80 to 50
comparison_operator: GreaterThanThreshold
treat_missing_data: breaching

And again run:

paco provision -y netenv.wnet.prod

Increase the server size

As the monitoring graph above showed, the default t3.micro server size is already sitting in 20% swap out of the gate. A tiny instance size is healthy for your budget, but it can’t handle much traffic. You can increase your server’s instance size to give it more CPU and memory.

Once again in netenv/wnet.yaml look for the instance_type: t3.micro and replace this a larger EC2 size such as t3.small or t3.medium.

These sizes will take you from 1 GB of memory to 2 GB and then 4 GB. Each increase will double the cost — a t3.micro is just over 1 cent per hour, a t3.small is 2 cents and a t3.medium is 4 cents.

Once you’ve put your new instance size in the configuration run:

paco provision -y netenv.wnet.prod

All done and a special kind of clean-up

If you’re finished, remember from part 1 of this blog post we showed you how to delete everything with the paco delete command:

paco delete netenv.wnet.prod
paco delete resource.snstopics # only if you enabled monitoring

However, as you’ve seen in the examples above, Paco has enabled fields throughout the configuration. It’s possible to enable or disable an application or an entire environment. When you provision an enabled environment, Paco will create all your AWS Resources. When you change that environment to enabled: falseand then provision it again, Paco will remove those resources. This is almost the same as a full delete, except Paco will leave the CloudFormation stacks for that environment with only empty placeholders, in addition, for resources such as IAM Roles and LaunchConfigurations which do not cost you any money but are time-consuming to create and delete, these resources will remain. An environment which is disabled only removes resources which cost you money. This way if you plan to re-enable an environment at a later date, you can do so more quickly.

Let’s disable the production environment by changing the enabled: field for the prod environment in the file netenv/wnet.yaml :

environments:
prod:
title: "Production Environment"
us-west-2:
enabled: false # changed from true

Now run paco provision and your environment will be disabled.

paco provision -y netenv.wnet.prod

Be careful when disabling environments and applications, as if you have any data on resources they will be forever lost!

At Waterbear Cloud, when we provision a production environment with Paco, we always run paco provisionwithout the -y flag. This flag automatically answers yes to prompts about configuration changes to each resource. Without this flag, you have a preview of the changes that Paco is about to make and you can confirm them or abort:

paco provision netenv.wnet.prod

We hope you enjoyed this infrastructure exercise with Paco. Please let us know if you have any questions in the comments.

Professional Paco by Waterbear Cloud

This solution was used by Waterbear Cloud to host https://speechteach.club. Want professional Paco help? Talk to Waterbear Cloud for custom Paco work and complete AWS solutions.