Upgrade workers from m3 to c5d
Gather some data and see if runtime is improved. Newrelic should help here. The c5d.large instances have newer, faster CPUs and directly attached NVME SSDs, but half the RAM of m3.large instances. Cost drops from $0.133/hour to $0.096/hour (and should finish faster to reduce the cost further, assuming the RAM isn't a bottleneck).
c5d instance type announcement: https://aws.amazon.com/about-aws/whats-new/2018/05/introducing-amazon-ec2-c5d-instances/
EC2 instance type spec and cost comparison: https://ec2instances.info/
Steps to Reproduce
Reason for Variance
User Impact Summary
We might also want to upgrade github-status-worker from t2.micro to t3.micro; it should yield a modest performance gain, although the cost savings are negligible. I don't see a good upgrade path for coverage-worker away from m3.medium; the updated tiers no longer have a "medium" instance type, and all the cheaper options use EBS instead of a direct SSD.
Also look into the ECS containers. We had seen better performance gains by increasing containers rather than increasing processes per container, which is why we currently only run 1 xdist process on each. Maybe a newer instance type will change.