As everything is run in a Docker container, we spawn every process in a single container, which has the added benefit of straightforward scaling of workers; every worker is a single container.

As long as there is a stable server holding the queue, then the available workers to retrieve new jobs can be in flux.

The Spot Instance marketplace provides machines at steep discounts, and the use of SpotInst provides an incredibly straightforward way to manage the complexities of bidding, provisioning, and managing the replacement of spot instances that expire without any downtime.

We scale multiple machines every weekend that have the necessary CPU and Memory specs to handle our worker load. Using Rancher, our workers will auto-scale on the newly available machines, automatically connect to Rabbit and Postgres, and start consuming available jobs immediately.

The boxes along the top represent worker containers, which are automatically distributed across spot instance machines as they become available.

At Pit Rho, we found a clear use case for Spot Instances in one of our key products that helps us control costs without compromising performance.

While Spot Instances do not solve every problem, we’re now constantly looking for ways we can use them.