Why we keep a host's worth of capacity free in every location

If a host fails, the services on it have to be rebuilt from backup somewhere. "Somewhere" has to exist before the host fails, or the recovery plan is a wish.

So every active Balta location holds unallocated capacity, and we stop selling in that location before the capacity is consumed rather than after.

The formula, and the version of it that was wrong #

The first version reserved the allocated capacity of the largest host. That sounds right and is not, because it counts the free capacity on the host being lost as available to absorb that host's own workload.

The worked example that exposed it:

• Three hosts, 100 GB allocatable each. Total allocatable 300 GB.
• 74 GB allocated on each. Total allocated 222 GB, total free 78 GB.
• Old formula: reserve 74 GB, report 4 GB sellable. Green.

Now lose a host. The two survivors have 26 GB free each. That is 52 GB against a 74 GB workload to re-home. It does not fit. The check passed a location that could not survive a host loss, which is the exact thing the check exists to prevent.

The corrected constraint is the one you get by writing down what actually has to be true: free capacity on the surviving hosts must absorb the lost host's allocation. That reduces to a rule that is cheap to compute and easy to explain.

For each resource dimension R in (vCPU, RAM, storage, IOPS), evaluated independently:

  reserve_required(location, R)
      = MAX over hosts h of ALLOCATABLE(h, R)      <- allocatable, not allocated
        x recovery_reserve_hosts

  available_for_placement(location, R)
      = allocatable(location, R)
      - allocated(location, R)
      - reserve_required(location, R)

Reserving the largest host's allocatable capacity is exactly right, because losing a host costs you the workload it carried and the headroom it was contributing.

Re-run the example: reserve 100, allocatable 300, allocated 222, available is minus 22. Correctly red. To be green those three hosts may hold 200 GB in total, at which point a 66.6 GB workload lands on two survivors with 33.3 GB free each. It fits exactly, as it should.

The largest host can be a different host in each dimension, so the maximum is taken independently for vCPU, RAM, storage and IOPS rather than by picking one host and applying its whole profile.

Aggregate capacity is not enough either #

There is a second, subtler failure, and we found it while trying to make the check cheaper.

Counter-example: the failed host carries three 30 GB services. Two survivors have 50 GB free each. Aggregate free is 100 GB against 90 GB needed, so the aggregate check passes. The largest single service is 30 GB and fits on a 50 GB survivor, so a largest-service check passes too.

But each survivor accepts only one 30 GB service. Two get placed and the third has nowhere to go. Both checks pass and the location still cannot survive a host loss. Fitting N services onto M hosts is a packing problem, and the only honest way to answer it is to try the packing.

What happens as a location fills up #

The location closes to new services before the reserve is consumed, not after. Selling the last of the recovery capacity and discovering it during an incident is the specific failure this prevents.

The same computation runs when we deliberately drain a host for maintenance. A drain whose workload cannot fit on the remaining hosts is refused, and the control plane shows the arithmetic rather than the refusal alone.

What it costs #

The reserve is capacity we pay for and do not sell. That is the honest description, and it is worth being direct about the size of it.

In a three-host location, holding back one host's allocatable capacity is roughly a third of the location. That is a large fraction, and it is the reason a launch location should grow past the three-host minimum reasonably quickly rather than sit there.

We carry the reserve as a first-class input and output in the economics model, so it is priced deliberately. The alternative is what usually happens: the reserve is not modelled, margin gets tight, and the reserve quietly erodes the first time a location runs short — which is precisely when you need it.

If the reserve turns out not to be affordable at the price we set, the price is wrong. The reserve is not the variable to cut, because it is a condition of the availability commitment rather than an optimisation on top of it.

Why this is on a marketing site at all #

Because "we have spare capacity" is the kind of claim that is impossible to check and therefore worth nothing. The formula, the failure mode it corrects, the packing check and the point at which we stop selling are all checkable statements. You can hold us to them.

What we are not claiming: this does not make a service continuously available through a host loss. It makes the rebuild possible. There is no standby and no automatic failover in the first version, and the reliability page says so in the same sentence it says this.