Theodolite’s Scalability Metrics

Theodolite’s scalability metrics are based on the following definition:

Scalability is the ability of [a] system to sustain increasing workloads by making use of additional resources. – Herbst et al. (2013)

Based on this definition, scalability can be characterized by the following three attributes:

Load intensity is the input variable to which a system is subjected. Scalability is evaluated within a range of load intensities.
Service levels objectives (SLOs) are measurable quality criteria that have to be fulfilled for every load intensity.
Provisioned resources can be increased to meet the SLOs if load intensities increase.

Scalability Metrics

Theodolite uses these attributes to define two scalability metrics:

Resource Demand Metric	Load Capacity Metric
The resource demand metric quantifies scalability by describing how the amount of minimal required resources (i.e., all SLOs are fulfilled) evolves with increasing load intensities.	The load capacity metric quantifies scalability by describing how the maximal processable load (i.e., all SLOs are fulfilled) evolves with increasing resources.
Example: Scalability of two stream processing engines measured with the demand metric.	Example: Scalability of two stream processing engines measured with the capacity metric.

Formal Definition

For a more formal definition of both metrics, we define the load type as the set of possible load intensities for that type, denoted as \(L\). Similarly, we define the resource type as the set of possible resources, denoted as \(R\). We also require that there exists an ordering on both sets \(L\) and \(R\). We define the set of all SLOs as \(S\) and denote an SLO \(s \in S\) as Boolean-valued function \(\text{slo}_s: L \times R \to \{\text{false},\text{true}\}\) with \(\text{slo}_s(l,r) = \text{true}\) if a system deployed with \(r\) resource amounts does not violate SLO \(s\) when processing load intensity \(l\).

We can denote our resource demand metric as \(\text{demand: } L \to R\), defined as:

\[\forall l \in L: \text{demand}(l) = \min\{r \in R \mid \forall s \in S: \text{slo}_s(l,r) = \text{true}\}\]

And similarly denote our resource capacity metric as \(\text{capacity: } R \to L\), defined as:

\[\forall r \in R: \text{capacity}(r) = \max\{l \in L \mid \forall s \in S: \text{slo}_s(l,r) = \text{true}\}\]

Theodolite’s Scalability Metrics

Scalability Metrics

Formal Definition

Further Reading