---
lang: en-GB
papersize: a4
geometry: margin=2cm
titlepage: true

title: "Enhancing Kubernetes Autoscaling on IBM Cloud"
subtitle: "A Comparative Study of Karpenter and Kubernetes Cluster Autoscaler"
author: "Josephine Pfeiffer"
date: "18 December 2024"
class: "COM00151M - Independent Research Project"

lof: true
lot: true

numbersections: true
csl: university-of-york-ieee.csl
mainfont: Source Sans Pro

header-includes:
  - \hypersetup{pdfborder=0 0 0}
  - \usepackage{fvextra}
  - \DefineVerbatimEnvironment{Highlighting}{Verbatim}{breaklines,commandchars=\\\{\}}
  - \usepackage{fontspec}
  - \setmainfont{Source Sans Pro}
  - \usepackage{pdflscape}
  - \newcommand{\blandscape}{\begin{landscape}}
  - \newcommand{\elandscape}{\end{landscape}}
...

<!-- NB: Executive Summary, Acknowledgements, Contents Page, List of Tables/Figures, Glossary, Reference List and Appendices are NOT included in the word count. -->

\newpage

<!-- 500w max-->

# Abstract {-}

This research project investigated resource efficiency and cost savings in cloud environments, particularly focusing on infrastructure scaling using Kubernetes.

With the rise of cloud-native applications, autoscaling has become a fundamental feature for optimizing resources and controlling costs. This project specifically examined the effectiveness of Karpenter, an open-source cluster autoscaler, compared to the traditional methods -- namely using the Kubernetes Cluster Autoscaler, especially in the context of IBM Cloud.

The motivation behind this research stemmed from the need for more efficient infrastructure management in the cloud. As cloud usage grows, infrastructure costs rise significantly, driven by inefficient resource provisioning. Current solutions often fail to scale resources dynamically, resulting in either over-provisioning, which wastes money, or under-provisioning, which affects performance. This inefficiency is particularly problematic for organizations that require highly scalable, flexible solutions but are constrained by budgetary considerations. The passion for more sustainable and cost-effective cloud infrastructure underpinned the research, intending to find tangible ways to improve efficiency while reducing expenses.

This study's methods were experimental, comparative, and quantitative. The experimental component involved deploying Kubernetes clusters on IBM Cloud and integrating Karpenter to automate scaling. Comparative analysis was used to measure the performance of Karpenter against the Kubernetes Cluster Autoscaler. The same experiments were conducted on AWS to serve as a control group. Quantitative data was collected from real-time metrics on scaling speed, resource utilization, and cost efficiency. These metrics were used to assess how well each method performed under varying workloads. Cloud platform APIs and Prometheus were leveraged to capture performance data, and statistical methods were applied to identify any significant differences between the approaches.

The results demonstrated that Karpenter outperformed the Cluster Autoscaler in key metrics such as scaling speed, resource efficiency, and cost savings. On AWS, Karpenter achieved a 33% reduction in monthly costs and significantly improved latency and node count optimization compared to CAS. On IBM Cloud, Karpenter reduced monthly costs by approximately 20% and improved memory utilization and node count efficiency, although its performance was constrained by platform-specific limitations such as slower instance provisioning and higher baseline resource usage. These findings validated Karpenter's potential as a more effective autoscaling solution while underscoring the influence of underlying cloud infrastructure on its performance.

There were no major legal, social, or ethical issues directly associated with the research. There were no professional conflicts. All data collection was confined to performance metrics from cloud infrastructure, which did not involve any human participants or sensitive information.

Professionally, the research aligned with best practices for cloud infrastructure management and has the potential to contribute positively to industry standards by demonstrating the benefits of efficient scaling mechanisms.

Commercially, the implications are significant. Cloud service providers and enterprises could see substantial cost reductions if Karpenter proves to be a more efficient solution in real-world customer scenarios. Furthermore, the findings could prompt service providers like IBM to integrate or improve autoscaling solutions in their offerings.

\newpage

# Acknowledgements {-}

I would like to express my sincere gratitude to my supervisor, Nida Zeeshan, for her support, insightful feedback, and guidance throughout the course of this research. Her expertise and encouragement were invaluable in shaping the direction and quality of this work.

I would like to thank Marco Kuendig (Chief Technology Officer at copebit) who presented Karpenter at a Cloud Native Compute Conference we were both speaking at for inspiring me to look into Karpenter.

I am also deeply grateful to Krishna Ratakonda (IBM Fellow, VP & CTO, Cloud for Financial Services), and Igor Naumov (Senior Architect, IBM Cloud for Financial Services) for their support and guidance on the IBM Cloud aspects of this project. Their technical insights and advice were crucial in advancing the research.

A special thanks goes to Michael McCune (Senior Principal Software Engineer, OpenShift Cloud Infrastructure Team at Red Hat) for his assistance with Karpenter and his generous sharing of expertise, which greatly contributed to the project's success.

Finally, I would like to extend my heartfelt appreciation to my family and friends for their constant support, patience, and encouragement throughout this journey.

\newpage

\tableofcontents
\newpage

<!-- # Main body (y/9,000w)

You should write 7,000 to 9,000 words, aimed at an academic in Computer Science, but one who is not necessarily a specialist in your field of study.

- Where the maximum word count of the document/section has been exceeded, a line will be drawn at that limit, and any text after this point will not contribute to your grade.
- Documents/sections that do not meet the minimum word count will not be reviewed. While no particular structure is mandated, it is expected that the following sections will be covered, clearly identifiable and supported by relevant literature:
- Introductory material
- Background and review of related work
- Discussion of philosophical approach, research methodology, experimental design, or design and your computational approach
- Results and analysis
- Concluding material
 -->

# Introduction

## Problem Background

The ability to dynamically scale resources in response to workload demands is an important topic in the fields of distributed systems and infrastructure management for economic, environmental, and efficiency reasons [@awsscale2024].

Dynamic resource scaling can significantly impact economic efficiency by reducing operational costs. For instance, cloud services that adapt their provisioning to variable workloads can increase resource utilization and reduce operating costs [@Lombardi2019PASCAL]. Inefficient resource management, such as over-provisioning, leads to unnecessary expenses, while under-provisioning can result in performance degradation and potential revenue loss [@Benifa2018RLPAS].

From an environmental perspective, energy consumption is an important issue [@Shaari2020The]. Data centres, which support cloud computing, consume substantial amounts of energy, contributing to environmental degradation [@Whitehead2014Assessing]. Efficient resource utilization can minimize energy consumption, thereby reducing the environmental footprint. For example, dynamic voltage and frequency scaling [DVFS] techniques can reduce energy costs by adjusting computation frequency based on demand [@Zhang2020Energyefficient]. Additionally, software-defined data centres [SDDCs] can optimize resource allocation, leading to significant energy savings—up to 27.9% in some cases [@Kaur2020A].

Efficiency in resource scaling also plays a crucial role in maintaining system performance and reliability. Traditional scaling methods that do not accurately estimate resource demand can lead to performance issues and energy waste due to scaling delays [@Hu2018Fast]. Proactive auto-scaling methods, which predict future workloads and adjust resources accordingly, can maintain desired performance levels at reduced costs [@Iqbal2018Dynamic]. For instance, a dynamic concurrency management framework can achieve higher throughput and shorter latency compared to traditional methods [@Wang2019Integrating].

Using container orchestration platforms to streamline and optimize resource scaling for enterprise-scale production workloads is gaining traction. According to the Diamanti 2024 Kubernetes Adoption Survey [@amandeep2024aug], 84% of organizations plan to migrate applications to containers managed by Kubernetes within the next 12 months. Kubernetes is a widely used open-source container orchestration platform [@k8s]. The Kubernetes Cluster Autoscaler [CAS] [@clusterautoscaler] and the Machine-Autoscaler [@machineaurtoscaler] allow administrators to automatically adjust the number of nodes in a cluster or the underlying machine resources based on workload demands. However, they have shortcomings, which stem from the requirement of maintaining static node pools or machine inventories which often leads to over-provisioning, where resources are wasted during low-demand periods, and under-provisioning, causing performance issues during peak times [@Tamiru2020An]. Additionally, static node pools can struggle with heterogeneous workloads, as they lack the flexibility to dynamically adjust node types and sizes based on specific application needs [@Toka2021Machine], [@workloadautoscale]. These mechanisms and how they interact with each other is illustrated in [@fig:cas]. Overall, the aforementioned limitations result in higher costs, inefficient resource utilization, and potential application performance degradation.

![Overview of the autoscaling flow used by the Cluster Autoscaler. Adapted from [@rhgraphic].](cas-diagram.png){width="560px" height="260px" #fig:cas}

Karpenter is an open-source Kubernetes Cluster Autoscaler designed to improve the speed and efficiency of scaling operations in cloud environments by leveraging spot instances and dynamic node right-sizing [@karpenter]. Karpenter providers exist for Amazon Web Services [AWS] [@kaws] and Azure Cloud Platforms [@kazure], while another provider for Alibaba Cloud is in development [@alibabakarpenter].

This research focuses on developing a Karpenter provider for IBM Cloud and evaluating its performance compared to traditional Kubernetes cluster autoscaling mechanisms, namely CAS. The primary unit of analysis includes scaling speed, resource utilization, and cost efficiency, for a broad assessment of the solutions' performance. The potential advantages include faster scaling times and more efficient resource management compared to traditional methods [@clusterautoscaler], [@machineaurtoscaler], [@k8sclusterautoscaler].

The rise of cloud computing's pay-as-you-go model and AI technologies has heightened the need for scalable, cost-effective infrastructure solutions [@cloudaccounting], with cloud elasticity being crucial for optimizing resource allocation and reducing costs [@elastic]. This research addresses this topic by evaluating mechanisms that improve efficiency. The research hypothesis posits that Karpenter will outperform traditional machine autoscaling mechanisms in several key metrics.

## Research Questions

$RQ_1$
: How does the scaling speed of Karpenter compare to traditional Kubernetes autoscaling mechanisms in responding to fluctuating workloads in IBM Cloud environments?

The question aims to determine which autoscaling method offers faster response times to handle fluctuating workloads better. It critiques traditional Kubernetes autoscaling for inefficiencies [@Kakade2023Proactive] and highlights new methods like proactive scaling schemes for potential improvements. Karpenter is said to be more efficient [@kfan1], [@kfan2], [@kfan], but this lacks comprehensive evaluations. The study seeks to fill this gap by comparing Karpenter with traditional methods on IBM cloud, providing practical insights and guidance for cloud practitioners, and laying a foundation for future research into autoscaling optimization.

$RQ_2$
: How efficiently do Karpenter and traditional Kubernetes autoscaling mechanisms manage CPU and memory resources under varying workload conditions on IBM Cloud?

Traditional methods often lead to suboptimal resource use due to fixed policies and the need to maintain node pools/machine inventories [@nodepool], which can result in over-provisioning or under-provisioning [@scaler]. Evaluating resource efficiency, assessing CPU/memory usage, and highlighting sustainability benefits address gaps in empirical research and the findings will provide insights into resource management practices, and promote greener cloud practices.

$RQ_3$
: What are the differences in operational costs between Karpenter and traditional Kubernetes autoscaling mechanisms in IBM Cloud environments?

Traditional autoscaling mechanisms often lead to cost inefficiencies due to over-provisioning and slower scaling, resulting in higher operational costs [@Kakade2023Proactive]. Innovative approaches like COPA and deep learning-based predictive autoscaling have already shown potential for cost optimization [@Ding2021COPA]. Karpenter, leveraging spot instances and dynamic node right-sizing, promises significant cost savings [@karpenter], but lacks comprehensive empirical evaluation. This research addresses that gap by providing a detailed cost analysis, comparing real-world costs and resource efficiency.

## Hypotheses

Based on the research aims and objectives, the following hypotheses are proposed for this study:

$H_1$
: Karpenter will have a significantly faster scaling response time compared to traditional Kubernetes autoscaling mechanisms in IBM Cloud environments.

$H_2$
: Karpenter will result in lower operational costs compared to traditional Kubernetes autoscaling mechanisms by minimizing over-provisioning and utilizing spot instances effectively.

$H_3$
: Karpenter will utilize CPU and memory resources more efficiently than traditional autoscaling mechanisms, leading to reduced resource waste and improved performance under varying workloads.

These hypotheses will be tested through performance testing and cost analysis, addressing the research questions and providing insights into the operational benefits of Karpenter in comparison to traditional autoscaling methods.

## Ethics Statement

The aspects of informed consent, purpose limitation, accuracy, honesty, transparency, and sustainability are taken into account in the thesis to align closely with the ethics policy [@ethics].

All participants, including industry experts, cloud providers, and technical advisors, are fully informed about the research objectives, methods, and potential outcomes. Informed consent is obtained, ensuring understanding of their roles and the use of their information.

Data collected is used exclusively for this study, with findings reported accurately and without manipulation. Limitations and biases are disclosed to provide a balanced view.

By terminating the lab cluster when it is not in use, unnecessary energy consumption and wasting credits is avoided. Additionally, the research aims in itself support more sustainable cloud computing practices. Efficient resource usage not only minimizes energy consumption but also lowers carbon emissions, resulting in a net positive effect on the environment [@Chien2019Owning].

## Contributions

In addressing the research aims through the methodologies detailed above, the key contributions of this thesis are:

1. This thesis provides an empirical evaluation of autoscaling methods, highlighting the strengths and weaknesses of each in terms of resource efficiency, scaling speed, and cost-effectiveness in cloud environments.

2. The creation of automation tools for provisioning, running experiments, and decommissioning Kubernetes clusters ensures repeatability in experiments and consistency in data collection, setting a standard for future research in cloud performance testing.

3. By leveraging detailed metrics collection through ansible and Prometheus, this thesis offers a fine-grained analysis of how different autoscaling mechanisms impact, node count, node latency, CPU and memory usage, as well as operational costs, providing valuable insights for cloud practitioners.

\newpage

# Literature review

This chapter reviews research on the infrastructure aspects of Kubernetes autoscaling. As cloud services expand, there is a growing need to improve the performance of datacenter infrastructure [@Choi2015The], [@Mazumdar2017Power], particularly in the context of containerized applications [@Mulahuwaish2021Improving], [@Nath2021Container-based].

As such, there is a focus on pulling from and relating back to Kubernetes literature and architecture examples where relevant.

## Kubernetes Autoscaling: Foundations and Challenges

Rosso et al. (2021) discuss various concepts related to operating Kubernetes clusters at scale in production environments [@rosso2021production]. Chapter 13 specifically covers autoscaling, introducing the Kubernetes CAS and its mechanism for adjusting node numbers based on resource demand. While effective for general scaling, this method may be limited when dealing with workloads requiring specific and diverse resources.

In contrast, Karpenter offers a more flexible approach by provisioning nodes tailored to the specific needs of applications, providing finer control over resource allocation. This reduces over provisioning and optimizes both cost and performance, particularly in environments with variable and complex workloads [@Deepeshjaiswal2023Aug].
Karpenter provisions nodes tailored specifically to application needs by creating custom Kubernetes resources called "provisioners." This approach allows Karpenter to dynamically allocate the exact resources necessary for each workload, thus minimizing both overprovisioning and underutilization. Literature suggests that this capability not only enhances performance in environments with heterogeneous workloads but also yields significant cost savings by reducing unnecessary cloud resource usage. Karpenter's emphasis on workload-specific scaling provides a nuanced level of customization and flexibility, particularly beneficial in scenarios characterized by diverse resource demands and the need for optimal cost-performance balance.

While the Cluster Autoscaler is suitable for general scaling, Karpenter’s advanced resource management is advantageous for organizations needing precise control over their Kubernetes infrastructure, especially in cloud environments where resource optimization and cost control are a priority.

Dimolitsas et al. [@Dimolitsas2023Multi-Application], [@Dimolitsas2022AHP4HPA] highlight the importance of minimizing energy consumption while maintaining high performance in cloud computing and edge environments, given the operational and environmental implications involved. This emphasis is driven by multiple factors. First, increased energy usage due to the rising demand for computational resources has significant environmental impacts, aligning energy minimization with global sustainability goals. Second, energy is a major contributor to operational costs for infrastructure providers, making optimization an economic strategy. Third, edge environments face inherent constraints in resource availability, necessitating efficient allocation to avoid over-provisioning and underutilization. Moreover, quality of service [QoS] requirements, such as low latency and high availability, must be balanced with energy efficiency to maintain user satisfaction without unnecessary energy expenditure. Finally, dynamic workloads in contexts like smart cities require adaptable and scalable resource management to meet fluctuating demand while minimizing energy waste. The proposed frameworks, including AHP4HPA and the hierarchical multi-application autoscaling system, utilize multi-criteria decision-making [MCDM] to achieve these objectives. By prioritizing energy efficiency alongside performance metrics, these approaches demonstrate the practical benefits of energy-aware orchestration in modern platforms like Kubernetes.

## Cluster Autoscaling Approaches

The existing academic literature mostly covers concepts around workload autoscaling, namely the Horizontal Pod Autoscaler [HPA] [@hpa_docs], rather than cluster autoscaling. Such examples include Galande et al. who proposes a customized metric scaler to enhance Kubernetes' HPA in [@galande2024event], whereas Horn et al. proposes a multi-objective autoscaling mechanism for microservices in Kubernetes that outperforms the standard Kubernetes HPA [@horn]. In [@caslul], Tamiru et al. present an experimental evaluation of the Kubernetes CAS in Google Kubernetes Engine. The paper covers different configurations but lacks a comparison to other cluster autoscalers or cloud platforms.

When it comes to Karpenter in particular, there seems to be no published research or academic sources as yet. However, there is literature in the form of blog posts comparing how Karpenter works to CAS [@Fontanilla2024Oct], [@mccune], as well as technical documentation explaining high-level concepts and configuration options [@karpenter_docs]. The documentation currently lacks a section on custom provider development [@karpenter_faq].

## Evaluating Autoscaling Performance Metrics

Studies have compared different configurations of CAS, finding that multi-node pool selection outperforms single-node pool selection [@caslul]. Researchers have investigated the impact of using absolute versus relative metrics for autoscaling decisions, with absolute metrics showing better performance for CPU-intensive workloads [@Casalicchio2019]. Common evaluation metrics include under- and over-provisioning accuracy, timeshare, instability of elasticity, and deviation from the theoretical optimal autoscaler [@caslul].

The two studies focus on different aspects of Kubernetes autoscaling: Cluster Autoscaling [CA] vs. Pod Autoscaling [KHPA]. The CA study evaluates two configurations, default CA and Node Auto-Provisioning [CA-NAP]. CA-NAP showed better resource matching but only limited cost savings, highlighting the need for more refined configuration, especially around scale-in policies.

In contrast, the KHPA study proposes KHPA-A, an enhancement using absolute CPU metrics for better pod-level scaling decisions. KHPA-A consistently outperformed KHPA, particularly under CPU-intensive workloads, by allocating more pods to maintain response time and QoS. KHPA-A’s use of absolute metrics allowed for quicker adaptation, whereas CA-NAP remained more static.

Both studies struggle with resource over-provisioning: CA with delayed scale-ins, and KHPA with inaccurate scaling due to relative metrics. Limitations included constrained test environments and lack of cost analysis. Future research could focus on dynamic configuration, multi-metric scaling, and larger-scale evaluations.

## Research Aims and Objectives

The aim of this research is to evaluate the effectiveness of Karpenter as an autoscaling solution in Kubernetes environments by comparing it to the Kubernetes CAS in terms of scaling performance, resource efficiency, and cost-effectiveness. The study seeks to provide a deeper understanding of how these solutions influence resource efficiency, scaling performance, and cost management in cloud-based Kubernetes clusters. To achieve this, the research addresses the following research aims:

$RA_1$
: Analyse and compare the performance of Karpenter and traditional Kubernetes autoscaling mechanisms by evaluating scaling speed, resource provisioning accuracy, and cost-efficiency in cloud environments.

$RA_2$
: Develop an automated testing framework that enables consistent provisioning, experimentation, and performance data collection across Kubernetes clusters, ensuring the repeatability of performance and scalability tests.

$RA_3$
: Quantify the impact of Karpenter and traditional autoscaling methods on CPU, memory usage, and overall resource consumption across varying workload intensities and conditions.

$RA_4$
: Evaluate and compare the operational costs associated with Karpenter and traditional autoscaling mechanisms, focusing on cloud cost-efficiency, including minimization of over-provisioning and improved resource allocation.

## Research Methodology

### Research Philosophy

The research philosophy is based on measurable and objective data and the hypotheses will be validated through experimentation.

This research follows a positivist approach, aiming to objectively measure and compare the performance of autoscaling mechanisms through quantitative analysis [@bookresdes]. The positivist paradigm aligns with the study's reliance on empirical data and hypothesis testing to assess real-world performance in cloud environments.

The expected outcomes of this research include a better understanding of the performance differentials between Karpenter and traditional autoscaling mechanisms, and insights into the potential improvements in cost efficiency, performance, and resource utilization.

Ultimately, this study seeks to contribute to the broader field of cloud infrastructure management by identifying more effective strategies for autoscaling in cloud environments, thereby enhancing both economic and operational efficiencies.

To validate the hypothesis that Karpenter will exceed traditional mechanisms in key operational and economic metrics, the study will conduct a detailed comparison through experimentation.

The relevance of this study is made even more apparent by the explosive growth of AI applications, which demand highly dynamic and scalable computing resources to manage unpredictable and intensive workloads [@Mungoli2023Scalable].

### Research Methodology

The study adopts a quantitative research method due to its ability to provide measurable and objective data [@book], which is necessary for comparing the performance of different autoscaling mechanisms. Quantitative methods will enable the precise evaluation of metrics such as scaling speed, resource utilization, and cost efficiency.

#### Data Sources and Collection Methods {-}

This study employs Prometheus, a cloud-native monitoring tool [@prometheus], to capture essential metrics such as CPU utilization, memory usage, and scaling speed, which are used to evaluate autoscaling performance in Kubernetes environments [@promstandard]. Prometheus was chosen due to its widespread use and ability to gather detailed, time-series performance data, making it well-suited for measuring resource efficiency and autoscaling responsiveness.

To evaluate cost efficiency, resource usage data collected by Prometheus is combined with pricing information retrieved from the IBM Cloud and AWS billing APIs. Using tagging mechanisms [@ibm_tagging], [@aws_tagging] and Universally Unique Identifiers [UUIDs], the study tracks costs associated with each experiment run, allowing for an accurate comparison of operational expenses between cloud platforms.

Automation tools Terraform [@tf] and Ansible [@ansible] ensure the consistency and repeatability of experiments by automating the lifecycle management of cloud infrastructure. The use of Kube-burner [@burner] for simulating workloads and Helm [@helm] for configuring Kubernetes clusters further standardizes the experimental setup, minimizing variability and human error.

Evaluation metrics focus on:

- Scaling speed, measured by node provisioning times,
- Resource efficiency, assessed via CPU and memory utilization,
- Cost efficiency, comparing operational costs per hour.

These metrics provide a comprehensive basis for comparing the effectiveness of Karpenter and traditional autoscaling mechanisms under various workload scenarios.

#### Data Analysis Techniques {-}

The study will apply descriptive statistics to summarize key performance metrics, including scaling speed, CPU and memory utilization, and cost efficiency. Descriptive statistics—such as mean, median, and standard deviation—provide a clear summary of central trends and variability in the data [@williamson2018research], allowing for an objective comparison of both Karpenter and the Kubernetes Cluster Autoscaler across various scenarios. These metrics are crucial because they represent the core factors cloud infrastructure teams consider when optimizing autoscaling solutions.

To determine whether the observed differences between the two autoscaling mechanisms are statistically significant, we will use paired t-tests and analysis of variance [ANOVA] [@schervish1995theory]. The paired t-test is particularly appropriate here since it compares performance in scenarios where both Karpenter and the Cluster Autoscaler are tested under identical conditions. This method ensures that the results reflect true differences in performance rather than variability due to external factors. ANOVA will be used when comparing multiple performance metrics simultaneously or across different workload intensities, providing a broader understanding of how both systems behave under various conditions.

There will also be a delta analysis to calculate the performance difference at each time point. This approach quantifies how Karpenter and the Cluster Autoscaler diverge in real-time, providing more granular insights into the consistency and magnitude of their performance differences. These deltas will help to highlight where one autoscaler may outperform the other during specific phases of workload changes, such as during rapid scaling events.

To support interpretation, the data will be visualized through time-series graphs. These graphs will display key metrics, such as the number of nodes, CPU utilization, and memory usage, over time, providing an intuitive way to compare the real-time behaviour of both autoscalers across various scenarios. This visual representation will make it easier to observe performance trends and resource allocation efficiency in dynamic cloud environments.

By combining statistical tests with time-series visualizations, this analysis ensures a comprehensive and objective comparison between Karpenter and traditional autoscaling methods, while offering meaningful insights into which system performs better under varying conditions.

\newpage

# Overview of Kubernetes Cluster Autoscaler

CAS operates by interacting with preconfigured node groups — groups of nodes defined within the cluster's infrastructure [@casgh]. When workloads increase, CAS assesses available resources and scales out by adding nodes from these predefined groups to meet demand. Conversely, when workloads decrease, CAS scales in by removing underutilized nodes, helping to optimize resource allocation and reduce costs [@cask8s].

Configuring CAS requires setting node groups with scaling parameters tailored to specific infrastructure needs. Node groups are defined by the cluster administrator and include parameters such as minimum and maximum node counts, resource constraints, and policies for scaling [@casaws]. These configurations vary by provider and are essential for CAS to understand the type and quantity of nodes it can scale [@cask8s]. This approach, however, can limit flexibility, as node groups must be configured manually to reflect the range of instances that may be required.

One key operational aspect of CAS is its response to unschedulable and pending pods. In cases where pods are pending due to lack of allocatable capacity on existing nodes, CAS will scale out by adding more nodes of the same type. If pods are unschedulable due to node type mismatches, CAS can provision nodes from zero-capacity node groups configured specifically for on-demand scaling. This allows CAS to handle both incremental scaling — meaning adding nodes to accommodate gradual workload growth — and burst scaling — meaning spinning up zero-capacity nodes to handle sudden demand [@casaws].

CAS follows a simplistic model for resource management (see [@fig:cas]), with features such as node cordoning and draining to manage underutilized nodes. When node utilization falls below a user-defined threshold, CAS will cordon and drain nodes to allow for graceful termination [@casaws]. However, CAS lacks active resource rebalancing found in tools like Karpenter, meaning it doesn’t actively consolidate workloads to optimize node usage but rather removes low-utilization nodes based on inactivity timers.

\newpage

# Karpenter Architecture and Functionality

Karpenter was initially developed by AWS in response to the growing need for more flexible and efficient autoscaling solutions in Kubernetes environments [@awskarp].

After its initial production release in November 2021, Karpenter has gained traction in the form of numerous code contributions by the Kubernetes community. AWS thus proposed contributing Karpenter to the Cloud Native Compute Foundation to further the community-driven development and facilitate compatibility across different cloud providers. In 2023, AWS officially donated karpenter-core to the Cloud Native Compute Foundation [@cncfdono].

## Overview

Karpenter manages pending pods by evaluating the resources required to schedule them. If the cluster lacks existing capacity that meets a pod's requirements, Karpenter dynamically provisions new nodes, essentially adding just-in-time [JIT] capacity [@jit]. For pods that can be accommodated by existing nodes, Karpenter schedules them without creating additional capacity [@Deepeshjaiswal2023Aug]. When workloads require resources beyond the current setup, such as a different instance type, Karpenter provisions nodes specifically tailored to meet these needs, based on predefined configurations [@Fontanilla2024Oct]. The system is designed to add only the exact resources necessary, thereby minimizing waste and ensuring that scaling is directly tied to actual demand [@mccune]. This combination of using existing capacity first and adding JIT capacity only when required makes Karpenter highly efficient at handling variable workloads without resorting to over-provisioning.

![Overview of the autoscaling flow used by Karpenter. Adapted from [@karpenter].](karpenter-diagram.png){width="560px" height="200px" #fig:karp}

## Operators

Kubernetes operators automate the management of complex applications on Kubernetes, encapsulating human operational knowledge within a software pattern known as the "operator pattern". Using custom controllers and Custom Resource Definitions [CRDs], operators extend the Kubernetes API to manage application-specific lifecycle tasks — such as deployments, upgrades, scaling, and recovery — without human intervention [@operatorframework].

![Operator pattern diagram adapted from [@operatorpattern].](operator-diagram.png){width="560px" height="120px" #fig:operatorpattern}

The operator pattern follows the idea of a "controller" monitoring a desired state, reconciling actual states to maintain that target state, the pattern is illustrated in [@fig:operatorpattern]. This model is ideal for applications requiring frequent, repetitive adjustments based on cluster conditions, as it enables autonomous, self-healing, and scalable behaviour tailored to application needs. Operators are widely used to manage stateful and complex systems, like databases and monitoring solutions, that benefit from automated operational intelligence [@operatorpattern], [@operatorsdk], [@operatorframework].

## Custom Resource Definitions

Karpenter introduces several custom resources to Kubernetes. How they relate to each other is illustrated in @fig:karpentercrd.

### NodeClaim

`NodeClaim` is a custom resource representing an active request to provision a node based on current workload needs. When a workload is unschedulable due to insufficient resources, Karpenter generates a `NodeClaim`, which outlines the specific resource requirements, such as CPU, memory, and other constraints, that the node must fulfil. Each `NodeClaim` is processed in real time according to the configurations specified in related Provisioner and `NodeClass` resources, ensuring that newly provisioned nodes meet the exact requirements of pending workloads [@nodeclaims]. This allows Karpenter to respond dynamically to scaling demands, creating nodes only as needed to efficiently allocate resources while minimizing costs.

### NodeClass

`NodeClass` is a custom resource that defines specific characteristics for nodes that may be provisioned, focusing on cloud provider-specific details. It allows users to configure node properties such as labels, taints, and networking settings (e.g., security groups or subnets). By abstracting these attributes, `NodeClass` provides a reusable blueprint for nodes, ensuring that all provisioned nodes meet required criteria for specific workloads or policies [@nodeclasses].

### NodePool

`NodePool` is a custom resource that groups multiple nodes managed by Karpenter under shared configuration parameters, often for scaling and resource management. It defines policies for capacity, scaling behaviour, and node lifecycle management, allowing for centralized control over node scaling within a designated group [@nodepools].

![How Karpenter Custom Resources interact with each other.](karpenter-crds-diagram.png){width="560px" height="320px" #fig:karpentercrd}

## Controller

To better understand the components of the controller functions, a UML diagram was drawn based on the code (see @fig:awsuml). The diagram illustrates that the `NodeClaimController` interacts with the `CloudProvider` to create, update, or delete compute and networking resources in the cloud, ensuring that node requests are appropriately provisioned or removed. The `InstanceProvider` plays a key role in matching resource requests with optimal instance types while managing the lifecycle of compute instances, including launching, maintaining, and terminating them as needed. The `LaunchTemplateProvider` is responsible for generating or retrieving launch templates, which act as blueprints for compute instances, defining their configuration, such as instance type, storage, networking, and user data scripts.

![UML Diagram of AWS Karpenter Provider. Adapted from [@kaws].](karpenter-aws-uml.png){width="560px" height="380px" #fig:awsuml}

\newpage

# Development of the IBM Cloud Karpenter Provider

At the time of development, the documentation for building a custom Karpenter provider was limited, necessitating additional support from the community and experts with practical experience. One such expert, Michael McCune, who had previously developed the Cluster API Karpenter provider, provided valuable insights and recommendations. His experience in writing a provider from scratch was instrumental in navigating the challenges associated with the development of the IBM Cloud Karpenter provider.

McCune recommended the use of the Kubernetes Without Kubelet [KWOK] [@kwok] Karpenter provider [@kwok_karpenter] as a minimal blueprint for building the IBM Cloud provider.

Additionally, the AWS Karpenter provider code was reviewed to provide a frame of reference to understand how the most mature Karpenter provider works.

## Secure Software Development Lifecycle

The Secure Software Development Life Cycle [SSDLC] [@ssdlc] for the IBM Karpenter provider adhered to a security-focused methodology. The process began with requirements gathering and a risk assessment, during which a design document was created to outline the foundational features of the IBM Karpenter implementation. An important security concern identified during this phase was the potential for cloud credential leakage, which could lead abuse. To mitigate this risk, the secure design principle of Short-Lived Token-Based Authentication was employed. This principle relies on the use of a master token -- which in this case was stored in a Kubernetes Secret resource. This master then token generates short-lived tokens on demand, minimizing the risk of abuse in case a short-lived token were to be leaked [@Kumar2024Security]. A common use case of Short-Lived Token-Based authentication is users logging into websites that use JSON Web Tokens [JWT] or the OAuth2 protocol. Threat modelling didn't identify any further immediate risks.

During development, static analysis tool Checkmarx [@checkmarx] was used to identify security flaws in the codebase. Additionally, security scanning, and proactive dependency vulnerability checks via Dependabot [@dependabot] were integrated into the workflow. To ensure code quality, linting and testing tools for GO code [@golint], [@gotest] and Helm charts [@helmlint], [@helmtest] were incorporated into CI pipelines, allowing potential issues to be identified and resolved before deployment.

The deployment phase involved deploying the provider to a test cluster, where bugs were addressed, and the implementation was fine-tuned before live experimentation. CI pipelines automated builds ensuring rapid, standardized, and secure lifecycle management. Within the CI/CD workflow, container images were scanned for vulnerabilities on ghcr.io [@ghcr] using the built-in automated scanning tools to detect and report security issues. This approach minimized risks across the software lifecycle.

## Architecture

The Karpenter Provider for IBM Cloud extends Karpenter’s node provisioning capabilities by integrating with IBM Cloud infrastructure. It utilizes two distinct API keys: the IBM Cloud API key for general operations and Global Catalog access, and the VPC API key specifically for VPC operations. The provider implements token management, including auto-generating short-lived tokens for the Global Catalog API. This approach incorporates token caching and automatic refresh five minutes before expiration, reducing reliance on long-lived credentials and strengthening the security posture.

The architecture consists of two main layers: the Controller Layer and the Provider Layer. The Controller Layer includes the Main Controller, which acts as the entry point and orchestrates component interactions, and Custom Resource Controllers, which manage resources such as `IBMNodeClass`, `NodePool`, and `NodeClaim`. The Provider Layer handles IBM Cloud-specific logic through several components: the Cloud Provider -- for core integration logic, Instance Provider -- for VM lifecycle management, Instance Type Provider -- for instance validation and selection, and Subnet Provider -- for advanced subnet management. The Subnet Provider incorporates strategies like capacity-based scoring, IP fragmentation analysis [@Yin2009Measuring], tag-based filtering, and zone distribution.

The provider is deployed using a Helm chart [@helm], supporting namespace isolation [@ns] and including health and readiness monitoring [@garrison2017cloud]. This architecture effectively extends Karpenter’s dynamic scaling capabilities to IBM Cloud.

### Instance Type Generator

The instance type generator is used for dynamically identifying and selecting appropriate node types, as Karpenter requires up-to-date information on available instance types for optimized provisioning. It retrieves and compiles data on various instance types, such as CPU, memory, and cost, which is necessary to match workloads to the best-fitting infrastructure. In the IBM Cloud context, implementing this was challenging due to fragmented API sources and inconsistent data availability. To overcome this, multiple IBM Cloud APIs were used as sources, combining data from infrastructure and pricing sources to ensure a comprehensive view of instance types.

![Overview of Instance Type Generator written for IBM Cloud Karpenter Provider.](gen-instance-diagram.png){width="560px" height="200px" #fig:ibmgen}

IBM VPC API retrieves detailed information on instance profiles (e.g., CPU and memory specifications) [@ibmvpc] through the `fetchInstanceProfiles` function.

IBM Global Catalog API provides pricing data for instance types [@ibmcatalog]. The `fetchPricing` function queries the IBM Global Catalog API to get cost-related information for each profile, enabling Karpenter to make optimized decisions based on workload requirements and cost efficiency.

Where data gaps existed, fallbacks and defaults were implemented, allowing the generator to maintain reliable output even when specific information was missing. The program outputs a json file containing all the metadata, which is taken into account by the Karpenter Controller. The data flow can be observed in @fig:ibmgen.

## Controller

The development of the IBM Karpenter Provider's controller began with a minimal viable implementation to establish basic functionality. However, during the iterative development process, additional mechanisms were identified as essential for the provider to function effectively in real-world scenarios. The final implementation consists of two main controllers: the `IBMCloudReconciler` (`NodeClaim` Controller) and the Autoplacement Controller. These controllers work in tandem to manage node provisioning and instance type selection, ensuring seamless integration with IBM Cloud infrastructure.

The `IBMCloudReconciler`, located in `cmd/controller/main.go`, is the primary controller responsible for handling the lifecycle of `NodeClaims`. It provisions new nodes in IBM Cloud when a `NodeClaim` is created and ensures proper deletion and clean-up of resources when a `NodeClaim` is marked for deletion, using Kubernetes finalizers. This controller relies on the `CloudProvider` interface to interact directly with IBM Cloud infrastructure, orchestrating node provisioning and deletion in response to workload requirements.

![Simplified UML Diagram of IBM Karpenter Provider.](karpenteropnew.png){width="560px" height="250px"}

The Autoplacement Controller, located in `pkg/controllers/nodeclass/autoplacement/controller.go`, manages the `IBMNodeClass` resources by handling the automatic selection of instance types and subnets based on the specifications defined in the `IBMNodeClass`. It updates the `IBMNodeClass` with selected instance profiles and maintains status conditions reflecting the current state of autoplacement.

The two controllers work collaboratively within a reconciliation loop, following standard Kubernetes controller patterns [@controller]. The Autoplacement Controller determines the most appropriate instance types and subnets for the specified requirements in `IBMNodeClass` resources. The `NodeClaim` Controller then uses this information to provision nodes in IBM Cloud. By integrating with providers for instance, instancetype, and subnet management, the controllers enable dynamic, efficient, and scalable infrastructure provisioning.

The metrics exposed by the Autoplacement Controller provide valuable insights into system performance and efficiency, allowing operators to monitor success and failure rates of selections, resource usage, latency, and the overall health of the autoplacement functionality.

\newpage

# Development of Lab Automation and Tooling

In order to ensure the consistency, accuracy, and repeatability of the experiments conducted during this research, it was necessary to create automation tools that could handle the entire lifecycle of the cloud infrastructure [@Aviv2023Infrastructure],[@Shetty2021Cloud], [@Tamagnan2024Generation]. This minimizes human error and variability in the setup, providing a reliable and repeatable basis for comparing performance metrics [@Chandakanna2014A]. Additionally, automation reduces the time and effort needed to run multiple experiments, allowing for a more thorough and consistent data analysis across different cloud autoscaling methods. Without automation, reproducing identical conditions for each test would be impractical, making the results less reliable and harder to compare.

## Provisioning Cloud Infrastructure

Contain the whole experiment process from setup to teardown in an end-to-end fashion while maintining flexibility was achieved through selective task execution via task tagging. A high-level overview of the steps covered by the automation can be observed in @fig:playbook.

![Overview of Lab Lifecycle Playbook.](experiment-automation.png){width="560px" height="170px" #fig:playbook}

Kubeconfig files provide secure authentication to Kubernetes clusters. These files contain the configuration details required for a user or application to connect to a cluster, including the API server address, user credentials, and the necessary certificates for secure communication. [@kubeconfig]

Secrets management practices like using Ansible vault to encrypt sensitive information such as credentials, and gitignore policies [@gitignore] to exclude files such as kubeconfigs from version control are in place. Dependabot is used on GitHub to scan the code for vulnerabilities in dependencies.

<!-- prettier-ignore -->
~~~~~~{#lst:vault .yaml .numberLines caption="Ansible Vault content example."}
IBM_CLOUD_TOKEN: "redacted"
RESOURCE_GROUP_ID: "redacted"
VPC_ID: "redacted"
SUBNET_ID:
  US_SOUTH_1: "redacted"
  US_SOUTH_2: "redacted"
  US_SOUTH_3: "redacted"
AWS_ACCESS_KEY_ID: "redacted"
AWS_SECRET_ACCESS_KEY: "redacted"
SUDO_PASSWORD: "redacted"
~~~~~~

## Configuring the Clusters

Helm, as a Kubernetes-native package manager, allowed for declarative management of complex applications [@helm], reducing the likelihood of human error. The Kube Prometheus Stack Helm Chart [@promstack] was deployed in each cluster to establish the monitoring infrastructure. Depending on the cluster configuration, either CAS or Karpenter was deployed. For CAS, the Helm chart was used with parameters tailored to the cloud provider (e.g., AWS EKS or IBM IKS). Karpenter was deployed using a custom Helm chart that aligned with the provider-specific configurations developed during this study.

Service accounts were created for Prometheus to allow the automation to scrape metrics. The principle of least privilege [POLP] [@plop] was strictly followed to minimize the risk of unauthorized access. Role-based access control [RBAC] configurations were applied to limit permissions to only what was necessary for metrics collection.

Initially, kube-burner [@burnerprom] was used to run load test and capture metric during the load test. One challenge encountered during setup was the intermittent unavailability of Prometheus endpoints during load tests. To mitigate this, the Ansible playbook was extended to take a complete backup of the Prometheus time-series database [TSDB] [@tsdbprom] after each experiment using the admin API [@adminapi]. This ensured data integrity and completeness for post-experiment analysis.

Additionally, retry logic was implemented for tasks prone to failure, such as fetching metrics or verifying endpoint readiness. Configurable retry limits and delays allowed for resilience against transient errors, minimizing disruptions during experimentation. By decoupling dependencies and implementing robust error-handling strategies, the architecture maintained flexibility and reliability, aligning with loose coupling for distributed systems [@Osbakk2015Architectural].

<!-- prettier-ignore -->
~~~~~~{#lst:retry .yaml .numberLines caption="Retry logic example from the metrics collection task."}
- name: Fetch metrics from prometheus
  ansible.builtin.command:
    cmd: "curl -s -f 'http://127.0.0.1:{{ item.1.port }}/api/v1/query_range?query={{ item.0.query | urlencode }}&start={{ start_time }}&end={{ end_time }}&step=30s'"
  register: query_response
  until: query_response.rc == 0
  retries: "{{ metrics_collection.query_retries }}"
  delay: "{{ metrics_collection.query_delay }}"
  loop: "{{ prometheus_queries | product(cluster_list) | list }}"
  loop_control:
    loop_var: item
    label: "{{ item.1.cluster }} - {{ item.0.metricName }}"
  vars:
    cluster_list:
      - {cluster: 'karpenter', port: "{{ '9093' if cloud_provider == 'eks' else '9091' }}"}
      - {cluster: 'cas', port: "{{ '9094' if cloud_provider == 'eks' else '9092' }}"}
~~~~~~

## Running Load Tests

Running load tests is also automated end-to-end from setup to metrics collection. A series of workload manifests are deployed to each cluster and later removed using ansible.

Timestamp-based queries for the selected metrics to the Prometheus API ensure that experiment data is accurately recorded during the experiment windows.
Additionally, cloud provider billing metrics are collected from AWS and IBM Cloud using tag-based resource filtering and respective APIs [@ibm_tagging], [@aws_tagging], with the data stored in structured JSON files organized by provider, cluster, and workload scenario. To ensure reliability, the automation incorporates health checks for service readiness, configurable retry mechanisms, and robust error handling to manage any issues with service management or data retrieval.

## Collecting Metrics

The metrics selected for this analysis were chosen to address the research questions by providing a comprehensive view of the autoscaling performance, resource efficiency, and operational costs across AWS and IBM Cloud environments. These metrics—node latency, node count, CPU utilization, memory utilization, and cost—are central to evaluating the effectiveness of autoscaling solutions. By quantifying these parameters, the study aligns with the research aims of assessing scaling speed, resource provisioning accuracy, and cost-efficiency under varying workloads.

Node latency was included as a primary indicator of scaling speed and responsiveness, directly addressing $RQ_1$: "How does the scaling speed of Karpenter compare to traditional Kubernetes autoscaling mechanisms in responding to fluctuating workloads in IBM Cloud environments?" Faster node provisioning and lower latency are important in dynamic cloud environments where delayed resource allocation can result in performance bottlenecks [@Son2019Latency-aware].

Node count serves as a measure of the scaling efficiency and workload consolidation achieved by the autoscaling mechanisms. It directly reflects the ability of the autoscaler to provision the minimum necessary resources, thereby reducing costs and optimizing infrastructure utilization [@Cheng2023ProScale]. This metric is integral to answering $RQ_3$, which focuses on operational costs.

CPU and memory utilization are key indicators of resource efficiency, addressing $RQ_2$: "How efficiently do Karpenter and traditional Kubernetes autoscaling mechanisms manage CPU and memory resources under varying workload conditions on IBM Cloud?" These metrics provide insights into how well the autoscaler allocates and utilizes resources to meet workload demands while avoiding over- or under-provisioning.

Finally, operational cost metrics—including hourly and monthly costs—offer a tangible measure of economic efficiency. By correlating resource utilization with cost, these metrics address both $RQ_2$ and $RQ_3$, illustrating how effectively each autoscaling mechanism minimizes financial overhead while maintaining performance.

The selection of these metrics ensures a holistic evaluation of the autoscalers, balancing technical performance, resource efficiency, and cost considerations. Together, they provide the necessary framework to assess the research hypotheses and draw meaningful comparisons across cloud platforms and autoscaling approaches.

## General Experiment Parameters

### Infrastructure

To have a frame of reference for collected experiment metrics, the same experiments are carried out on the AWS Cloud Platform using parameters that match the IBM Cloud setup as closely as possible.

Flagship datacenter locations are used for both providers. For IBM Cloud this is their "us-south" region (Dallas, United States) [@ibmdcs] and for AWS this is their "us-east-1" region (N. Virginia, United States) [@awsdcs]. This allows a fair comparison based on datacenter standard. The datacenter's physical location is disregarded as it is not a factor for the self-contained experiments.

Instance types closely aligned general purpose, non burst with same specifications. Possible that there are deviations on processor model and thus clock speed. Larger sample size/repeating experiments should normalize such small individual deviations.

: Comparison of cluster specifications on AWS and IBM Cloud.

| Cloud Provider | Instance Type | Processor  | CPU Cores | CPU Architecture | Memory (GiB) | Node Operating System |
| -------------- | ------------- | ---------- | --------- | ---------------- | ------------ | --------------------- |
| IBM Cloud      | bx2.2x8       | Intel Xeon | 2         | x86_64           | 8            | Ubuntu                |
| AWS            | m5.large      | Intel Xeon | 2         | x86_64           | 8            | Amazon Linux          |

### Cost tracking

Both IBM Cloud and AWS offer tagging mechanisms that were used to track the cluster resources for cost comparison purposes [@ibm_tagging], [@aws_tagging]. This approach allowed for a clear understanding of cost differences specific to cluster management and usage between the two platforms.

Promotional discounts and non-linear pricing can be a challenge. Therefore, the list prices are compared to the pricing data provided by the billing APIs as a control mechanism.

### Kubernetes Clusters

The experiments were conducted on managed Kubernetes services provided by each cloud vendor: IBM Kubernetes Service [IKS] [@iks_def] and Amazon Elastic Kubernetes Service [EKS] [@eks_def]. Both clusters were deployed across three availability zones to ensure high availability [@multiazk8s] and to simulate a typical production environment [@rosso2021production]. The Kubernetes versions used in the experiments were the same across both platforms, specifically matching the same major and minor versions to avoid discrepancies due to version differences [@skew]. Each cluster had a pool of three worker nodes at creation time with a hard limit of a maximum set to 10 worker nodes.

![Overview of Kubernetes Clusters created in the experiments across different platforms.](clusters-overview-diagram.png){width="560px" height="220px"}

### Experiment length

Each experimental scenario was executed for a duration of one hour to ensure the collection of meaningful performance and cost data. This time frame aligns with IBM Cloud’s hourly billing model, which provides cost granularity suited to the nature of the experiments. AWS, however, offers more fine-grained billing, but for consistency and ease of comparison, the same hourly interval was applied to both environments. This uniform duration ensures that the results are directly comparable, particularly in terms of resource usage and associated costs. The main experiments happen in the first 20 minute slice. Analysing the first 20 minutes allowed for a detailed evaluation of scaling speed, and efficiency during the most demanding period of resource provisioning, making it the most interesting slice of the overall experimental timeline.

### Load Test Scenario 1: Homogenous Workloads

The first scenario benchmarks autoscaling performance using uniform workloads across both Karpenter and CAS clusters. 40 replicas of identical workloads are deployed, each with consistent CPU and memory resource requests and limits. The workloads are categorized into four types: CPU-intensive, memory-intensive, network-intensive, and I/O-intensive, with 28 replicas assigned to each category. This homogeneous workload setup serves as a baseline for performance comparison, enabling a thorough evaluation of autoscaling efficiency under consistent resource demands.

### Load Test Scenario 2: Heterogeneous Workloads

The heterogeneous workload scenario evaluates autoscaling performance under varying resource demands, simulating real-world conditions where production clusters handle a mix of workloads with different resource intensities. This test deploys workloads in three sizes—small, medium, and large—across CPU and memory-intensive tasks. For CPU-intensive workloads, the distribution is 20 replicas for large, 30 for medium, and 40 for small, with each category. These workloads are run concurrently, reflecting the dynamic and unpredictable nature of real-world applications. This setup allows for an evaluation of how effectively the autoscaling system handles concurrent workloads of varying sizes and resource demands, providing a more comprehensive assessment of autoscaling performance in environments with diverse and fluctuating workloads.

### Common Features Across Both Scenarios

Both load test scenarios share several key features designed to provide the desired insights into the autoscaling performance of the clusters. Debugging tasks are conducted both before and after the experiments to ensure the reliability of the results and address any potential issues. To maintain consistency between workload transitions, a stabilization period is implemented between workload changes, allowing the system to adjust and reach a stable state before further evaluation. The tests include a clean-up phase upon completion to trigger a scale-down event. This scale-down event can be observed well in e.g. @fig:nodelatency, @fig:awscpu, @fig:ibmcpu at around the 11 minute mark.

\newpage

# Discussion

The discussion section is organized into two main parts: an Overview of Key Findings, which summarizes the primary results of the study, and an Interpretation of Findings, which provides a deeper analysis of these results in relation to the research questions and existing literature. The Overview of Key Findings focuses on quantifiable outcomes of experiments while the Interpretation of Findings contextualizes these outcomes, addressing platform-specific constraints and their alignment with prior research.

## Overview of Key Findings

The data suggest that IBM Karpenter shows measurable improvements over IBM CAS in key metrics such as node count reduction and memory utilization efficiency, with statistical analysis supporting these differences. For example, under Scenario 1 (homogeneous workloads), IBM Karpenter reduced the average node count to 5.10 compared to 6.44 for IBM CAS, with standard deviations of 2.01 and 2.24, respectively, confirming a statistically significant difference (t-test p = 0.0124, ANOVA p = 0.0124; see [@tbl:nodehomo]; [@fig:ibmnodecount]). Similarly, memory utilization improved with IBM Karpenter consuming an average of 6,513 MB compared to 7,569 MB for IBM CAS, though the standard deviations (2,376.90 vs. 2,867.59 MB) and higher p-values (t-test p = 0.2243, ANOVA p = 0.2243) suggest this improvement is also statistically significant (see [@tbl:memoryhomo]; [@fig:ibmmem]). These gains, however, are modest compared to AWS Karpenter, which achieved an average node count of 3.61 (std dev = 0.49, t-test p = 0.0001) and memory utilization of 4,332 MB (std dev = 1,747.90, t-test p = 3.38e-08) under the same conditions (see [@tbl:comphomo]; [@fig:awsnodecount]; [@fig:awsmem]).

While both t-tests and ANOVA were conducted, the results for two-group comparisons (e.g., IBM Karpenter vs IBM CAS) are equivalent. The use of ANOVA was included to ensure consistency across analyses and to accommodate potential comparisons involving more than two groups in future expansions.

: Overview of the Load Test Scenario Metrics. {#tbl:summary}

| **Scenario**   | **Metric**                    | **AWS CAS** | **AWS Karpenter** | **IBM CAS** | **IBM Karpenter** |
| -------------- | ----------------------------- | ----------- | ----------------- | ----------- | ----------------- |
| **Scenario 1** | Average Node Count            | 5.37        | 3.61              | 6.44        | 5.10              |
|                | Hourly Cost                   | $0.87       | $0.58             | $0.71       | $0.57             |
|                | Monthly Cost                  | $626.40     | $417.60           | $511.20     | $410.40           |
|                | Node Latency                  | 22.46 ms    | 13.34 ms          | 16.00 ms    | 17.42 ms          |
|                | Avg. Total CPU Utilization    | 4.82 cores  | 5.02 cores        | 3.72 cores  | 3.62 cores        |
|                | Avg. Total Memory Utilization | 4176.86 MB  | 4332.18 MB        | 7568.99 MB  | 6513.64 MB        |
| **Scenario 2** | Average Node Count            | 4.62        | 3.51              | 6.61        | 5.00              |
|                | Hourly Cost                   | $0.75       | $0.71             | $0.73       | $0.56             |
|                | Monthly Cost                  | $540.00     | $511.20           | $525.60     | $403.20           |
|                | Node Latency                  | 16.86 ms    | 8.95 ms           | 13.46 ms    | 14.59 ms          |
|                | Total CPU Utilization         | 3.91 cores  | 2.92 cores        | 2.74 cores  | 2.33 cores        |
|                | Total Memory Utilization      | 2834.39 MB  | 2486.21 MB        | 6510.66 MB  | 4686.19 MB        |

The data supports the hypothesis that Karpenter offers enhanced resource provisioning efficiency compared to CAS. On AWS, Karpenter demonstrated significant advantages, achieving the lowest node latency of 11.76 ms (std dev = 1.42, t-test p = 3.94e-10, ANOVA p = 2.68e-11) compared to 21.16 ms for AWS CAS (std dev = 6.40; see [@tbl:nodelatencyhomo]; [@fig:awsnodelatency]). On IBM Cloud, Karpenter reduced latency to 17.07 ms (std dev = 2.28) compared to 18.18 ms for IBM CAS (std dev = 2.40), with statistical significance (t-test p = 9.06e-12, ANOVA p = 8.18e-12; see [@tbl:nodelatencyhomo]; [@fig:nodelatency]). These latency reductions translated into substantial cost savings, with AWS Karpenter reducing monthly costs by 33% (from $626.40 to $417.60, p < 0.01), whereas IBM Karpenter achieved a 20% reduction (from $511.20 to $410.40, p < 0.05; see [@tbl:summary]).

: Node Latency Statistics (Homogeneous). {#tbl:nodelatencyhomo}

| **Platform** | **Provider** | **Mean (ms)** | **Mode (ms)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ------------- | ------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 21.16         | 11.76         | 6.40        | 0.019              | 0.0198            |
| AWS          | Karpenter    | 11.76         | 10.00         | 1.42        | 3.94e-10           | 2.68e-11          |
| IBM          | CAS          | 18.18         | 15.32         | 2.40        | 6.65e-12           | 6.58e-12          |
| IBM          | Karpenter    | 17.07         | 14.55         | 2.28        | 9.06e-12           | 8.18e-12          |

: Node Latency Statistics (Heterogeneous). {#tbl:nodelatencyhetero}

| **Platform** | **Provider** | **Mean (ms)** | **Mode (ms)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ------------- | ------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 15.67         | 14.05         | 0.69        | 0.273              | 0.2691            |
| AWS          | Karpenter    | 13.93         | 11.27         | 1.07        | 1.74e-06           | 3.30e-08          |
| IBM          | CAS          | 16.04         | 15.65         | 0.15        | 0.5298             | 0.5211            |
| IBM          | Karpenter    | 13.31         | 11.65         | 1.06        | 0.0017             | 0.0009            |

Persistent inefficiencies in IBM’s scaling mechanisms constrain Karpenter’s potential on IBM Cloud. For instance, IBM Karpenter required an average of 5.10 nodes (std dev = 2.01) compared to 6.44 nodes for IBM CAS (std dev = 2.24), whereas AWS Karpenter required significantly fewer nodes at 3.61 (std dev = 0.49, t-test p = 1.31e-05) compared to AWS CAS at 5.37 (std dev = 1.20; see [@tbl:nodehomo]; [@fig:ibmnodecount]). Similar patterns are evident in memory utilization, where IBM Karpenter achieved modest improvements over IBM CAS but fell short of the efficiencies demonstrated by AWS (see [@tbl:memoryhomo]; [@fig:ibmmem]).

: Node Count Statistics (Homogeneous). {#tbl:nodehomo}

| **Platform** | **Provider** | **Mean** | **Mode** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | -------- | -------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 5.37     | 6.00     | 1.20        | 0.3499             | 0.352             |
| AWS          | Karpenter    | 3.61     | 4.00     | 0.49        | 7.48e-05           | 6.89e-05          |
| IBM          | CAS          | 6.44     | 6.00     | 2.24        | 0.0147             | 0.0124            |
| IBM          | Karpenter    | 5.10     | 3.00     | 2.01        | 0.9942             | 0.9941            |

: Node Count Statistics (Heterogeneous). {#tbl:nodehetero}

| **Platform** | **Provider** | **Mean** | **Mode** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | -------- | -------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 4.62     | 6.00     | 1.81        | 0.0037             | 0.0032            |
| AWS          | Karpenter    | 3.51     | 4.00     | 1.23        | 1.38e-05           | 1.31e-05          |
| IBM          | CAS          | 6.61     | 9.00     | 2.24        | 0.039              | 0.0345            |
| IBM          | Karpenter    | 5.00     | 7.00     | 1.86        | 0.513              | 0.514             |

CPU utilization also highlights platform-specific limitations. AWS Karpenter demonstrated higher CPU utilization efficiency with a mean of 5.02 cores (std dev = 4.82, p = 1.69e-07) compared to 4.82 cores for AWS CAS (std dev = 4.52), while IBM Karpenter showed marginal improvements at 3.62 cores (std dev = 4.12) compared to 3.72 cores for IBM CAS (std dev = 4.28, p = 4.54e-05; see [@tbl:cpuhomo]; [@fig:ibmcpu]). For heterogeneous workloads, these trends persisted, with AWS Karpenter maintaining the most efficient scaling across all metrics (see [@tbl:cpuhetero]; [@fig:awscpu]).

Overall, the statistical values and comparative performance strongly establish AWS Karpenter as the most efficient autoscaling solution across all tested scenarios, addressing $RQ_1$ by demonstrating superior scaling speed and responsiveness. While IBM Karpenter shows measurable improvements over CAS, particularly in node count and memory utilization, IBM’s platform-specific inefficiencies, such as slower instance provisioning and higher baseline resource usage, limit its performance, aligning with $RQ_2$ on resource efficiency. These findings emphasize the importance of robust cloud infrastructure in fully leveraging advanced autoscaling tools like Karpenter. The cost metrics respond to $RQ_3$ by emphasizing that while Karpenter reduces operational costs compared to CAS, the degree of savings is influenced by the underlying cloud infrastructure.

## Interpretation of Findings

<!-- why does it matter-->

<!-- This section will typically be the longest one in the chapter, as this is where you’ll spend time unpacking and interpreting your findings and explaining what they mean within your context. In your previous chapter (the results chapter), you purely described the data, whereas in this chapter you will explain what it means in relation to the golden thread. -->
<!-- An important note – everything you discuss in this chapter needs to be rooted in data that you presented in your previous chapter (i.e., the results chapter). So, make sure that you only include in this chapter what you reported in your results chapter – don’t introduce new information. If you need to, you can always go back and expand upon your results chapter so that there is a strong foundation for the discussion chapter. -->
<!-- At its core, the goal of this section is to assess how your findings compare to those of the existing research (i.e., the studies cited in your literature review chapter). By doing this, you’ll communicate what the data means in your context, and you’ll position it in relation to existing studies (i.e., does it align or misalign with existing research?).  -->

<!-- Below are a few questions to help guide your discussion in this section: -->
<!-- How do your findings relate with those of previous studies? -->
<!-- If your findings differ from those of previous studies, why may this be the case? -->
<!-- What do your findings contribute to your field of research? -->
<!-- What other explanations could there be for your findings? -->

<!-- One way is to structure this section similarly to how you structured the results chapter. This could be by research question, variables or groups of variables. -->

The findings indicate that Karpenter outperforms CAS in key areas such as scaling efficiency, responsiveness, and resource utilization, aligning with prior non-academic sources [@Fontanilla2024Oct], [@mccune] that emphasize Karpenter’s dynamic provisioning capabilities and workload-specific optimization (see [@tbl:summary]; [@tbl:nodelatencyhomo]). Unlike CAS, which relies on pre-configured node pools and static scaling parameters, Karpenter dynamically provisions nodes based on real-time workload requirements, adapting more effectively to heterogeneous environments. This ability was demonstrated through Karpenter’s ability to consistently provision fewer nodes than CAS while maintaining similar or better performance across both AWS and IBM Cloud.

The literature underscores the benefits of workload-specific scaling mechanisms, as highlighted by Rosso et al. (2021). Karpenter’s ability to dynamically provision resources tailored to application requirements aligns with the notion of optimized resource allocation for diverse workloads. This approach reduces over-provisioning and enhances performance, making Karpenter particularly well-suited for environments with variable and complex resource needs [@rosso2021production].

Both Rosso et al. [@rosso2021production] and Dimolitsas et al. [@Dimolitsas2023Multi-Application], [@Dimolitsas2022AHP4HPA] highlight the necessity of adaptability in scaling mechanisms to handle dynamic and heterogeneous workloads. Karpenter's real-time provisioning capabilities address this need by allowing for the rapid adjustment of resources to meet fluctuating demand. This feature is particularly valuable in edge environments or contexts like smart cities, where resource constraints and quality of service requirements demand agile scaling solutions.

### Cluster Topology Observations

Karpenter demonstrated superior node scaling efficiency compared to CAS, as seen in its ability to provision fewer nodes to handle equivalent workloads (see [@tbl:nodehomo]; [@fig:awsnodecount]). Similarly, IBM Karpenter reduced node count (see [@tbl:nodehomo]; [@fig:ibmnodecount]), albeit less dramatically. These findings align with prior research that highlights the inefficiencies of CAS in handling dynamic scaling needs due to its reliance on pre-defined node pools [@Fontanilla2024Oct], [@mccune]. However, IBM Karpenter’s performance relative to AWS Karpenter suggests that the underlying cloud platform also significantly impacts scaling efficiency.

![Comparison of Node Counts across scenarios on AWS.](aws_node_count_comparison.png){width="620px" height="280px" #fig:awsnodecount}

![Comparison of Node Counts across scenarios on IBM Cloud.](ibm_node_count_comparison.png){width="620px" height="280px" #fig:ibmnodecount}

Dimolitsas et al. emphasize the importance of minimizing energy consumption in cloud environments while maintaining high performance [@Dimolitsas2023Multi-Application], [@Dimolitsas2022AHP4HPA]. While this study does not directly measure energy efficiency, Karpenter’s improvements in resource utilization, such as higher CPU and memory efficiency, indirectly contribute to reduced energy consumption. By provisioning fewer nodes and utilizing resources more effectively, Karpenter aligns with energy-aware orchestration principles, which are critical for sustainability and operational cost reduction

### Performance Observations

In terms of latency, Karpenter outperformed CAS across both cloud platforms, with AWS Karpenter achieving the lowest latency of 11.76 ms under homogeneous workloads, compared to 21.16 ms for AWS CAS and 17.07 ms for IBM Karpenter (see [@tbl:nodelatencyhomo]). This is indicative of Karpenter’s ability to make rapid scaling decisions and provision resources within seconds, a feature particularly critical for workloads with unpredictable demand. IBM CAS and Karpenter, while closer in performance, showed higher latency overall (see [@fig:nodelatency]), which may point to IBM Cloud’s architectural limitations, including slower instance provisioning and higher baseline overheads.

![Comparision of Node Latency across scenarios on IBM Cloud.](ibm_node_latency_comparison.png){width="620px" height="280px" #fig:nodelatency}

Resource utilization further underscores Karpenter’s advantage. AWS Karpenter maintained a balance between higher CPU utilization (5.02 cores versus 4.82 cores for AWS CAS) and improved memory efficiency (4,332 MB versus 4,176 MB) (see [@tbl:cpuhomo]; [@tbl:memoryhomo]). By contrast, IBM Karpenter reduced memory usage significantly compared to IBM CAS (6,513 MB vs. 7,569 MB) (see [@fig:ibmmem]) but still fell short of AWS in terms of overall efficiency. This suggests that while Karpenter’s optimization mechanisms are effective, their implementation on IBM Cloud may be constrained by platform-specific inefficiencies, such as limited support for features like spot instances and slower instance type matching.

![Comparison of Total Memory Utilization across scenarios on IBM Cloud.](ibm_memory_utilization_absolute_comparison.png){width="620px" height="280px" #fig:ibmmem}

These findings align with those reported by Tamiru et al. (2020), who evaluated the performance of the Kubernetes Cluster Autoscaler in various cloud environments [@caslul]. Their results included CAS's limitations in handling dynamic workloads due to its dependency on pre-defined node pools, which leads to slower scaling decisions and suboptimal resource allocation. By contrast, Karpenter addresses these shortcomings through real-time provisioning, resulting in improved scaling efficiency and resource utilization. Additionally, Tamiru et al. found that the underlying cloud platform significantly influences the effectiveness of autoscaling strategies, a conclusion echoed in this study's observation of IBM Cloud's platform-specific inefficiencies in supporting Karpenter's advanced features. These parallels reinforce the importance of considering both autoscaler design and cloud platform capabilities in optimizing workload performance. These findings also reinforce the broader conclusion from Dimolitsas et al. [@Dimolitsas2023Multi-Application], [@Dimolitsas2022AHP4HPA] that effective resource management is tightly linked to the underlying platform’s capabilities.

### Real-World Usage Examples

The findings suggest that Karpenter is particularly well-suited for environments with fluctuating workloads requiring rapid scaling. For example, scenarios involving dynamic workloads, such as e-commerce platforms during seasonal sales or machine learning pipelines with irregular demand, could benefit from Karpenter’s ability to provision appropriately sized nodes in real time. AWS Karpenter, in particular, demonstrates the potential to leverage cost-saving mechanisms like spot instances effectively, achieving monthly cost reductions of up to 33% compared to CAS ($417.60 vs. $626.40 under homogeneous workloads) (see [@tbl:summary]).

IBM Karpenter, while improving on CAS, appears less effective in real-world usage due to the limitations of the IBM Cloud platform. For instance, the lack of advanced features like predictive scaling or optimized spot instance handling, as well as higher baseline memory usage, reduces its competitiveness relative to AWS Karpenter. These limitations highlight the importance of the underlying cloud infrastructure in enabling advanced autoscaling capabilities. This aligns with insights from Lee and Kim (2019), who note that cloud platform architecture plays a pivotal role in determining the success of autoscaling strategies, with factors such as provisioning speed and support for cost-optimization mechanisms influencing overall performance [@Choi2015The].

\newpage

# Conclusions

This chapter summarizes the study’s findings on Karpenter’s performance compared to CAS, discusses its limitations, and offers recommendations for future research to address gaps and build on these results.

## Summary of Findings

This study aimed to evaluate the performance of Karpenter in comparison to the traditional CAS, focusing on three key dimensions: scaling speed, operational costs, and resource efficiency. These dimensions were analyzed within the context of IBM Cloud environments, with additional comparisons to AWS as a benchmark.

$RQ_1$
: How does the scaling speed of Karpenter compare to traditional Kubernetes autoscaling mechanisms in responding to fluctuating workloads in IBM Cloud environments?

The results indicate that Karpenter provides faster scaling compared to CAS, particularly on AWS. In homogeneous workloads, AWS Karpenter reduced node latency, compared to AWS CAS (see [@tbl:nodelatencyhomo]). On IBM Cloud, Karpenter exhibited less dramatic improvements, with latency increasing slightly under homogeneous conditions (see [@tbl:summary]). These differences highlight the influence of platform-level optimizations, with AWS offering more support for rapid resource provisioning than IBM Cloud.

$RQ_2$
: What are the differences in operational costs between Karpenter and traditional Kubernetes autoscaling mechanisms in IBM Cloud environments?

Karpenter demonstrated significant cost savings compared to CAS, especially on AWS. For homogeneous workloads, AWS Karpenter reduced monthly costs by 33%, while IBM Karpenter reduced costs by approximately 20% (see [@tbl:summary]). These savings reflect Karpenter's ability to dynamically provision resources, minimizing over-provisioning. The smaller cost reduction on IBM Cloud is attributable to limitations like higher baseline resource requirements and the absence of features such as spot instance optimization.

$RQ_3$
: How efficiently do Karpenter and traditional Kubernetes autoscaling mechanisms manage CPU and memory resources under varying workload conditions on IBM Cloud?

Karpenter consistently outperformed traditional Kubernetes autoscaling mechanisms in resource utilization, with AWS Karpenter achieving superior CPU and memory efficiency (see [@tbl:summary] and [@tbl:cpuhomo]). On IBM Cloud, Karpenter reduced memory usage and node count, indicating better resource optimization (see [@tbl:memoryhomo] and [@tbl:nodehomo]). However, the efficiency gains were smaller compared to AWS, reflecting platform constraints such as slower instance provisioning and limited dynamic scaling features.

In conclusion, this study confirms Karpenter's advantages over CAS in scaling speed, cost efficiency, and resource utilization. However, the platform-specific constraints of IBM Cloud limit Karpenter's ability to fully exploit its dynamic provisioning capabilities, emphasizing the critical role of underlying infrastructure in autoscaler performance.

## Contributions to the Field

This research addresses a gap in Kubernetes autoscaling by providing empirical data that validate Karpenter’s advantages over CAS in latency, node count, and cost efficiency. It highlights Karpenter’s dynamic provisioning capabilities, which significantly improve performance and reduce costs, making it well-suited for dynamic workloads like e-commerce [@Liu2022Coordinating] and machine learning [@Wajahat2017MLscale].

The findings also emphasize the importance of cloud platform architecture in autoscaling effectiveness. While Karpenter excelled on AWS, its performance on IBM Cloud was constrained by platform-specific limitations, underscoring the need for better infrastructure support for advanced autoscaling features.

By evaluating Karpenter in IBM Cloud environments, this study fills a gap in the literature and provides actionable insights for improving both cloud platforms and autoscaling tools. The results lay a foundation for optimizing scaling practices and advancing cloud infrastructure to support more efficient, cost-effective solutions.

## Limitations

Several limitations in the study should be acknowledged. First, the experimental setup involved a relatively short timeline and ultimately compared single run for each test scenario. While the data collected is robust, rerunning experiments multiple times over a longer period would increase the sample size and improve the reliability of the findings by capturing variability and outliers in system performance.

Second, the experimental setup involved fairly modest cluster sizes, both in terms of node count and node specifications. While this approach helped control costs, and reflected an average production cluster, it may not fully represent the complexities of enterprise environments, which often involve larger clusters. Larger nodes and higher node counts could alter scaling dynamics, resource utilization, and cost metrics, providing a more accurate depiction of real-world enterprise settings.

Third, the workloads used in the experiments, though representative, were simulated and may not fully capture the complexity and unpredictability of real-world production environments. Real-world workloads often involve nuanced scaling requirements, diverse application profiles, and unexpected surges or lulls, which were only partially modelled in this study.

Additionally, the metrics used focused on economic and resource efficiency, but the environmental impact of improved efficiency was not directly quantified. While more efficient autoscaling reduces energy consumption, a dedicated analysis of energy savings and associated carbon emission reductions would provide a clearer picture of Karpenter's environmental benefits.

Finally, the IBM Cloud implementation of Karpenter was evaluated in its current state, which lacks several advanced features available on other platforms. This limitation may have constrained the observed benefits of Karpenter in IBM Cloud environments and does not reflect its full potential when integrated with a more feature-complete implementation.

## Recommendations for Future Research

To better understand the implications of the findings, future research could explore Karpenter’s long-term cost efficiency in dynamic, real-world environments where workload demands vary unpredictably. Such studies would provide deeper insights into how Karpenter performs under sustained and irregular usage patterns, particularly for organizations requiring high elasticity. Implementing more advanced automation frameworks for autoscaling tests could also enhance reproducibility and precision, enabling researchers to systematically assess scenarios involving complex workload variations. Additionally, future experiments might examine the tipping point between homogeneous and heterogeneous workloads to identify conditions where Karpenter or CAS is the most effective solution. Another promising addition for exploration is quantifying the environmental impact of Karpenter’s improved efficiency, particularly in reducing energy consumption and carbon emissions in cloud environments.

Further research is needed to address the gaps in the Karpenter Provider for IBM Cloud. The current implementation lacks certain features, including support for spot instances, price optimization strategies, and advanced scaling capabilities such as predictive scaling and batch-aware scaling. Investigating the integration of these functionalities would help realize Karpenter’s full potential in IBM Cloud environments.

## Conclusion

Given the growing importance of optimizing cloud infrastructure scaling, this study evaluated the performance of Karpenter compared to the traditional Kubernetes Cluster Autoscaler, particularly within IBM Cloud environments. The findings demonstrated that Karpenter significantly outperformed CAS in key areas, including node provisioning efficiency, scaling speed, and cost reduction. However, platform-specific constraints within IBM Cloud, such as slower instance provisioning and limited optimization features, restricted Karpenter's full potential. This is evidenced by a larger delta of key metrics in the AWS control benchmarks.

Overall, Karpenter proved to be a more dynamic and efficient autoscaling solution, offering measurable improvements in resource utilization and cost savings. These results underscore the necessity for further platform-specific enhancements and provide a foundation for future research into advanced autoscaling mechanisms and sustainable cloud infrastructure practices.

\newpage

# Glossary {-}

ANOVA
: Analysis of variance is a broad-based methodology for analysing data from an experiment in which the dependent or response variables are considered functions of the independent or ancillary variables. Each independent variable is tested (controlled) at specified values (levels) of that variable [@statsanal].

AWS
: Amazon Web Services is a cloud computing platform provided by Amazon [@aws_gloss].

CAS
: Kubernetes Cluster Autoscaler [@rosso2021production].

Container
: A lightweight, standalone, and executable software package that includes everything needed to run a piece of software [@ramirez2023containers], such as the code, runtime, libraries, and system tools [@walsh2022podman]. Containers are isolated from each other and the host system, but share the host's kernel [@rice2020containersecurity].

Container Runtime
: The software responsible for running containers. It manages container lifecycle operations like starting, stopping, and deleting containers, and interfaces with the underlying operating system [@schenker2019containerization].

Controller
: A control loop that watches the state of a Kubernetes cluster and makes changes to move the current state closer to the desired state [@controller].

Control Plane
: The set of components in a Kubernetes cluster that manages the cluster's overall state and controls how, when, and where workloads run. The control plane makes global decisions (e.g., scheduling) and provides an interface for managing Kubernetes objects [@ctlplane].

Cordon
: Preventing new pods from scheduling on a Node.

Drain
: Migrating existing pods away from a Node.

CRD
: A CustomResourceDefinition in Kubernetes is a way to define custom resources that extend the Kubernetes API without needing to implement a new API server. CRDs are often used in conjunction with custom controllers to create a declarative API for managing custom applications or services [@crd].

EKS
: Amazon Elastic Kubernetes Service is a managed Kubernetes service to run Kubernetes in the AWS cloud and on-premises datacenters [@eks_def].

HPA
: Horizontal Pod Autoscaler

IKS
: IBM Kubernetes Service is a managed container orchestration platform that automates the deployment, operation, and scaling of Kubernetes clusters on IBM Cloud [@iks_def].

Kubelet
: An agent that runs on each node in a Kubernetes cluster. It ensures that containers are running in pods as defined by the pod specs. The kubelet interacts with the container runtime to start, stop, and manage containers and continuously monitors the state of the node, reporting back to the control plane [@kubelet].

Kubernetes
: An open source system for automating deployment, scaling, and management of containerized applications [@k8s].

Kube Proxy
: A network component that runs on each node in a Kubernetes cluster. It manages network rules and load-balancing to ensure that traffic is properly routed between services and pods [@kubeproxy].

Node
: A Node in Kubernetes is a physical or virtual machine that runs containers and is managed by the control plane. Each node includes services like the kubelet, a container runtime, and kube-proxy [@kubeproxy]. Nodes provide the compute resources (CPU, memory, storage) needed to run Pods [@no].

Pending Pod
: A pending Pod in Kubernetes is one that hasn’t been assigned to a node yet, usually because there aren't enough available resources or it’s blocked by scheduling constraints (like affinity rules, node taints, or anti-affinity rules).

Pod
: A Pod in Kubernetes is the smallest deployable unit that can be created and managed. It represents a group of one or more containers that share network, storage, and a specification for how they should run. All containers in a Pod are co-located on the same host and share the same network namespace [@po].

Pod Lifecycle
: Pods in Kubernetes go through a lifecycle that includes phases like pending, running, succeeded, failed, and unknown. These phases reflect the high-level status of the pod [@polife].

UUID
: A Universally Unique Identifier is a 128-bit label used for information in computer systems [@uuid].

Unschedulable Pod
: An "unschedulable" Pod is one that can't be placed on any node due to constraints like resource limits, node taints, or affinity rules.

VPC
: A Virtual Private Cloud is a public cloud offering that lets an enterprise establish its own private cloud-like computing environment on shared public cloud infrastructure [@Ibm2024Aug].

\newpage

# Appendicies {-}

## Artefact Directory {-}

- Code for Experiment Automation
- Code for Karpenter IBM Cloud Provider
- Jupyter Notebook for Data Analysis
- Raw Experiments Data

## Figures {-}

![Comparision of Node Latency across scenarios on AWS.](aws_node_latency_comparison.png){width="620px" height="280px" #fig:awsnodelatency}

![Comparison of Total Memory Utilization across scenarios on AWS.](aws_memory_utilization_absolute_comparison.png){width="620px" height="280px" #fig:awsmem}

![Comparison of CPU Utilization across scenarios on AWS.](aws_cpu_utilization_comparison.png){width="620px" height="280px" #fig:awscpu}

![Comparison of CPU Utilization across scenarios on IBM Cloud.](ibm_cpu_utilization_comparison.png){width="620px" height="280px" #fig:ibmcpu}

\newpage

## Tables {-}

: CPU Utilization Statistics (Homogeneous). {#tbl:cpuhomo}

| **Platform** | **Provider** | **Mean (Cores)** | **Mode (Cores)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ---------------- | ---------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 4.82             | 0.12             | 4.52        | 2.62e-08           | 2.20e-08          |
| AWS          | Karpenter    | 5.02             | 0.08             | 4.82        | 1.64e-07           | 1.69e-07          |
| IBM          | CAS          | 3.72             | 0.19             | 4.28        | 0.007              | 0.0069            |
| IBM          | Karpenter    | 3.62             | 0.16             | 4.12        | 4.54e-05           | 3.86e-05          |

: CPU Utilization Statistics (Heterogeneous). {#tbl:cpuhetero}

| **Platform** | **Provider** | **Mean (Cores)** | **Mode (Cores)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ---------------- | ---------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 3.91             | 0.13             | 3.63        | 1.55e-07           | 1.61e-07          |
| AWS          | Karpenter    | 2.92             | 0.10             | 2.72        | 1.89e-07           | 1.97e-07          |
| IBM          | CAS          | 2.74             | 0.18             | 3.62        | 0.0253             | 0.0236            |
| IBM          | Karpenter    | 2.33             | 0.16             | 3.18        | 0.0149             | 0.0131            |

: Memory Utilization Statistics (Homogeneous). {#tbl:memoryhomo}

| **Platform** | **Provider** | **Mean (MB)** | **Mode (MB)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ------------- | ------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 4176.86       | 1530.11       | 1676.06     | 4.75e-06           | 3.39e-06          |
| AWS          | Karpenter    | 4332.18       | 2323.93       | 1747.90     | 3.38e-08           | 3.06e-08          |
| IBM          | CAS          | 7568.99       | 3433.59       | 2867.59     | 0.7007             | 0.6966            |
| IBM          | Karpenter    | 6513.64       | 3482.69       | 2376.90     | 0.238              | 0.2243            |

: Memory Utilization Statistics (Heterogeneous). {#tbl:memoryhetero}

| **Platform** | **Provider** | **Mean (MB)** | **Mode (MB)** | **Std Dev** | **t-test p-value** | **ANOVA p-value** |
| ------------ | ------------ | ------------- | ------------- | ----------- | ------------------ | ----------------- |
| AWS          | CAS          | 2834.39       | 1522.28       | 1043.34     | 0.0001             | 0.0001            |
| AWS          | Karpenter    | 2486.21       | 1279.91       | 914.20      | 9.40e-05           | 9.18e-05          |
| IBM          | CAS          | 6510.66       | 3444.23       | 1941.74     | 0.0585             | 0.0522            |
| IBM          | Karpenter    | 4686.19       | 1814.58       | 2002.39     | 0.9028             | 0.9023            |

: Comparison of Providers per Scenario (Homogeneous). {#tbl:comphomo}

| **Platform** | **Metric**   | **CAS** | **Karpenter** |
| ------------ | ------------ | ------- | ------------- |
| AWS          | Latency (ms) | 21.16   | 11.76         |
| IBM          |              | 18.18   | 17.07         |
| AWS          | CPU (cores)  | 4.82    | 5.02          |
| IBM          |              | 3.72    | 3.62          |
| AWS          | Memory (MB)  | 4176.86 | 4332.18       |
| IBM          |              | 7568.99 | 6513.64       |
| AWS          | Node Count   | 5.37    | 3.61          |
| IBM          |              | 6.44    | 5.10          |

: Comparison of Providers per Scenario (Heterogeneous). {#tbl:comphetero}

| **Platform** | **Metric**   | **CAS** | **Karpenter** |
| ------------ | ------------ | ------- | ------------- |
| AWS          | Latency (ms) | 15.67   | 13.93         |
| IBM          |              | 16.04   | 13.31         |
| AWS          | CPU (cores)  | 3.91    | 2.92          |
| IBM          |              | 2.74    | 2.33          |
| AWS          | Memory (MB)  | 2834.39 | 2486.21       |
| IBM          |              | 6510.66 | 4686.19       |
| AWS          | Node Count   | 4.62    | 3.51          |
| IBM          |              | 6.61    | 5.00          |

\newpage

# References
