////
Purpose
-------
This page gives an introduction to OpenShift Container Platform installation and update process
////
= Installation and update

== Installation overview
The OpenShift Container Platform installation program offers {customer} flexibility. {customer} can use the installation program to deploy a cluster on infrastructure that the installation program provisions and the cluster maintains or deploy a cluster on infrastructure that {customer} prepares and maintains.

These two basic types of OpenShift Container Platform clusters are frequently called installer-provisioned infrastructure clusters and user-provisioned infrastructure clusters.

Both types of clusters have the following characteristics:

* Highly available infrastructure with no single points of failure is available by default

* Administrators maintain control over what updates are applied and when

{customer} uses the same installation program to deploy both types of clusters. The main assets generated by the installation program are the Ignition config files for the bootstrap, master, and worker machines. With these three configurations and correctly configured infrastructure, {customer} can start an OpenShift Container Platform cluster.

The OpenShift Container Platform installation program uses a set of targets and dependencies to manage cluster installation. The installation program has a set of targets that it must achieve, and each target has a set of dependencies. Because each target is only concerned with its own dependencies, the installation program can act to achieve multiple targets in parallel. The ultimate target is a running cluster. By meeting dependencies instead of running commands, the installation program is able to recognize and use existing components instead of running the commands to create them again.

The following diagram shows a subset of the installation targets and dependencies:

.OpenShift Container Platform installation targets and dependencies
image::OCP-4x-base/installation-overview.png[]

After installation, each cluster machine uses Red Hat Enterprise Linux CoreOS (RHCOS) as the operating system. RHCOS is the immutable container host version of Red Hat Enterprise Linux (RHEL) and features a RHEL kernel with SELinux enabled by default. It includes the `kubelet`, which is the Kubernetes node agent, and the CRI-O container runtime, which is optimized for Kubernetes.

Every control plane machine in an OpenShift Container Platform {ocp_version} cluster must use RHCOS, which includes a critical first-boot provisioning tool called Ignition. This tool enables the cluster to configure the machines. Operating system updates are delivered as an Atomic OSTree repository that is embedded in a container image that is rolled out across the cluster by an Operator. Actual operating system changes are made in-place on each machine as an atomic operation by using rpm-ostree. Together, these technologies enable OpenShift Container Platform to manage the operating system like it manages any other application on the cluster, via in-place upgrades that keep the entire platform up-to-date. These in-place updates can reduce the burden on operations teams.

If {customer} uses RHCOS as the operating system for all cluster machines, the cluster manages all aspects of its components and machines, including the operating system. Because of this, only the installation program and the Machine Config Operator can change machines. The installation program uses Ignition config files to set the exact state of each machine, and the Machine Config Operator completes more changes to the machines, such as the application of new certificates or keys, after installation.

=== Supported platforms for OpenShift Container Platform clusters

In OpenShift Container Platform {ocp_version}, {customer} can install a cluster that uses installer-provisioned infrastructure on the following platforms:

* Amazon Web Services (AWS)

* Google Cloud Platform (GCP)

* Microsoft Azure

* Red Hat OpenStack Platform (RHOSP) version 13 and 16

** The latest OpenShift Container Platform release supports both the latest RHOSP long-life release and intermediate release. For complete RHOSP release compatibility, see the link:https://access.redhat.com/articles/4679401[OpenShift Container Platform on RHOSP support matrix].

* Red Hat Virtualization (RHV)

* VMware vSphere

* VMware Cloud (VMC) on AWS

* Bare metal

For these clusters, all machines, including the computer that {customer} runs the installation process on, must have direct internet access to pull images for platform containers and provide telemetry data to Red Hat.

[IMPORTANT]
====
After installation, the following changes are not supported:

* Mixing cloud provider platforms
* Mixing cloud provider components, such as using a persistent storage framework from a differing platform than what the cluster is installed on
====

In OpenShift Container Platform {ocp_version}, {customer} can install a cluster that uses user-provisioned infrastructure on the following platforms:

* AWS

* Azure

* GCP

* RHOSP

* RHV

* VMware vSphere

* VMware Cloud on AWS

* Bare metal

* IBM Z or LinuxONE

* IBM Power Systems

With installations on user-provisioned infrastructure, each machine can have full internet access, {customer} can place its cluster behind a proxy, or {customer} can perform a _restricted network installation_. In a restricted network installation, {customer} can download the images that are required to install a cluster, place them in a mirror registry, and use that data to install its cluster. While {customer} requires internet access to pull images for platform containers, with a restricted network installation on vSphere or bare metal infrastructure, its cluster machines do not require direct internet access.

The link:https://access.redhat.com/articles/4128421[OpenShift Container Platform 4.x Tested Integrations] page contains details about integration testing for different platforms.

=== Installation process

When {customer} installs an OpenShift Container Platform cluster, {customer} downloads the installation program from the appropriate link:https://console.redhat.com/openshift/install[Infrastructure Provider] page on the Red Hat OpenShift Cluster Manager site. This site manages:

* REST API for accounts

* Registry tokens, which are the pull secrets that {customer} uses to obtain the required components

* Cluster registration, which associates the cluster identity to its Red Hat account to facilitate the gathering of usage metrics

In OpenShift Container Platform {ocp_version}, the installation program is a Go binary file that performs a series of file transformations on a set of assets. The way {customer} interacts with the installation program differs depending on its installation type.

* For clusters with installer-provisioned infrastructure, {customer} delegates the infrastructure bootstrapping and provisioning to the installation program instead of doing it itself. The installation program creates all of the networking, machines, and operating systems that are required to support the cluster.

* If {customer} provisions and manages the infrastructure for its cluster, {customer} must provide all of the cluster infrastructure and resources, including the bootstrap machine, networking, load balancing, storage, and individual cluster machines.

{customer} uses three sets of files during installation: an installation configuration file that is named install-config.yaml, Kubernetes manifests, and Ignition config files for its machine types.

IMPORTANT: It is possible to modify Kubernetes and the Ignition config files that control the underlying RHCOS operating system during installation. However, no validation is available to confirm the suitability of any modifications that {customer} make to these objects. If {customer} modifies these objects, {customer} might render its cluster non-functional. Because of this risk, modifying Kubernetes and Ignition config files is not supported unless {customer} are following documented procedures or are instructed to do so by Red Hat support.

The installation configuration file is transformed into Kubernetes manifests, and then the manifests are wrapped into Ignition config files. The installation program uses these Ignition config files to create the cluster.

The installation configuration files are all pruned when {customer} runs the installation program, so be sure to back up all configuration files that {customer} wants to use again.

IMPORTANT: {customer} cannot modify the parameters that {customer} set during installation, but {customer} can modify many cluster attributes after installation.

==== The installation process with installer-provisioned infrastructure

The default installation type uses installer-provisioned infrastructure. By default, the installation program acts as an installation wizard, prompting {customer} for values that it cannot determine on its own and providing reasonable default values for the remaining parameters. {customer} can also customize the installation process to support advanced infrastructure scenarios. The installation program provisions the underlying infrastructure for the cluster.

{customer} can install either a standard cluster or a customized cluster. With a standard cluster, {customer} provides minimum details that are required to install the cluster. With a customized cluster, {customer} can specify more details about the platform, such as the number of machines that the control plane uses, the type of virtual machine that the cluster deploys, or the CIDR range for the Kubernetes service network.

If possible, use this feature to avoid having to provision and maintain the cluster infrastructure. In all other environments, {customer} uses the installation program to generate the assets that {customer} requires to provision its cluster infrastructure.

With installer-provisioned infrastructure clusters, OpenShift Container Platform manages all aspects of the cluster, including the operating system itself. Each machine boots with a configuration that references resources hosted in the cluster that it joins. This configuration allows the cluster to manage itself as updates are applied.

==== The installation process with user-provisioned infrastructure

{customer} can also install OpenShift Container Platform on infrastructure that {customer} provides. {customer} uses the installation program to generate the assets that {customer} requires to provision the cluster infrastructure, create the cluster infrastructure, and then deploy the cluster to the infrastructure that {customer} provided.

If {customer} do not use infrastructure that the installation program provisioned, {customer} must manage and maintain the cluster resources itself, including:

* The underlying infrastructure for the control plane and compute machines that make up the cluster

* Load balancers

* Cluster networking, including the DNS records and required subnets

* Storage for the cluster infrastructure and applications

If its cluster uses user-provisioned infrastructure, {customer} has the option of adding RHEL compute machines to its cluster.

==== Installation process details

Because each machine in the cluster requires information about the cluster when it is provisioned, OpenShift Container Platform uses a temporary bootstrap machine during initial configuration to provide the required information to the permanent control plane. It boots by using an Ignition config file that describes how to create the cluster. The bootstrap machine creates the control plane machines (also known as the master machines) that make up the control plane. The control plane machines then create the compute machines, which are also known as worker machines. The following figure illustrates this process:

.Creating the bootstrap, control plane, and compute machines
image::OCP-4x-base/installation-process.png[]

Bootstrapping a cluster involves the following steps:

. The bootstrap machine boots and starts hosting the remote resources required for the control plane machines to boot. (Requires manual intervention if {customer} provisions the infrastructure)

. The bootstrap machine starts a single-node etcd cluster and a temporary Kubernetes control plane.

. The control plane machines fetch the remote resources from the bootstrap machine and finish booting. (Requires manual intervention if {customer} provisions the infrastructure)

. The temporary control plane schedules the production control plane to the production control plane machines.

. The Cluster Version Operator (CVO) comes online and installs the etcd Operator. The etcd Operator scales up etcd on all control plane nodes.

. The temporary control plane shuts down and passes control to the production control plane.

. The bootstrap machine injects OpenShift Container Platform components into the production control plane.

. The installation program shuts down the bootstrap machine. (Requires manual intervention if {customer} provisions the infrastructure)

. The control plane sets up the compute nodes.

. The control plane installs additional services in the form of a set of Operators.

. The result of this bootstrapping process is a fully running OpenShift Container Platform cluster. The cluster then downloads and configures remaining components needed for the day-to-day operation, including the creation of compute machines in supported environments.

==== Installation scope

The scope of the OpenShift Container Platform installation program is intentionally narrow. It is designed for simplicity and ensured success. {customer} can complete many more configuration tasks after installation completes.

== Update overview

The OpenShift Update Service (OSUS) provides over-the-air updates to OpenShift Container Platform, including Red Hat Enterprise Linux CoreOS (RHCOS). It provides a graph, or diagram, that contains the vertices of component Operators and the edges that connect them. The edges in the graph show which versions {customer} can safely update to. The vertices are update payloads that specify the intended state of the managed cluster components.

The Cluster Version Operator (CVO) in its cluster checks with the OpenShift Update Service to see the valid updates and update paths based on current component versions and information in the graph. When {customer} requests an update, the CVO uses the release image for that update to upgrade its cluster. The release artifacts are hosted in Quay as container images.

To allow the OpenShift Update Service to provide only compatible updates, a release verification pipeline drives automation. Each release artifact is verified for compatibility with supported cloud platforms and system architectures, as well as other component packages. After the pipeline confirms the suitability of a release, the OpenShift Update Service notifies {customer} that it is available.

IMPORTANT: The OpenShift Update Service displays all valid updates. Do not force an update to a version that the OpenShift Update Service does not display.

Two controllers run during continuous update mode. The first controller continuously updates the payload manifests, applies the manifests to the cluster, and outputs the controlled rollout status of the Operators to indicate whether they are available, upgrading, or failed. The second controller polls the OpenShift Update Service to determine if updates are available.

IMPORTANT: Only upgrading to a newer version is supported. Reverting or rolling back its cluster to a previous version is not supported. If its upgrade fails, contact Red Hat support.

During the upgrade process, the Machine Config Operator (MCO) applies the new configuration to its cluster machines. The MCO cordons the number of nodes as specified by the `maxUnavailable` field on the machine configuration pool and marks them as unavailable. By default, this value is set to `1`. The MCO then applies the new configuration and reboots the machine.

If {customer} uses Red Hat Enterprise Linux (RHEL) machines as workers, the MCO does not update the kubelet because {customer} must update the OpenShift API on the machines first.

With the specification for the new version applied to the old kubelet, the RHEL machine cannot return to the `Ready` state. {customer} cannot complete the update until the machines are available. However, the maximum number of unavailable nodes is set to ensure that normal cluster operations can continue with that number of machines out of service.

The OpenShift Update Service is composed of an Operator and one or more application instances.
	
NOTE: During the upgrade process, nodes in the cluster might become temporarily unavailable. The `MachineHealthCheck` might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, remove any `MachineHealthCheck` resource that {customer} has deployed before updating the cluster. However, a `MachineHealthCheck` resource that is deployed by default (such as `machine-api-termination-handler`) cannot be removed and will be recreated.
