////
Purpose
-------
Capture information about how to use the deployed and related systems.

Recommended to primarily be workflow diagrams, possible with labeled and annotated steps.

Examples
--------
* Continues Integration and Continues Deployment/Delivery processes
* DeclarativeOps / GitOps processes
* Update/Upgrade processes
* Self service processes,
  - EX: create a CR for a custom operator to create a new namespace and attach it to the correct team RBAC.

////
[id="processes_{context}"]
= Processes

== Cluster Lifecycle

=== Installation
New clusters are installed using the ROSA cli and bootstrap scripts.

=== Upgrades
The HCP team performs two minor version (e.g. 4.14.x => 4.16.y) upgrades per year with weekly patches. Weekly pathes all on he same day to ensure all stages are on the same patch version.

== Secure Software Development Lifecycle
Developers and the HCP team operate on the basis of a shared responsibility model where the platform and certain SSDLC enhancing services are provided by the HCP team but the ultimate responsibility for operating applications lies with the developers themselves. 

The core tools used are GitHub Enterprise for version control and CI/CD pipelines, Nexus for artifact management, and ACS for vulnerability scanning. In addition, in house tools such as lifecycle and the chargeback tool are used.

=== Ordering a Namespace

To order a namespace, users need a LeanIX number. LeanIX is an asset inventory tracking system akin to a configuration management database [CMDB]. The LeanIX number is entered into a ServiceNow ticket along with the target stage (non-prod or prod), and a name. Naming of namespaces follows a `<country>-<name>-<stage>` convention.

image::order-ns.png[Process of ordering a namespace visualised. Adapted from workshop notes.] 

The namespcae is created according to a project template. Upon creation, RBAC role bindings are created and an in-house operator creates a GitOps repository on GitHub. The namespace creation itself is not yet gitopsed. To begin using the GitOps repository, developers need to copy paste a secret value from OpenShift to GitHub. 

=== Increasing Resource Quotas

Up to a certain limit, developers are able to increase their resource quota in a self-service fashion. 

=== Releasing and deploying Software

image::release.png[Process of releasing, and deploying software to a cluster visualised. Adapted from workshop notes.]

=== Ordering managed services

ServiceNow => manual using terraform
(potential for templating with developer hub maybe)

=== Vulnerability Management

The HCP team's vulnerability management approach focuses on identifying and mitigating Common Vulnerabilities and Exposures [CVEs] at runtime. ACS and Gatekeeper are leveraged for policy enforcement, along with RBAC admission controls to limit access and reduce the attack surface. ACS is configured with a default policy set, augmented by a few custom policies tailored to specific operational needs. However, the setup currently lacks enforced security quality gates, meaning that identified vulnerabilities don’t halt deployments but are instead flagged for remediation. File integrity and compliance operators have been decommissioned, as Node level security and compliance is deemed a part of the ROSA managed service offering.

=== Debugging

pod logs, tracing, OCP + splunk

Support is managed through ServiceNow tickets, where engineers on call respond to issues during business hours.

=== End-User Training

The HCP team has created training courses on the internal IT academy learning platform that focus on {cust} specific processes and technologies. The courses are adapted to different levels of know-how and range from read + click, to screen casts, and homework project-based trainings.

=== Conculsions 

Overall, the processes owned by the HCP team seem to be designed to make the developer experience as frictionless as possible. Based on the imformation shared, the time required for developers to go from zero to production seems significantly shorter than the average when compared to other Red Hat customers.

The targeted, on-demand training offering is especially impressive and far exceeds what average customers offer their end-users.
