Network Design

Network Design

Architecture Roadmap - Frame 9.jpg

In this project we differentiate between 3 distinct network types, which are in alignment with the Phoenix common practices:

Type

IP-Ranges

Routing

Administration

Usecase

Examples

Type

IP-Ranges

Routing

Administration

Usecase

Examples

Red Network

Public IP Addresses

Yes

By Phoenix

  • Direct Access to Internet services.

  • Offering Services to the outside without Proxying

public provider Network to offer external services

Yellow Network

Private IP Addresses

Phoenix Internal. Outside gets NATed

By Phoenix

  • Standard traffic for internal services

  •  

 

Green Network

Private IP Addresses

No

Project Specific

  • Project internal Traffic.

  • Isolated Networks without connectivity

  • Storage Replication Network

 

Version 2

Logical Design

Dedicated Networks

There will be multiple different networks defined, which are transported in the Backbone via VLANs

Network name

Identifier

Used By

Classification

needs external routing

Description

 

Network name

Identifier

Used By

Classification

needs external routing

Description

 

OOB

OOB

Cluster/Admin/Jumphost

Yellow

Yes

Out of Band Management of the cluster, only used to connect “from the Internet” to the jumphost .

 

PXE

PXE

Jumphost/OCP/OSP

Green

 

used for bootstrapping new nodes in the network. Used by

  • the OCP installer from the jumphost to bootstrap the OCP Master nodes

  • the OCP cluster to bootstrap OCP Worker nodes

  • the RHOSO Operator to bootstrap OSP Compute nodes

 

OCP

OCP

OCP

Yellow

Yes

OpenShift main network

 

OCP Multus

MULTUS

OCP

Green

 

 

 

Storage Access

STORAGE

ODF/OCP/OSP

Yellow

Yes

Storage Access Network for Consumers of the ODF Ceph Storage

 

Storage Replication

STOREP

ODF

Green

 

Storage Replication Network for syncronization tasks of the ODF Storage

 

OCP-V LiveMigration

OCPVLIVE

OCP

Green

 

Network for OCP-Virt Live Migration

 

OSP Controlplane

OS-CTLPLANE

OSP/OCP

Yellow

Yes

Controlplane and Management of OpenStack

 

OSP IntApi

OS-INTAPI

OSP

Green

 

Internal API Network of OpenStack

 

OSP Tenant

OS-TENANT

OSP

Green

 

Network for OpenStack Tenant Network encapsulation

 

Provider Network Ext

PROVEXT

OCP/OSP

Red

Yes

External Provider Network, for Workloads that are directly connected “to the internet”

 

Provider Network Int

PROVINT

OCP/OSP

Yellow

Yes

Internal Provider Network, for workloads that are not exposed “to the internet”

 

 

Physical Design

Bonding

  • All nodes are connected to the network switches via port channel/bonding 802.3ad.

  • All network interfaces of the systems are added to the bond, with a variable to change the Priority to specific interfaces over others.

  • Bonding is negotiated via LACP (Link Aggregation Control Protocol) between the servers and switches, with fallback to normal “access” port for deployment in case it doens’t receive LACP packets.

VLANs

  • The networks defined above are transported to the servers within the bond via VLAN tagging.

  • Sub-interfaces are created for each VLAN.

  • The only exception is the PXE network and a respective deployment VLAN, which is transported untagged (so it will be directly in the interface bond0) in Bootstrap mode (see below)

  • Switches will provide the tagged VLANs on all servers. The servers will select the VLANs they need to use.

  • The untagged fallback VLAN which is provided when no LACP can be established, depends on the role of the node. (see table below)

 

Hardware

  • Two different hardware types are planned—the design can be easily adapted to other hardware types:

    • Lenovo Servers (4x1GB): All four ports are configured in the bond.

    • Dell Servers (2x100GB): Both ports are configured in the bond.

Warning

Note: Bootstrap Mode

In Bootstrap Mode the Network Port-Configuration on the Switches/Routers may be different:

  • The first Port of the host will not be setup in the Portchannel/Bond but as normal Network port with an untagged PXE Vlan

  • All other available Ports of the host will still be configured for Bonding with the required VLANs in the Portchannel and the selected Fallback VLAN

After the Bootstrap of the node, and a verified deployment, the first port will added to the Bond/Portchannel to enhance reliability and performance

VLAN Assignments to hosts by role

Role

tagged VLANs in Portchannel

Bonding Fallback-VLAN

remarks

Role

tagged VLANs in Portchannel

Bonding Fallback-VLAN

remarks

 

for OCP

for Storage(legacy)

for OCP-Virt

 

 

OCP Master

  • OCP

  • MULTUS

  • STORAGE

 

OCP

 

OCP Worker

  • OCP

  • MULTUS

  • STORAGE

  • PROVEXT

  • PROVINT

  • STORAGE

  • OCPVLIVE

OCP

 

OCP Storage

  • OCP

  • MULTUS

  • STORAGE

  • PROVINT

  • STORAGE

  • STOREP

 

OCP

 

OS Compute

  • OCP

  • MULTUS

 

 

OCP

 

The network switches should provide all the mentioned vlans on the portchannels. It is the task of the host to define and access the VLANs that are needed by the role.

 

[WiP] Communication Matrix and Firewalling

The following requirements for Communication are needed:

Jumphost:

  • SSH Access from Internet to SSH-Management port on external Network (via Port-Forwarding/NAT is fine

  • Full Access to Internet via external interface

  • OCP Hosts BMC via IPMI/RedFish

  • OCP Hosts BMC must access Jumphost via HTTP(/S)

  •  

Related content